Intercept and redirect file requests

This is a discussion on Intercept and redirect file requests within the Windows Programming forums, part of the Platform Specific Boards category; Lets get right to the point. I have an application that access a file about 1gb in size. In order ...

  1. #1
    Registered User
    Join Date
    Jun 2010
    Posts
    6

    Intercept and redirect file requests

    Lets get right to the point.

    I have an application that access a file about 1gb in size. In order to shrink this application i need to shrink the 1gb file.
    Using Procmon i have determined the program only access about 40mb of the file for the features i use. This was verified by creating a dummy file with only the data it will access, the rest zeroed out.

    So the delima i am in is as follows, the program we will call application.exe:

    Application.exe accesses file large.bin from address 0x11F34A02 to 0x143CFEC5.
    File small.bin contains only the data accessed.
    I need to intercept the request for data from 0x11F34A02 at large.bin and redirect to 0x00000000 at small.bin.

    The only way i can think of to do this is by launching the program through an application i create and redirecting on the fly.
    Up until now, i have had very limited experience in calling external files, and no experience in intercepting data requests.

    Any help or direction here would be appreciated. I don't mind reading up on information (in fact i would prefer it) but i dont have a starting point. Hopefully someone can help!

  2. #2
    'Allo, 'Allo, Allo
    Join Date
    Apr 2008
    Posts
    611
    Google for Sparse Files

  3. #3
    Registered User
    Join Date
    Jun 2010
    Posts
    6
    yes that would work if my file resided on only windows partitions.
    But this file will be on ext2 and ntfs systems.

    That is if i am understanding what i read correctly.

    In any case, i beileve i could figure this out if i knew how to monitor a process in much the way procmon does. Any other suggestions?

  4. #4
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,535
    Or you could just go from forum to forum spamming the same question and not bothering to talk to anyone who responds.

    Intercept And Redirect File Requests
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  5. #5
    Registered User
    Join Date
    Jun 2010
    Posts
    6
    Oh i posted a reply to that thread. It never posted on the site, i dont know why. I moved onto a different site that i thought could help more (based on other threads).

    As as per your response on dreamincode, that simply opens a file and reads code, i can do that already. I need to access a program that is already running (or launch that program) and tell it to read from a different file. I do not have access to the source of the original program otherwise i would implement your code.

    I may have misread your prior post, if so i would love to know. Im not avoiding answers here, that would be foolish.

    Sorry for the misunderstanding. I also prefer a site with less ads, i can barely read the other without my browser full screen.
    Last edited by GamezR2EZ; 06-02-2010 at 02:59 PM. Reason: only supports bbcode, not html

  6. #6
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,535
    > But this file will be on ext2 and ntfs systems.
    Doing this on one OS would be hard enough, but on two - is it really worth it?
    Whatever technique you end up using for windows will have to be re-invented for Linux.

    It's gonna be weeks of work, even for someone who knows what they're doing.
    If you go digging deep into the respective kernels, the chances are you'll trash your disk at least once.

    If you just remove the end of the file, you would be reading the last 40MB of a 340MB file (less than half the original).
    Disk space is cheap enough, and is very easy to accomplish.

    > In order to shrink this application i need to shrink the 1gb file.
    What does this mean?
    Application size and file size are totally different.
    A copy program is a few KB, but it can read a GB file no problem. Plus it can choose to copy the file 1 byte at a time, or load the whole file (or large chunks of it) into memory.

    Likewise, the memory resident footprint of the program and it's data is usually less than the total allocated to the program (through the actions of the swapper).
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  7. #7
    Registered User
    Join Date
    Jun 2010
    Posts
    6
    > But this file will be on ext2 and ntfs systems.

    Actually, the file will be sitting on and ext2 partition accessed through windows, that part is out of my control as it is dual booting currently. The application will only be run in windows environments.

    > In order to shrink this application i need to shrink the 1gb file.

    Really what i meant was in terms of total application size, the total size of the information it uses is 57mb. I have striped out alot of the applications "useless" data. But with the main archive being upwards of a gig, you can see the wasted space. This being on an UMPC, with limited disk space, but more than capable of many many tasks, i want to consolidate. Also this method in my mind would be very useful on alot of fronts, now i just have a reason and time to explore it.

    The comment about taking weeks of work, i just dont see that. However, you seem to know more than me on this subject, so if you have an idea of how to do this, why not give me some functions to research? As i have stated i would prefer to learn this all by myself, but a pointer would be nice. I am not new to cpp in general, but here i am basically empty.

  8. #8
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,535
    > Actually, the file will be sitting on and ext2 partition accessed through windows
    I didn't think windows could ordinarily read an ext2 partition.

    I'm sure it could, with some reading of IFS Kit - Installable File System Kit

    But if someone has already done that (they have btw, search for it), then you should be able to get the source code for that (given that ext2 is GPL to begin with - IANAL).
    From that, creating the redirect should be fairly straight-forward.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  9. #9
    Registered User
    Join Date
    Jun 2010
    Posts
    6
    I must have missed something. How would it be straight forward?

    I dont have a problem reading ext2, that isnt the issue.

    I am aware of how ext2 and ntfs file systems work, but i dont see how that is going to help me.

    There are several ways i can do this, all without my desired results. Sparse files for one (thanks adeyblue), but that doesnt quite meet my needs.

    I would like to have a single file, only the size of what is accessed. Keep the original application untouched. Use something to redirect where the file is being accessed at. I am quite aware it is possible, just not how to do it. Do you know how to do that?

    In any case thanks for your help, and to anyone else who can help.

  10. #10
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,535
    Easy, as in "if file1 is open and offset is x, then read from offset y in file2 instead".

    As far as the low level file system is concerned, it's just reading from a different block list, that's all.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  11. #11
    Registered User
    Join Date
    Jun 2010
    Posts
    6
    Quote Originally Posted by Salem View Post
    Easy, as in "if file1 is open and offset is x, then read from offset y in file2 instead".
    That is what I had in mind yes, but I don't see what you are pointing me at. If you're suggesting the file remain 1gb, but really only take up 40mb based on a modified block list, not quite what i had in mind.

    What i had in mind was more like this. Although I am having a bit of trouble getting it to do what I want. Not sure if it even will.

Popular pages Recent additions subscribe to a feed

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21