Getting FILE * to a string in memory

This is a discussion on Getting FILE * to a string in memory within the C Programming forums, part of the General Programming Boards category; hi I have a code library that was written to read from a file stream (FILE *) doing mostly getc ...

  1. #1
    Registered User
    Join Date
    Feb 2008
    Posts
    6

    Angry Getting FILE * to a string in memory

    hi

    I have a code library that was written to read from a file stream (FILE *) doing mostly getc ungetc, and processing that data. However I actually need to run against a string already in memory. So I can convert all the code to work over my string array, but what I'd like is somehow get a FILE * to my string.

    I don't see any way to do this though, and this code does need to be fast so I'd like to avoid anything that requires writing out the data to disk to get an fd. If I have to I will just convert it all to work on an array vs getc ungetc but there's a fair amount of code to convert.

    Any suggestions? (btw this is linux platform)

    thanks

  2. #2
    uint64_t...think positive xuftugulus's Avatar
    Join Date
    Feb 2008
    Location
    Pacem
    Posts
    355
    You should check out mmap, i am not sure i am just it reading it myself.
    If you are on linux, it is located in: sys/mman.h
    Code:
    ...
        goto johny_walker_red_label;
    johny_walker_blue_label: exit(-149$);
    johny_walker_red_label : exit( -22$);
    A typical example of ...cheap programming practices.

  3. #3
    Registered User
    Join Date
    Feb 2008
    Posts
    6
    thanks, I think mmap will require me to write it to disk first, then map that to mem which I'm trying to avoid, since I already have it in memory.

    Loran

  4. #4
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,248
    Sure, anything is possible:

    Code:
    #include <string.h>
    #include <stdlib.h>
    #include <stdio.h>
    
    int main()
    {
       int c;
       FILE *str_fp;
       char *str = "your string data";
       char *buff = malloc(strlen(str) +8);
       sprintf(buff, "echo '&#37;s'", str);
       str_fp = popen(buff, "r");
       while((c = fgetc(str_fp)) != EOF)
       {
          putchar(c);
       }
       pclose(str_fp);
       free(buff);
    }
    It has limitations -- you can't have (unescaped) single quote characters or null bytes in the string. It's more a fun toy than a real method.

    Personally, I'd rewrite your code to abstract out the I/O. Then you could just drop in different methods.

  5. #5
    uint64_t...think positive xuftugulus's Avatar
    Join Date
    Feb 2008
    Location
    Pacem
    Posts
    355
    Yes it was an interesting question and i am doing research on it. But the real solution which is to map the memory to a FILE *, i don't suppose it could be done by tinkering with the FILE * structure?
    I would probably redefine the used IO functions to some simple implementation, but still i am trying to find a way to abstract it out.
    And if efficiency is important, then using an external program is out of the question right?
    Code:
    ...
        goto johny_walker_red_label;
    johny_walker_blue_label: exit(-149$);
    johny_walker_red_label : exit( -22$);
    A typical example of ...cheap programming practices.

  6. #6
    Registered User
    Join Date
    Feb 2008
    Posts
    6
    Quote Originally Posted by brewbuck View Post
    Sure, anything is possible:

    It has limitations -- you can't have (unescaped) single quote characters or null bytes in the string. It's more a fun toy than a real method.

    Personally, I'd rewrite your code to abstract out the I/O. Then you could just drop in different methods.
    thanks, yeh that would be a bit much overhead, and this is some xml data that can be in many different charsets also.

    Yeh i may end up having to do that

  7. #7
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,248
    Quote Originally Posted by xuftugulus View Post
    Yes it was an interesting question and i am doing research on it. But the real solution which is to map the memory to a FILE *, i don't suppose it could be done by tinkering with the FILE * structure?
    Yes, it can be. This is how popen() does what it does. You'd basically be writing something akin to popen() but which gets data from a memory buffer instead. This is going to be extremely platform-specific, down to the C library and header files. In other words, not portable at all. But possible.

    My popen() example is at least a click more portable because it works everywhere popen() is available, which is a lot of places.

    And if efficiency is important, then using an external program is out of the question right?
    Like I said, my preferred choice would be rewriting it so it didn't directly use FILE* any more. But no, involving another program is not necessarily less efficient (although in this case it probably is).

  8. #8
    Registered User
    Join Date
    Feb 2008
    Posts
    6
    Quote Originally Posted by xuftugulus View Post
    Yes it was an interesting question and i am doing research on it. But the real solution which is to map the memory to a FILE *, i don't suppose it could be done by tinkering with the FILE * structure?
    I would probably redefine the used IO functions to some simple implementation, but still i am trying to find a way to abstract it out.
    And if efficiency is important, then using an external program is out of the question right?
    Thanks, yeh I agree mapping to FILE * is the ideal thing and I'm sure there's a way. Maybe I'll dig into its internals or see if there's anything in Steven's UP

  9. #9
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,248
    Quote Originally Posted by loran32 View Post
    Thanks, yeh I agree mapping to FILE * is the ideal thing and I'm sure there's a way. Maybe I'll dig into its internals or see if there's anything in Steven's UP
    Mapping it to FILE* is probably the worst way imaginable actually. All it takes is one small detail in the C library to change, and you get to rewrite your code again.

    Do it once, do it right, and forget about it.

  10. #10
    Registered User
    Join Date
    Feb 2008
    Posts
    6
    Quote Originally Posted by brewbuck View Post
    Mapping it to FILE* is probably the worst way imaginable actually. All it takes is one small detail in the C library to change, and you get to rewrite your code again.

    Do it once, do it right, and forget about it.
    thanks

    Yeh i don't mean doing something unsupported, I may just take a look at the underlying struct. Now that I think about it i may just try redefing getc and ungetc for that file

  11. #11
    uint64_t...think positive xuftugulus's Avatar
    Join Date
    Feb 2008
    Location
    Pacem
    Posts
    355
    I hacked a quick solution to the problem...

    Code:
    #include <stdio.h>
    
    typedef struct
    {
            FILE _file;
            char *data;
            size_t l;
    } MYFILE;
    
    #define MYfgetc(f) ( ((MYFILE *)f)->l--? *((MYFILE *)f)->data++ : EOF)
    
    #define fgetc MYfgetc
    
    char s[] = "Hello wrapped FILE!";
    
    int main()
    {
            MYFILE mf;
    
            mf.data = s;
            mf.l = strlen(s);
            iofunc((FILE *)&mf);
    
            return 0;
    }
    
    int iofunc(FILE *f)
    {
            int c;
    
            while( (c = fgetc(f)) != EOF)
            {
                    putchar(c);
            }
            return 0;
    }
    This way though you lose the original functionality of <stdio.h>. But you can always #undef the macros when the job is done. It also means that you will have to implement the rest of the <stdio.h> routines, or at least the most basic ones.
    Last edited by xuftugulus; 02-29-2008 at 02:39 PM.
    Code:
    ...
        goto johny_walker_red_label;
    johny_walker_blue_label: exit(-149$);
    johny_walker_red_label : exit( -22$);
    A typical example of ...cheap programming practices.

  12. #12
    Registered User
    Join Date
    Feb 2008
    Posts
    6
    Quote Originally Posted by xuftugulus View Post
    I hacked a quick solution to the problem...


    This way though you lose the original functionality of <stdio.h>. But you can always #undef the macros when the job is done. It also means that you will have to implement the rest of the <stdio.h> routines, or at least the most basic ones.

    hey thanks very much, that's great!

    really appreciate your help

    Loran

  13. #13
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,248
    Quote Originally Posted by xuftugulus View Post
    This way though you lose the original functionality of <stdio.h>. But you can always #undef the macros when the job is done. It also means that you will have to implement the rest of the <stdio.h> routines, or at least the most basic ones.
    This is slightly less hackish than actually munging around in the FILE structures. I applaud your effort

  14. #14
    uint64_t...think positive xuftugulus's Avatar
    Join Date
    Feb 2008
    Location
    Pacem
    Posts
    355
    It also is a nice way to implement 'inheritance' using plain C.
    Code:
    ...
        goto johny_walker_red_label;
    johny_walker_blue_label: exit(-149$);
    johny_walker_red_label : exit( -22$);
    A typical example of ...cheap programming practices.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. File transfer- the file sometimes not full transferred
    By shu_fei86 in forum C# Programming
    Replies: 13
    Last Post: 03-13-2009, 12:44 PM
  2. String issues
    By The_professor in forum C++ Programming
    Replies: 7
    Last Post: 06-12-2007, 09:11 AM
  3. Replies: 3
    Last Post: 03-04-2005, 01:46 PM
  4. Something is wrong with this menu...
    By DarkViper in forum Windows Programming
    Replies: 2
    Last Post: 12-14-2002, 10:06 PM
  5. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 04:10 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21