Intialize An Array From A File?

This is a discussion on Intialize An Array From A File? within the C Programming forums, part of the General Programming Boards category; Hello, I'm wondering if there's a way to be able to initialize an array directly using a binary file, e.g. ...

  1. #1
    Registered /usr
    Join Date
    Aug 2001
    Location
    Newport, South Wales, UK
    Posts
    1,266

    Question Intialize An Array From A File?

    Hello,

    I'm wondering if there's a way to be able to initialize an array directly using a binary file, e.g. if I had:-
    Code:
    unsigned char buffer[4096];
    Is there a way of linking 4096 bytes of data from a file rather than specifying each individual byte within the source code? I've noticed that compilation time increases dramatically when I do this as the compiler has to parse each and every byte.

    My current workaround is to compile the program without initializing the array, locating the array in the executable output with a hex editor and overwriting the empty space.

    I don't mind if there are compiler-specific extensions that you are aware of.

  2. #2
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    Not sure what you mean. You want to have 4096 bytes put in there from the .exe file?

    If so, just open the file and read 4096 bytes into that buffer.

  3. #3
    Registered /usr
    Join Date
    Aug 2001
    Location
    Newport, South Wales, UK
    Posts
    1,266
    Well that would involve having code in your program to read from a file that you then have to cart around with the executable.

    I'm specifically referring to the linker phase of program construction. I would like it so that the only thing about the array that the compiler evaluates is that it is n bytes long (if that is necessary), then the linker should be able to associate the symbol generated by the compiler with the contents of a file.

    You could say that the resource data present in Windows executables is an example of this, although that isn't directly accessible to code without using external functions to retrieve it.

  4. #4
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    No you don't have to cart anything extra around. Just have the executable read itself into the array. If that's what you want. I'm still a little puzzled.

  5. #5
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    I don't know if the linker will help -- the linker is not really a very sophisticated tool (or I guess if it is I have completely missed it to date).

    But this may be a false efficiency: would you rather have this slow down once (at compile time) or every single time you run the program (and reading 4K from a file while the program is running is almost certainly slower than reading 4K from the file that is being compiled during compile-time).

  6. #6
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    False efficiency or I'm not reading it right. I'm sure retrieving the executable from the disk drive and setting up whatever internals to start it running is way more expensive than reading in 4K at the very beginning of the program. Therefore the 4K read is negligible in the scheme of things.

  7. #7
    Registered /usr
    Join Date
    Aug 2001
    Location
    Newport, South Wales, UK
    Posts
    1,266
    nonoob, I mean that when the program starts those 4096 bytes are already initialized with data.

    Say I wanted to store a 4KB picture. I could have an external .bmp file and read it into memory at run-time, or I could have it so that the executable already contains it and refers to it directly via a pointer without any further set-up or use of any functions.

    To expand on what I was saying about Windows resources, the resource compiler (e.g. Microsoft's rc.exe) would pack the bitmap up into a binary resource directory and then the linker would attach this directory inside the executable. This I assume requires that the linker knows what the resource directory is, rather than some random object.

    tabstop, I don't really have a problem just sticking it into the source code, but it just seems a bit silly that most of what is supposed to be a human-readable document is taken up by lines and lines of data that no-one needs to see, which then has to be interpreted by the compiler back into its original form.

  8. #8
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    Hmmm. How about defining a global variable,

    extern unsigned char[];

    which is properly initialized in another module... ".obj" or ".lib" whatever is used these days. The linker puts them all together.

  9. #9
    Registered /usr
    Join Date
    Aug 2001
    Location
    Newport, South Wales, UK
    Posts
    1,266
    Indeed. As I am following this thread my Googling has also reached the same conclusion.
    Any suggestions for a tool to convert files to linkable .obj files?
    I'm half-tempted to look up the COFF format and write one at this stage.

  10. #10
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    I think any compiler can produce .obj files as its target. Probably does anyhow temporarily. gcc has some command-line flag, but I don't know how to in Visual Studio.

  11. #11
    Registered User C_ntua's Avatar
    Join Date
    Jun 2008
    Posts
    1,853
    Ok. So you have a 4k picture. You store it in an array format, reading it with a proper program (like matlab). You copy that array into a file. Like image.c. There you just initialize an array like unsigned char image[] = { your array/pic}; Then you have to make your array look like val1, val2, ...., valn so it can be initialized in a C syntax. You can make a simple C program to do that for you. Then you compile the file and have an .o. Then you use that .o with your main program and you are done. I suppose that is what nonoob meant?
    I don't think there is a better way to do this in order to gain maximum performance

    EDIT: the flac is -c for gcc to make an object (.o). You just put gcc -c my.c and you get a my.o

  12. #12
    Registered User
    Join Date
    Sep 2008
    Location
    Toronto, Canada
    Posts
    1,832
    I'd write a little tool to read a .jpg file or whatever, and generate
    Code:
    unsigned long image[] = { 0x00000000, 0xFFFFFFFF, ... };
    This will be compiled into a separate .obj that's to be linked later.

    That's probably the most compact. Yeah it's still a strange workaround to do what you want.

  13. #13
    Registered User C_ntua's Avatar
    Join Date
    Jun 2008
    Posts
    1,853
    Quote Originally Posted by nonoob View Post
    I'd write a little tool to read a .jpg file or whatever, and generate
    Code:
    unsigned long image[] = { 0x00000000, 0xFFFFFFFF, ... };
    This will be compiled into a separate .obj that's to be linked later.

    That's probably the most compact. Yeah it's still a strange workaround to do what you want.
    Well, it would be a nice practice, but note that reading a compressed image like a jpg isn't that easy. But you could find an appropriate program on the net.
    I don't find it strange though. It is EXACTLY what you want. The only bad thing is that you have to make the array in an appropriate format to initialize the array. But there is really no easier way either....
    To be more clear. You don't really want to initialize the array from a file, thus from information stored in the HDD. You want to initialize it with information from the RAM. Even more precise, you don't want to initialize the array, you just want to have a pointer to the information you need.

  14. #14
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,701
    In my years, whenever something like this was needed, we would just write a utility to generate a custom .h and .c file.
    Code:
    //custom_data.h
    extern unsigned char g_custom_data[];
    // custom_data.c
    unsigned char g_custom_data[] = {0x01, 0x02, 0x03, ..... };
    Then compile and link as usual. Using bytes as the data type prevents any endianess issues.

    gg

  15. #15
    Registered /usr
    Join Date
    Aug 2001
    Location
    Newport, South Wales, UK
    Posts
    1,266
    I did actually look up the COFF format and wrote a converter
    It's very basic, but from testing with a debug build program it does the trick:-
    Code:
    // requires Win32 Platform SDK for structs and constants
    #define WIN32_LEAN_AND_MEAN
    
    #include <stdio.h>
    #include <time.h>
    #include <windows.h>
    
    int main(int argc, char **argv)
    {
    	char szFilename[256], *szPtr;
    	int bLong = 0, iData;
    	unsigned long ulSize;
    	FILE *fpIn, *fpOut;
    	IMAGE_FILE_HEADER header;
    	IMAGE_SECTION_HEADER section;
    	IMAGE_SYMBOL symbol;
    
    	printf("Binary File -> Object File Converter\n");
    	if (argc < 3 || argc > 4)
    	{
    		printf("Usage: bin2obj <filename> <symbol> [output filename]\n");
    		return -1;
    	}
    
    	fpIn = fopen(argv[1], "rb");
    	if (!fpIn)
    		return -1;
    
    	fseek(fpIn, 0, SEEK_END);
    	ulSize = ftell(fpIn);
    	fseek(fpIn, 0, SEEK_SET);
    	header.Machine = IMAGE_FILE_MACHINE_I386;
    	header.NumberOfSections = 1;
    	header.TimeDateStamp = time(NULL); // really necessary?
    	header.PointerToSymbolTable = sizeof(header) + sizeof(section);
    	header.NumberOfSymbols = 1;
    	header.SizeOfOptionalHeader = 0;
    	header.Characteristics = 0;
    	memset(&section, 0, sizeof(section));
    	memcpy(section.Name, ".data", 5);
    	section.SizeOfRawData = ulSize;
    	section.PointerToRawData = sizeof(header) + sizeof(section) + sizeof(symbol);
    	section.Characteristics = IMAGE_SCN_MEM_READ | IMAGE_SCN_MEM_WRITE | IMAGE_SCN_CNT_INITIALIZED_DATA;
    	memset(&symbol, 0, sizeof(symbol));
    	// determine short/long symbol name storage
    	if (strlen(argv[2]) <= 8)
    		memcpy(symbol.N.ShortName, argv[2], strlen(argv[2]));
    	else
    	{
    		symbol.N.LongName[1] = 4;
    		section.PointerToRawData += strlen(argv[2]) + 5;
    		bLong = 1;
    	}
    
    	symbol.SectionNumber = 1;
    	symbol.StorageClass = IMAGE_SYM_CLASS_EXTERNAL;
    	// prepare output filename
    	if (argc == 4)
    		strcpy(szFilename, argv[3]);
    	else
    	{
    		// append extension to input filename
    		strcpy(szFilename, argv[1]);
    		szPtr = szFilename + strlen(szFilename) - 1;
    		while (szPtr > szFilename && *szPtr != '.')
    			szPtr--;
    
    		if (szPtr > szFilename)
    			strcpy(szPtr, ".obj");
    		else
    			strcat(szFilename, ".obj");
    
    	}
    
    	fpOut = fopen(szFilename, "wb");
    	if (!fpOut)
    		return -1;
    
    	fwrite(&header, sizeof(header), 1, fpOut);
    	fwrite(&section, sizeof(section), 1, fpOut);
    	fwrite(&symbol, sizeof(symbol), 1, fpOut);
    	if (bLong)
    	{
    		ulSize = strlen(argv[2]) + 5;
    		fwrite(&ulSize, 4, 1, fpOut);
    		fwrite(argv[2], strlen(argv[2]) + 1, 1, fpOut);
    	}
    
    	while ((iData = getc(fpIn)) != EOF)
    		putc(iData, fpOut);
    
    	fclose(fpIn);
    	fclose(fpOut);
    	return 0;
    }

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. File transfer- the file sometimes not full transferred
    By shu_fei86 in forum C# Programming
    Replies: 13
    Last Post: 03-13-2009, 01:44 PM
  2. Post...
    By maxorator in forum C++ Programming
    Replies: 12
    Last Post: 10-11-2005, 09:39 AM
  3. File I/O problems!!! Help!!!
    By Unregistered in forum C Programming
    Replies: 4
    Last Post: 05-17-2002, 09:09 PM
  4. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 05:10 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21