Thread: Max Size of an Array?

Hello! Im new to this site. Programming has become one of my side hobbies, even though I'm not very good at it Anyway, I have created a program that converts a text file that contains a HEX dump into a binary file. The text file looks like so...:

0xDDDDDDDD
0xDDDDDDDD
0xDDDDDDDD
0xDDDDDDDD
0xDDDDDDDD
... and so on

D = predetermined hex data

Heres the beginning of my code...


#include <stdio.h>

int main()
{
FILE *binfile;
FILE *textfile;

long int hexdata[499999];
long int x;
int *data;

... it also opens a text file called dump.txt, and uses fwrite to write each line into a binary file. The problem I am having is that if I increase the size of the array to say 7 digits (99999999) to accomidate for larger text files, the program crashes upon start. It doesnt even attempt to execute. Accordind to this C programming book, a long integer can be at max 10 digits (2,147,483,647) So why cant I declare my array to a similar size? Thanks for any help.

How much system memory do you have? That could be the issue. Because from what I know about C (which admittedly is not a ton, but it is enough to get me by), when it allocates an array, it creates a new variable (sorta) for each slot in the array, so if you don't have enough system memory to handle all the space taken up by an extremely large array, the program would segfault and go kaput. It may just be a hardware limitation.

Then again, I might be completely wrong. The above is just a guess, not a hard fact.

Shane

Originally Posted by Derek8588

Well, actually the text file will represent a 32 MB memory dump. Therefore, the text file is actually close to 100 MB, even though it represents 32 MB. Oh, can vectors be used within C and not just C++?

EDIT: To Answer 2nd reply: I have 1.5 GB of memory. I checked my process list and my app is only consuming 3 MB of memory while its running. i have 970 MB + of memory Free.

Sorry I missed the vector thing at first, see the edit.

Anyway, the size (be it 32MB, 100MB, or whatever) is not relevant, because you're not going to process the whole file at a time; if you ever even try to put the whole bloody file in memory, you're just being silly. It looks like you have one (four-byte) hex number per line? Read in the number (you should be able to use the %x format specifier; make sure the target is a four-byte unsigned), process, write it out. Lather, rinse, repeat until you're out of hex. No reason to deal with more than one line at a time.

Originally Posted by Derek8588

EDIT 2: I was just thinking of trying to use one variable to store one line of text, write that to disk in binary, clear the variable, and write the next line from the text file into it.... However, the character and string reading functions tend to require an array as an argument... *Scratches Head*

A string is just a bunch of characters all in a row, so is represented by char [] (or char *). However, if your text really is just a lot of hex numbers, you can read them in as such.

You can't make an array that large without it being dynamically allocated because it is allocated on the stack, which has a limited (small) capacity.

Just do the sensible thing and make all your variables global unless there is a specific need
for it to be local.

Then again you could follow the advice of the 'experts' and have endless troubles :O)

It's your choice , it's a free world!! Choose carefully!!

Originally Posted by esbo

Just do the sensible thing and make all your variables global unless there is a specific need
for it to be local.

Then again you could follow the advice of the 'experts' and have endless troubles :O)

It's your choice , it's a free world!! Choose carefully!!

Except the program has been "working like a charm" for 29 hours now.

Originally Posted by tabstop

Except the program has been "working like a charm" for 29 hours now.

Has it?
What was the problem then? I can't seem to find that in the thread.
What was the solution?

Originally Posted by robwhit

You can't make an array that large without it being dynamically allocated because it is allocated on the stack, which has a limited (small) capacity.

That does not make sense to me.
Variables allocated on the stack are dynamic by definition.
Did you mean static when you said dynamic?

search the board for malloc.

Originally Posted by robwhit

search the board for malloc.

I don't need to as malloc is not the only form of dynamic allocation and it is not the
form of dynamic allocation being discussed here. Variables local to a function are also
dynamic and that is what is beinig discussed here.
So when you say dynamically allocated variables you need to be clear as to what you mean.