how to read in files that are bigger than 3gigabyte?

Yes, the C and C++ IO libraries for your platform seems to choke on files larger than 4GB for 32 bit CPU because there is at most 2^32 memory address and > 4GB exceeded that.

I also tried CreateFile, ReadFile, and WriteFile (Windows API functions), doesn't work for files larger than 4GB too.

Thank you guys for your advices, I will go try iostreams and Boost.IOStreams.

Another option is to buy a 64bit CPU

So, fopen64(), fread64() doesn't work?

>> I also tried CreateFile, ReadFile, and WriteFile (Windows API functions), doesn't work for files larger than 4GB too.
They work for files of any size.

>> So, fopen64(), fread64() doesn't work?
Not part of the MS CRT. Those are in GNU libc.

gg

Originally Posted by Codeplug

>> I also tried CreateFile, ReadFile, and WriteFile (Windows API functions), doesn't work for files larger than 4GB too.
They work for files of any size.

>> So, fopen64(), fread64() doesn't work?
Not part of the MS CRT. Those are in GNU libc.

gg

I tried createfile and readfile, it doesnt work for > 4gb files

Code:

Char c;
DWORD wmWritten;
HANDLE a =CreateFile(result_file.c_str(), GENERIC_READ|GENERIC_WRITE,FILE_SHARE_READ,NULL,OPEN_ALWAYS,FILE_ATTRIBUTE_NORMAL,NULL); 
ReadFile(a, (char *)(&c), sizeof(int), &wmWritten, NULL);
CloseHandle(a);

That seems extremely unlikely, since that would be the API that created the file in the first place [unless the unlikely scenario is that you have a file created by a different OS on a removable media of some sort].

Of course, what you are doing in ReadFile doesn't seem quite right, since I expect that Char is some sort of "char" type, and you are asking to read sizeof(int) bytes, which would be, in most systems, 4 times larger than Char.

--
Mats

Originally Posted by matsp

Of course, what you are doing in ReadFile doesn't seem quite right, since I expect that Char is some sort of "char" type, and you are asking to read sizeof(int) bytes, which would be, in most systems, 4 times larger than Char.

--
Mats

because the input file is in binary format and it stores integer. I am reading in 4 bytes (size int) each time and store it in var c.

the code works for me when the file is less than 4GB

I am reading in 4 bytes (size int) each time and store it in var c.

But unless c is 4 bytes large (and the type name Char suggests it isn't) you're overwriting memory you shouldn't.

Originally Posted by franziss

because the input file is in binary format and it stores integer. I am reading in 4 bytes (size int) each time and store it in var c.

the code works for me when the file is less than 4GB

That may be pure coincidence. If Char is a char, then it can only hold ONE BYTE, and that is not enough to store 4 bytes (quite obviously).

I wrote a piece of code that generates an 8GB file, and then opens that file again. Works fine for me. I only have about 10GB free on my hard-disk, so I can't really do a much bigger file, but seeing as the failure is either at 4GB or some MUCH larger number, I don't think it matters. [Yes, I do need to clean up my disk or rebuild my computer]

--
Mats