Thread: char to binary and reverse

Hi all,
Im trying to reduce the file reading memory consumed for a problem.So i wanted to try doing this thing.
I have a text file of 2 million characters. This file when read each time by my code takes max memory. The process to manipulate from those character stored in system as second step doesnt consume so much time.
hence i decided to convert the character text file into binary file so that the system can directly store this binary file and use its information when required.

1.Can anyone please tell me how to take a text file and rewrite it as a binary file.
Once this is done, the binary file is now always used by my code as input data ....for as many executions of the problem.

2. use this binary file and store its information using my code, in a useful array of characters in the system.

I know this seems pointless but do let me know how this is done. and if this idea will reduce the memory
thanks
regards
specy

instead of reading a file with text characters i was thinking of

1.change the text input files -> binary data stored for input
2. use binary file and store it in a array. convert it as character and then try the manipulation

i hope i was not confusing
looking forward to some help here
-
KR/specy

I'm not totally sure what you are after, but everything is technically binary on a computer. What are you using to read the file? Usually there is a parameter or setting that dictates whether it will be read in as text or binary. Just set it to binary and you will get binary input. I don't think this is going to help you overall, though.

hey thanks. I have now done the reading mode of a file to binary... directly instead of characters...

I had a text file called base.txt
it had a few characters as ATGCCGCGCGCATTTTTCCCAAAA of several lines (total chars are 2 million)
i had to locate the CG in the txt file and store the location points in a seperate int array.
This is the problem

I used getline to read each line of the text and store in a char array.

the input file stream function takes so much time to read the file. so i thought it would reduce time if i read the input file as binary as you said..... would it reduce the time
thanks

Perhaps if you are trying to read a large amount of data and using ifstream, you should use the read function and provide a larger buffer. The problem you may be running into is that getline itself isn't performing optimally for your situation.

Not really sure he needs to use mmap or anything. On my not-so-fast laptop, I can load in a 2 MB file in less than 1 second, and in that time frame also count the number CG parts. I first get the file size using seekg/tellg and then creating a vector of the required size, then read all the data into this vector in one go using std::ifstream::read, remove all newline chars, and from there the data is ready to use, and can easily be converted to a string if desired?