char to binary and reverse

This is a discussion on char to binary and reverse within the C++ Programming forums, part of the General Programming Boards category; Hi all, Im trying to reduce the file reading memory consumed for a problem.So i wanted to try doing this ...

  1. #1
    Registered User
    Join Date
    May 2010
    Posts
    22

    char to binary and reverse

    Hi all,
    Im trying to reduce the file reading memory consumed for a problem.So i wanted to try doing this thing.
    I have a text file of 2 million characters. This file when read each time by my code takes max memory. The process to manipulate from those character stored in system as second step doesnt consume so much time.
    hence i decided to convert the character text file into binary file so that the system can directly store this binary file and use its information when required.

    1.Can anyone please tell me how to take a text file and rewrite it as a binary file.
    Once this is done, the binary file is now always used by my code as input data ....for as many executions of the problem.

    2. use this binary file and store its information using my code, in a useful array of characters in the system.

    I know this seems pointless but do let me know how this is done. and if this idea will reduce the memory
    thanks
    regards
    specy

  2. #2
    Registered User
    Join Date
    May 2010
    Posts
    22
    instead of reading a file with text characters i was thinking of

    1.change the text input files -> binary data stored for input
    2. use binary file and store it in a array. convert it as character and then try the manipulation

    i hope i was not confusing
    looking forward to some help here
    -
    KR/specy

  3. #3
    Registered User
    Join Date
    Mar 2010
    Posts
    109
    I'm not totally sure what you are after, but everything is technically binary on a computer. What are you using to read the file? Usually there is a parameter or setting that dictates whether it will be read in as text or binary. Just set it to binary and you will get binary input. I don't think this is going to help you overall, though.

  4. #4
    Registered User
    Join Date
    May 2010
    Posts
    22
    hey thanks. I have now done the reading mode of a file to binary... directly instead of characters...

    I had a text file called base.txt
    it had a few characters as ATGCCGCGCGCATTTTTCCCAAAA of several lines (total chars are 2 million)
    i had to locate the CG in the txt file and store the location points in a seperate int array.
    This is the problem

    I used getline to read each line of the text and store in a char array.

    the input file stream function takes so much time to read the file. so i thought it would reduce time if i read the input file as binary as you said..... would it reduce the time
    thanks

  5. #5
    Registered User
    Join Date
    Mar 2010
    Posts
    109
    Perhaps if you are trying to read a large amount of data and using ifstream, you should use the read function and provide a larger buffer. The problem you may be running into is that getline itself isn't performing optimally for your situation.

  6. #6
    Registered User jeffcobb's Avatar
    Join Date
    Dec 2009
    Location
    Henderson, NV
    Posts
    875
    Two words: memory map (mmap()). Lets you treat your two meg file as it it were a contiguous bit of memory while only using a small portion of actual memory. This is the basis of virtual memory or the idea of using more memory than you actually have.
    C/C++ Environment: GNU CC/Emacs
    Make system: CMake
    Debuggers: Valgrind/GDB

  7. #7
    Registered User
    Join Date
    Nov 2007
    Posts
    46
    Not really sure he needs to use mmap or anything. On my not-so-fast laptop, I can load in a 2 MB file in less than 1 second, and in that time frame also count the number CG parts. I first get the file size using seekg/tellg and then creating a vector of the required size, then read all the data into this vector in one go using std::ifstream::read, remove all newline chars, and from there the data is ready to use, and can easily be converted to a string if desired?

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Growing a Binary Tree
    By hdragon in forum C++ Programming
    Replies: 1
    Last Post: 03-28-2007, 02:41 AM
  2. What's problem of adding two binary number?
    By Mathsniper in forum C Programming
    Replies: 1
    Last Post: 01-12-2007, 05:12 AM
  3. help: dec to binary
    By tige in forum C Programming
    Replies: 5
    Last Post: 12-25-2002, 10:27 PM
  4. Reverse of a Binary
    By xeneize in forum C++ Programming
    Replies: 3
    Last Post: 10-07-2002, 08:29 AM
  5. strings
    By Unregistered in forum C++ Programming
    Replies: 11
    Last Post: 09-10-2001, 03:56 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21