Buffer help

This is a discussion on Buffer help within the C++ Programming forums, part of the General Programming Boards category; Hi, Ive written some code to decode a file, and write the decoded data out to a new file, the ...

  1. #1
    Registered User (TNT)'s Avatar
    Join Date
    Aug 2001
    Location
    UK
    Posts
    339

    Buffer help

    Hi,

    Ive written some code to decode a file, and write the decoded data out to a new file, the problem is at the moment its incredibly slow, it processing the file character by character and it could be a very large file.

    My current decoding process is:

    Read a line from the file, process each character individually, write out each decoded character individually to the file, Repeat for all lines.

    If i comment out the 'write each char to file' line of the code, then its a whole lot faster, so i think the best solution around this is to use a buffer of some kind, but i am not really sure how to implement it and what size would be most suitable etc

    I guess i could use a std::string as a buffer, and add a certain amount of chars to it then write it out, but i don't know how many chars there will be and what the memory limitations are on a string, and i am not too confident about how to correctly use a char * buffer and how to correctly allocate the memory for it and add a single char to it each loop iteration (strcpy()?), so if anyone could tell me how to optimize this code and kill the slowness it would be great.

    Heres my current code (the file output is native linux but i don't think thats relevant to the problem).

    Code:
    string tmp;
    char *buffer = new char[1024]; // Input read line buffer
    
    	do
    	{
    		istr.getline(buffer, 1024);
    
    		header_line = strstr(buffer, "=yend");	
    		
    		if(header_line == NULL)
    		{
    			int ascii_result = 0;
    			
    			// Decode line
    			for(int i = 0; i < strlen(buffer) - 1; i++)
    			{
                                   ...
                                   // Decoding maths on buffer[i]
                                   ...
    
                                    // Dump decoded char to output file
    				tmp = (char)ascii_result;
    				write(ostr, tmp.c_str(), 1);
    			}
    
    		}
    		
    	} while(header_line == NULL);

    Thanks for any help,

    Jack
    TNT
    You Can Stop Me, But You Cant Stop Us All

  2. #2
    Registered User
    Join Date
    Jan 2005
    Posts
    7,317
    There are no memory limitations on a string other than the limitations of your computer. You can keep using += to add to the end. However, a vector<char> might be faster for such things.

    Why not change each value directly in the buffer you already have for input, then output the buffer itself. No temporary string needed.

  3. #3
    System Novice siavoshkc's Avatar
    Join Date
    Jan 2006
    Location
    Tehran
    Posts
    1,231
    Its simple. Load whole the file into memory, decode it, write the output to a new file.
    Learn C++ (C++ Books, C Books, FAQ, Forum Search)
    Code painter latest version on sourceforge DOWNLOAD NOW!
    Download FSB Data Integrity Tester.
    Siavosh K C

  4. #4
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,333
    > for(int i = 0; i < strlen(buffer) - 1; i++)
    Well this is going to be slow as well - calling strlen EVERY time around the loop.

    Store tmp in another buffer of similar size, and write() the whole lot when the buffer is full.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 16
    Last Post: 10-29-2006, 04:04 AM
  2. writing a pack-style function, any advices?
    By isaac_s in forum C Programming
    Replies: 10
    Last Post: 07-08-2006, 08:09 PM
  3. buffer contents swapping
    By daluu in forum C++ Programming
    Replies: 7
    Last Post: 10-14-2004, 02:34 PM
  4. Tetris Questions
    By KneeGrow in forum Game Programming
    Replies: 19
    Last Post: 10-28-2003, 10:12 PM
  5. Console Screen Buffer
    By GaPe in forum Windows Programming
    Replies: 0
    Last Post: 02-06-2003, 04:15 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21