Thread: How to parse my buffer by size?

  1. #1
    Registered User
    Join Date
    Jan 2018
    Posts
    5

    How to parse my buffer by size?

    Hello.

    I need your help.

    I have a buffer (there are my data from binary file)
    Code:
    char *buffer;
    And I know lenght of buffer (it is size of my binary file)
    Code:
    unsigned long Len;
    The length of buffer is variable. For example 100Bytes to 2048Kbytes.

    And I need function, to generate array with specific size (for example, specific size):
    Code:
    size_t maxSize = 4096;
    But I don't know, how to generate array with data, parse by size.

    For example:
    Binary file is 8168Bytes. I create buffer with data, lenght LEN. And I need parse buffer by size. So I need output:
    OUT1 .... size 4096Bytes
    OUT2 .... size 4072Bytes

    Or
    Binary file is 100Bytes. I create buffer with data, lenght LEN. And I need parse buffer by size. So I need output:
    OUT1 .... size 100Bytes

    Or
    Binary file is 11560Bytes. I create buffer with data, lenght LEN. And I need parse buffer by size. So I need output:
    OUT1 .... size 4096Bytes
    OUT2 .... size 4096Bytes
    OUT3 .... size 3368Bytes

    Can anybody help me, with solution?

  2. #2
    Registered User
    Join Date
    Jan 2018
    Posts
    5
    And I have this function. And this function create xy files from one file by size.

    Code:
    int splitFile(char *fileIn, size_t maxSize)
    {
        int result = 0;
        FILE *fIn;
        FILE *fOut;
        char buffer[1024 * 16];
        size_t size;
        size_t read;
        size_t written;
    
        if ((fileIn != NULL) && (maxSize > 0))
        {
            fIn = fopen(fileIn, "rb");
            if (fIn != NULL)
            {
                fprintf(stderr, "\n");
                fOut = NULL;
                result = 1;   /* we have at least one part */
    
    
                while (!feof(fIn))
                {
                    /* initialize (next) output file if no output file opened */
                    if (fOut == NULL)
                    {
                        sprintf(buffer, "%s.%03d", fileIn, result);
                        fprintf(stderr, "%s.%03d\n", fileIn, result);
    
                        fOut = fopen(buffer, "wb");
                        if (fOut == NULL)
                        {
                            result *= -1;
                            break;
                        }
    
                        size = 0;
                    }
    
                    /* calculate size of data to be read from input file in order to not exceed maxSize */
                    read = sizeof(buffer);
                    if ((size + read) > maxSize)
                    {
                        read = maxSize - size;
                    }
    
                    /* read data from input file */
                    read = fread(buffer, 1, read, fIn);
                    if (read == 0)
                    {
                        result *= -1;
                        break;
                    }
    
                    /* write data to output file */
                    written = fwrite(buffer, 1, read, fOut);
                    if (written != read)
                    {
                        result *= -1;
                        break;
                    }
    
                    /* update size counter of current output file */
                    size += written;
                    if (size >= maxSize)   /* next split? */
                    {
                        fclose(fOut);
                        fOut = NULL;
                        result++;
                    }
                }
                fprintf(stderr, "\n");
                /* clean up */
                if (fOut != NULL)
                {
                    fclose(fOut);
                }
                fclose(fIn);
            }
        }
    
    
        return (result);
    }
    And how to modificate, to return x, y, z array with data?

  3. #3
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    Is maxSize really only known at run time, or can it be a compile time constant? If it can be the latter, then I recommend:
    Code:
    #define BUFFER_MAXSIZE 4096
    
    /* ... */
    
    char buffer[BUFFER_MAXSIZE];
    This way, you avoid dynamic memory location, and since a buffer's maximum size is likely to be relatively small, this should be perfectly fine as a non-static local variable. If maxSize is only known at run time, then using malloc and free would be appropriate, except that as an optimisation you would only call malloc once or twice: once if you only use the buffer once; twice if you use the buffer more than once, i.e., once on the first use, once on the last use.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  4. #4
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    Are you trying to return all the generated filenames, in addition to the number of files generated?

    Your use of feof is incorrect. If your input file is an exact multiple of maxSize, you end up with an empty last file.

    What happens when you split an empty file?
    What happens when you have more than 1000 parts?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  5. #5
    Registered User
    Join Date
    Jul 2002
    Posts
    33
    Cant you just copy the elements you want to parse into other arrays?

    binarydata[1000]
    data1[max_size];
    data2[max_size];

  6. #6
    Registered User
    Join Date
    Jul 2002
    Posts
    33
    Expanding on my answer above

    If you want to parse binarydata to two other arrays


    &ptr1=binarydata[75] //point at where the array will be parsed

    //put this in a for loop to assign elements
    data1[i]=*(ptr1++);


    //then do it again for the other parse data set
    &ptr1=binarydata[10]

    //put this in a for loop to assign elements
    data2[i]=*(ptr1++);

  7. #7
    Registered User
    Join Date
    Apr 2017
    Location
    Quetzaltenango
    Posts
    82
    Split, as one of the GNU coreutils, has been hacked/polished for 30 years. (split.c source) Or you might look at GBufferedInputStream in glib GIO.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Buffer size question.
    By Swerve in forum C++ Programming
    Replies: 6
    Last Post: 11-07-2009, 08:45 AM
  2. stdin buffer size
    By Kempelen in forum C Programming
    Replies: 6
    Last Post: 03-16-2009, 03:06 PM
  3. size of buffer
    By silentintek in forum C Programming
    Replies: 10
    Last Post: 10-29-2008, 07:21 PM
  4. Get file size when in buffer
    By Siphon in forum C++ Programming
    Replies: 10
    Last Post: 04-10-2008, 10:52 AM
  5. getchar buffer size
    By oncemyway in forum C Programming
    Replies: 3
    Last Post: 08-02-2005, 12:49 AM

Tags for this Thread