Thread: Need loop to copy chunks from byte array

  1. #1
    Registered User
    Join Date
    Feb 2009
    Posts
    1

    Smile Need loop to copy chunks from byte array

    I have to process a large byte array that is passed to my function. I need to copy the content from this incoming byte array in smaller "chunks" to an outbound byte array.

    For every "chunk" of data I need to call a web service.

    Upon return, I need to resume looping through the incoming byte array, continuing to pass a whole or partial chunk of data until the complete byte array is processed (i.e. sent to the web service in chunks).

    I am very new to C# and I am struggling with a loop that works. I know how to call the web service to handle a "chunk" but I can't get the looping correct. Here is a sketch of the pathetic mess I currently have:

    Code:
    int chunkSize = 10000;
    byte[] outboundBuffer = new byte[chunkSize];     
    while (BytesRead > 0)
    {
    	long i = 0;
    	foreach (byte x in incomingArray)
    	{
    	    BytesRead += 1;
    	    outboundBuffer[i] = incomingArray[i]
    	    i++;
    	}
    	uploadObject.Size = BytesRead;
    	uploadObject.MTOMPayload = outboundBuffer;
    	
    	// call web service here and pass the uploadObject 
    	
    	// get next "chunk" until incomingArray is fully processed 
    }
    Could one of you experts take this idea and show me the proper way to loop? Thanks very much in advance.

  2. #2
    Registered User valaris's Avatar
    Join Date
    Jun 2008
    Location
    RING 0
    Posts
    507
    Try Array.Copy() for a ".net" way of doing it. Either that or just create a new array of the size you want and fill it in a for loop. For your solution in particular though, it may be more effective to use a queue. This way you can fill the buffer with any bytes, and then dequeue the chunks you need and process them.
    Last edited by valaris; 02-24-2009 at 08:08 PM.

  3. #3
    Registered User
    Join Date
    Mar 2009
    Location
    england
    Posts
    209
    OK there's probably a better way of doing it than this, but it works!

    Code:
    byte[][] my_chunks = LargeByteArrayToChunks(incomingArray, 10000);
    
    foreach (byte[] chunk in my_chunks)
    {
        uploadObject.Size = chunk.Length;
        uploadObject.MTOMPayload = chunk;
    
        // call web service here and pass the uploadObject 
    }
    And the LargeByteArrayToChunks function as follows...

    Code:
    private byte[][] LargeByteArrayToChunks(byte[] original_data, int max_chunk_size)
    {
        List<byte> buffer = new List<byte>(original_data);
        List<byte[]> result = new List<byte[]>();
    
        while (buffer.Count > max_chunk_size)
        {
            result.Add(buffer.GetRange(0, max_chunk_size).ToArray());
            buffer.RemoveRange(0, max_chunk_size);
        }
    
        if (buffer.Count > 0) // remember to check if there is any residual
            result.Add(buffer.ToArray());
    
        return result.ToArray();
    }

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Have problems with copying my array!
    By AvaGodess in forum C Programming
    Replies: 11
    Last Post: 09-25-2008, 12:56 AM
  2. byte array (vector)
    By l2u in forum C++ Programming
    Replies: 7
    Last Post: 09-25-2006, 12:24 PM
  3. Copy multidimensional array to single array?
    By seepox in forum C Programming
    Replies: 9
    Last Post: 05-08-2006, 11:19 AM
  4. error: identifier "byte" is undefined.
    By Hulag in forum C++ Programming
    Replies: 4
    Last Post: 12-10-2003, 05:46 PM
  5. Struct *** initialization
    By Saravanan in forum C Programming
    Replies: 20
    Last Post: 10-09-2003, 12:04 PM