Binary Reader Buffer size

Hello,

I woud like to read a very big binary file in C#. Say, 256 Mega Bytes of data. Actually I should read from one big 256Mbytes file and append to another file.
How do I append to my original file?
can i use some thing like this?

Code:

public static byte[] ReadFully (Stream stream)
{
byte[] buffer = new byte[268435456];
using (MemoryStream ms = new MemoryStream())
{
while (true)
{
int read = stream.Read (buffer, 0, buffer.Length);
if (read <= 0)
return ms.ToArray();
ms.Write (buffer, 0, read);
}
}
}

any help, suggestions is appreciated.


Thanks,

As long as you have memory for it, you sure can. Just remember that the actual read will take a bit as I believe Streams default to 4k buffers (so it will have to do multiple disk reads to get the entire thing).

@Momerath: Thanks,
If I'm reading from a file and appending to another file, is there any simple way to do it? How do I append what I read to another file?

If you're always reading from a file the simplest way would be to use:

Code:

System.IO.File.ReadAllBytes("File.ext");

If not I'd rather use the following than implementing my own low-level reader:

Code:

public byte[] ReadFully(System.IO.Stream Stream)
{
	using(var MemoryStream = new System.IO.MemoryStream())
	{
		Stream.CopyTo(MemoryStream);
		return MemoryStream.ToArray();
	}
}

(CopyTo has an overload where you can specify the buffer size, for )

Reading your post in more detail I see the actual question was appending.
The simplest way of this, assuming it's a textfile (most common when appening I guess), is this:

Code:

System.IO.File.AppendAllText("AppendedFile.txt", System.IO.File.ReadAllText("FileToAppend.txt"));

@Magos: Thanks.
I liked using this,

Code:

System.IO.File.ReadAllBytes("File.ext")

Can it read 256Mega Bytes? How do I append it to another file? can i do it same tiem i read?

Its a .bin file, binary file. I shud use appendallbytes right?

Thanks a ton for the info

I tried this

Code:

 System.IO.

Code:

File.AppendAllBytes("tom.mpa", System.IO.File.ReadAllBytes("xyz.bin")); 

error CS0117: 'System.IO.File' does not contain a
definition for 'ReadAllBytes'
error CS0117: 'System.IO.File' does not contain a
definition for 'AppendAllBytes'

File.ReadAllBytes Method (System.IO)

Check there.


Quzah.

Code:

public bool AppendFile(String file_to_read, String file_to_append)
{
    using (FileStream fs1 = new FileStream(file_to_read, FileMode.Open, FileAccess.Read))
    {
        using (FileStream fs2 = new FileStream(file_to_append, FileMode.Append, FileAccess.ReadWrite))
        {
            byte[] buf = new byte[4096];
            int size = 0;

            while ((size = fs1.Read(buf, 0, 4096)) > 0)
                fs2.Write(buf, 0, size);
        }
    }

    return true;
}

Here's a way of appending the contents of one file onto the end of another. Performance-wise it might be slower than reading the full files as demonstrated of previous answers - I've not tested it on 200+mb files, however, because it is using file streams it means that your application's memory requirement won't suddenly spike at 200+mb.

Just pointing out another option available for you.

You don't really need to return anything there, since you don't have a false case, where you return false on error. (Just a nit pick.)


Quzah.

Originally Posted by theoobe

Code:

public bool AppendFile(String file_to_read, String file_to_append)
{
    using (FileStream fs1 = new FileStream(file_to_read, FileMode.Open, FileAccess.Read))
    {
        using (FileStream fs2 = new FileStream(file_to_append, FileMode.Append, FileAccess.ReadWrite))
        {
            byte[] buf = new byte[4096];
            int size = 0;

            while ((size = fs1.Read(buf, 0, 4096)) > 0)
                fs2.Write(buf, 0, size);
        }
    }

    return true;
}

Here's a way of appending the contents of one file onto the end of another. Performance-wise it might be slower than reading the full files as demonstrated of previous answers - I've not tested it on 200+mb files, however, because it is using file streams it means that your application's memory requirement won't suddenly spike at 200+mb.

Just pointing out another option available for you.

Or even simpler:

Code:

public void AppendFile(String file_to_read, String file_to_append)
{
    using (FileStream fs1 = new FileStream(file_to_read, FileMode.Open, FileAccess.Read))
    {
        using (FileStream fs2 = new FileStream(file_to_append, FileMode.Append, FileAccess.ReadWrite))
        {
            fs1.CopyTo(fs2);
        }
    }
 }

Or, if you hate nesting too much: :P

Code:

public void AppendFile(String file_to_read, String file_to_append)
{
  using(var fs1 = new FileStream(file_to_read, FileMode.Open, FileAccess.Read))
  using(var fs2 = new FileStream(file_to_append, FileMode.Append, FileAccess.ReadWrite))
  {
    fs1.CopyTo(fs2);
  }
}

Originally Posted by Magos

Or, if you hate nesting too much: :P

Code:

public void AppendFile(String file_to_read, String file_to_append)
{
  using(var fs1 = new FileStream(file_to_read, FileMode.Open, FileAccess.Read))
  using(var fs2 = new FileStream(file_to_append, FileMode.Append, FileAccess.ReadWrite))
  {
    fs1.CopyTo(fs2);
  }
}

How is that less nesting? Messing with indentation != not nesting. All you did was get rid of braces around a single nested statement, akin to changing:

Code:

if(true)
{
  a = b;
}

to...

Code:

if(true)
  a = b;