Thread: File buffer for video stream

Greetings,

I have been searching for days for a solution to my issue, but apparently lack the terminology required to gain valid results.

I will start with what I am trying to do, and maybe someone on this board could point me in the right direction.

I am trying to stream a video from mencoder to a special file buffer on the local machine.

Described below is what I want the file buffer to do.

First I want to specify the file size of the buffer.
Secondly I need the buffer to be a FIFO.
Operation of the buffer is as follows:
When the file is opened, data will be written as file data is normally written.
If the buffer is full, the data will be discarded in a first in first out manner to allow new data to be written.

On the read side the file is opened and the program can read the oldest bytes first.

As an example ffserver from the ffmpeg package has this feature built in, unfortunately I cannot locate that function in their source code.

I appreciate any reply. I am not looking for someone to write the code for me, just simply assist me in terminology so I may be able to perform a more lucid search. However, if you do happen to have example code, that information would be appreciated as well.

Thanks

Other than the FIFO buffer part, I'm clueless what you want, but here's my thought on it.

Two scenarios:
#1)
Divide it up into 2 halves, (A & B). The goal here is to stop the slow down that would be needed by shuffling data from the end of the buffer, to the start of it, as new data is being received. That's a performance killer in a large buffer.

Your buffer will be a multiple of 16 and BUFSIZE (if possible), on your system. (BUFSIZE is a macro your compiler will have that is moderate in size, but optimized for system performance. 1024 bytes is a good starting point to test). As the A half is being filled with data, all is well. B half is empty (and set to zero by using calloc when it's allocated, as was A half, before it).

When A is full, then B begins filling, and at the same time, A begins writing out it's data. When A is out of data, B is full, so A becomes the active receiver of data, and is memset to zero. B begins writing out data now, stressing that this is happening at the same time.

The effect of this synchronicity, is that throughput is greatly enhanced, and no shuffling of data from the end of the buffer to the start of the buffer, is needed. All while achieving FIFO.

#2)
Set your FIFO buffer to the same size as the buffer on your hard drive. When the buffer is full, you can write out it's contents in one call to a function, (ideally, to a non-system drive), then quickly go back to receiving another load of data in your buffer. If you get a different hard drive, check that the buffer is sized to match the size of the buffer in the hard drive, for best performance.

Simple & efficient, but you won't be able to receive data into your buffer, while data is being written out to the hd buffer. Not as fast as #1, but simpler, and quite possibly, all that your program needs.

The bigger the buffer, the more you will want to use #1. Both #1 and #2, can be made even faster if you use 2 or more threads, rather than one.

If you use a video program or library, this should already be built into or readily available from it.

Adak ... this is an application where one needs pipes... basically a buffered FIFO where you (metaphoically) write to one end and read from the other, between is storage. Windows offers them, PellesC has a library for them... beyond that....

It's been a LONG time since I played with this, so perhaps one of the more experienced members could chip in here.

So the buffer gets buffered!

Originally Posted by Adak

#1)
Divide it up into 2 halves, (A & B). The goal here is to stop the slow down that would be needed by shuffling data from the end of the buffer, to the start of it, as new data is being received. That's a performance killer in a large buffer.

Your buffer will be a multiple of 16 and BUFSIZE (if possible), on your system. (BUFSIZE is a macro your compiler will have that is moderate in size, but optimized for system performance. 1024 bytes is a good starting point to test). As the A half is being filled with data, all is well. B half is empty (and set to zero by using calloc when it's allocated, as was A half, before it).

When A is full, then B begins filling, and at the same time, A begins writing out it's data. When A is out of data, B is full, so A becomes the active receiver of data, and is memset to zero. B begins writing out data now, stressing that this is happening at the same time.

The effect of this synchronicity, is that throughput is greatly enhanced, and no shuffling of data from the end of the buffer to the start of the buffer, is needed. All while achieving FIFO.

This seems logical. I am trying it now.

Originally Posted by Adak

So the buffer gets buffered!

The buffering is in memory... there's little reason to use disk buffering when memory is so plentiful and fast.

Originally Posted by CommonTater

The buffering is in memory... there's little reason to use disk buffering when memory is so plentiful and fast.

Of course, you don't want to use a disk file for a buffer, unless you absolutely have to. You want to use as much RAM buffer as feasible, before using anything on the hard drive.

If your compiler has a built in feature for this, by all means, use it.

Originally Posted by Adak

Of course, you don't want to use a disk file for a buffer, unless you absolutely have to. You want to use as much RAM buffer as feasible, before using anything on the hard drive.

If your compiler has a built in feature for this, by all means, use it.

That's what pipes are about... you feed stuff in one end, it's buffered, you get it back out the other...

Here's the MSDN reference for it: Pipe Functions (Windows)

As I say it's been a long time since I did anything with this this and I'm sure I'm not the best person to explain it. But it does seem to be what the OP needs....