Coming from an assembler background, where I have to manage *everything* in regards to thread dispatching, memory isolation, synchronization, etc., to a higher level compiled language like C or C++, where the compiler takes care of several aspects of storage (the stack in C or otherwise in C++), I'm trying to get my head around what I need to worry about in regards to thread-safety, multi-threading I/O to the same file, and what I can take for granted in cases where the compiler will protect me.
For example, in my single threaded app, I do this today:
Code:
open input & output files
get an input buffer
get an output buffer
repeat:
read 10,000 (or last remaning portion thereof) records into input buffer
convert input records into output buffer
write the output buffer
loop to repeat until EOF
close both files
My multi-threaded design, I'm thinking, will be like this:
Code:
main thread {
open input & output files
create 4 converter threads via "init"
get 4 input buffers
repeat {
do until no input buffers available {
read 10K records into buffer
pass buffer to a waiting converter task via "work"
}
wait for input buffer to be available
loop to repeat until EOF
}
wait for all converter tasks to finish
tell converter tasks to shutdown via "term"
close both files
}
converter thread {
init:
get an output buffer
indicate ready for work
work:
convert the passed buffer data
write the data
mark input buffer free
term:
free output buffer
thread exit
}
So my questions are, initially, in regards to C, on Unix or Windows, will this design fly? Can I overlap writing to the same file by 4 threads at once? Or, do I need a "writer thread" that is serialized which the converter threads will post when an output buffer is ready to be written?
Thanks. More questions to come, I'm sure.
Todd