Thread: Parallel processing with fork()

I have a math intensive application, and I'm trying to utilize my quad processor to geta speedup. The job is trivially parallelizable; I'm working through a huge 2D array and the job on each row is totally independent. Fork crunches on the job easily and I get 1/4th of the results sitting in each thread, but I'm stuck on how to combine the results into one thing. Can this be done easily? I'm not an expert programmer and some of the stuff I searched up on shared memory was pretty daunting. If I could somehow just extract a pointer from the children that points to their data or something?

It looks like pthreads can do this type of thing, but as far as I can tell it requires functions, unlike fork. In my case this would involve passing a ridiculous number of variables in the function, so I'm hesitant to go this route.

Thanks for any help.

Originally Posted by Mastiff

It looks like pthreads can do this type of thing, but as far as I can tell it requires functions, unlike fork. In my case this would involve passing a ridiculous number of variables in the function, so I'm hesitant to go this route.

Yes, when you create a thread, a function is called. But the threads will share global variables, so it would not be strictly necessary to pass a lot of arguments to your thread function; just enough so it knows what part of the dataset it should be working on. Just don't forget to lock (pthread_mutex_lock(), etc) data that might be accessed concurrently.

Of course, you might (reasonably) scoff at the use of globals. Each method has its downsides, I suppose.

fork() creates new processes, not new threads.
Couldn't you pass a struct that holds all the data instead of individual variables? I think you'd have to anyways, since the thread function can only take a void* parameter.

>> I'm hesitant to go this route
Fear of the unknown is understandable....so make it known

https://computing.llnl.gov/tutorials/pthreads/
http://www.cs.umu.se/kurser/TDBC64/V...ead-primer.pdf

gg

Originally Posted by cpjust

fork() creates new processes, not new threads.
Couldn't you pass a struct that holds all the data instead of individual variables? I think you'd have to anyways, since the thread function can only take a void* parameter.

Yes, as I understand it, I'd have to get everything into a structure. So, how do people generally go about this, do they create such a structure and use it all throughout the program, inlcuding the non-threaded stuff? or do you copy everything to the structure, do the threaded part, and then copy it out back to the normal variables? Either way seems bad due to the sheer messiness of it, especially since the data includes several huge (4Kx8K) arrays.

Thanks for the help.