Hi,
I'm writing an algorithm in MPI which basically splits up a massive vector amongst different nodes (i.e. node 1 has entries 0-1,000,000, node 2 has 1,000,000 - 2,000,000, etc.) and basically my algorithm requires that some calculation be done on each element of the array and the result gets added to another component of a duplicate array of the same size and split up in the same way which, most likely, won't be located on the same node (i.e. after computing on element 1000 on node 1 I find that I have to add the result to element 1,000,102 which is on node 2). So basically, if each node has n components of the vector on it it's going to have to send the result to n different places AND receive lots of potential changes to its own section of the vector. Now my question is, what is the BEST way to do this? I'm torn between:
-after computing each element ISend the result (assuming it needs to be sent to another server) and then do a sweep of IReceives to see if there are any results from other servers that are trying to be sent to me.
-the same thing using buffered send (IBSend). However, memory usage is a huge issue (I'm basically going to make my vector as big as possible) so I don't know how many buffers I need. Should I set up one local buffer for each server (i.e. buffer 2 on server 1 is big enough for a single message and is reserved for IBSend's to server 2). But then what if I need to send to another server but there's still an outgoing message waiting to be received on the other side. I'll have to block until it gets picked up
-The final option i'm wondering about is MPI-2's windowing features and using the put command to simply place the result where it needs to be placed.
Can someone who knows a fair amount about MPI performance considerations help me determine which implementation will be the most effective. Any help is greatly appreciated
