Maximum Size of Data Transfers with MPI?

This is a discussion on Maximum Size of Data Transfers with MPI? within the Linux Programming forums, part of the Platform Specific Boards category; Hey guys, Does anyone happen to know the maximum amount of data that can be send in one MPI_Send and ...

  1. #1
    PhysicistTurnedProgrammer Cell's Avatar
    Join Date
    Jan 2009
    Location
    New Jersey
    Posts
    72

    Maximum Size of Data Transfers with MPI?

    Hey guys,

    Does anyone happen to know the maximum amount of data that can be send in one MPI_Send and received in one MPI_Recv transfer?

    I think that I had read once that the amount was not limited by MPI itself, but by other factors. That does make sense, but I'm having an annoying issue:

    I have a variable length array that takes a message size that the user specifies on the command line. This array is then filled up with a bunch of 'a' characters.

    After that, the character message is sent to another process. The receiving process then simply prints the specified amount of 'a' characters.

    When I run the program, if I specify any message size from 1-64, it receives and prints fine.

    If I try, say, 100 the program receives the message but only prints out 64 characters.

    A rough idea of my sending process code is:

    Code:
    	
    long long double TEST;
    	
    printf("How large should the message be?\n");
        	
    scanf("%lld", &TEST);
    	
    char test_string[TEST];
    	
    	for(long long double i = 0; i <= TEST; i++){
        	    test_string[i] = 'a';
           }	
    		
    	MPI_Status	status;
    	//MPI_Request	req;
    	
    	MPI_Init(&argc, &argv);
    	MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
    	MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);
    	
    	
    	if (my_rank == 0){
    	// Send the string
    	MPI_Send(test_string, /*4*/TEST, MPI_CHAR, 1, 99, MPI_COMM_WORLD);
    And the receiving process is:

    Code:
    	
    long long double TEST;
    char test_str[TEST];
    	
    MPI_Status	status;
    	
    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
    MPI_Comm_size(MPI_COMM_WORLD, &mpi_size);
    	
    	// Send the string
    	MPI_Recv(test_str, /*4*/UPPER_LIMIT, MPI_CHAR, 0, 99, MPI_COMM_WORLD, &status);
    	
    	// Print received string
    	printf("\n%s\n",test_str);
    Where UPPER_LIMIT is a #define of 50000000. I will eventually write this code so that the size of TEST is send first, and then the message is sent.

    A sample output is:

    How large should the message be?
    2

    aa
    And if I put 100:

    How large should the message be?
    100

    aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaa
    Any ideas?

    Edit - By the way, I am on a 64-bit architecture.

  2. #2
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Code:
    long long double TEST;
    char test_str[TEST];
    That shouldn't even compile.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  3. #3
    PhysicistTurnedProgrammer Cell's Avatar
    Join Date
    Jan 2009
    Location
    New Jersey
    Posts
    72
    Quote Originally Posted by CornedBee View Post
    That shouldn't even compile.
    It does!

    Why do you think it shouldn't compile?

  4. #4
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    1) long long double is not a valid type
    2) TEST is not initialized, so it does not form an integral constant expression, which makes it invalid as an array size specifier
    3) If TEST is supposed to be long double, it's not even integral, so it's even more invalid.

    What absurd compiler does this compile on, and what warnings do you get if you push up the warning level a bit?
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  5. #5
    PhysicistTurnedProgrammer Cell's Avatar
    Join Date
    Jan 2009
    Location
    New Jersey
    Posts
    72
    I am using gcc. I have switched to long long int, and now the process 0 program (the sending program) exists with signal 15.

    It gets as far as this statement:

    Code:
    printf("How large should the message be?\n");
    Prints that to the screen then the signal 15 occurs.

    Any idea what might be wrong?

  6. #6
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    Before or after you enter the number?
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  7. #7
    PhysicistTurnedProgrammer Cell's Avatar
    Join Date
    Jan 2009
    Location
    New Jersey
    Posts
    72
    Quote Originally Posted by CornedBee View Post
    Before or after you enter the number?
    Before I enter the number.

    I have no time to do anything -- right after I run the program it prints that statement and then exits.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Fixing my program
    By Mcwaffle in forum C Programming
    Replies: 5
    Last Post: 11-05-2008, 02:55 AM
  2. Adventures in labyrinth generation.
    By guesst in forum Game Programming
    Replies: 8
    Last Post: 10-12-2008, 01:30 PM
  3. Trouble with DMA Segmentation Faults
    By firestorm717 in forum C Programming
    Replies: 2
    Last Post: 05-07-2006, 09:20 PM
  4. Tab Controls - API
    By -KEN- in forum Windows Programming
    Replies: 7
    Last Post: 06-02-2002, 09:44 AM
  5. Determining Data Size For Network Send/Rec :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 6
    Last Post: 05-07-2002, 09:01 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21