The standard send function, MPI_Send, sends data to a specific process identified by the idest argument. The rank of the destination processor (idest) must be within the context of the communicator icomm -- that is, a group member of the communicator. The tag number is an integer that you define and set. ierr= MPI_Send(data, icount, itype, idest, itag, icomm)

The standard send call does not return until the data area is free to be overwritten. Many MPI implementations have local buffers to expedite the return: if the data and its associated message envelope can be immediately stored for safe keeping in the buffer, the send call returns immediately and the rest of the communication continues "behind the scenes". If there is insufficient buffer space to pack the message, the send function waits until a receive function in process idest accepts the send. In summary, MPI_Send safeguards the message data from being overwritten and the send may wait until an accepting receive is posted if insufficient space is available to buffer the message.

The standard receive function, MPI_Recv, has arguments identical to the default send function except for the additional integer message-status array and the substitution of a source variable for a destination variable.

Note: both isrc and itag are input variables, and they must match sender values on entry into the function. They are not output variables.

The receive call will wait until a message arrives and is completely transferred to the data location. Call it with this syntax:

ierr= MPI_Recv(data, icount, itype, isrc, itag, icomm, istatus)          

where:

In a send/receive pair, the MPI data type and the communicator must match. Since the data count (icount) specifies the size of the receiver's data array in the receiving process, it should equal or exceed that of the sender's value or you run the risk of the receiver writing past the end of your data storage array. The parameters MPI_ANY_TAG and MPI_ANY_SOURCE may be used in the receiver to accept any tag and any source, respectively. Below is an example of send/receive pairs which pass an integer (message) around a ring of processes. Starting at process (rank) 0, the value "1" is passed to a neighbor process having a rank of one greater than the sender.

Explanation: The message status array is sized by the MPI_STATUS_SIZE parameter. Modulo functions determine the source and destination ranks for each process thus creating a ring of processes around which the message token proceeds. Process 0 sends an integer one to process 1, while all other processes post a receive and wait. In sequence, process 1 completes its receive and sends to process 2; next, process 2 completes its receive and sends to process 3; ...; process npes-1 completes its receive and sends to process 0; process 0 completes its receive and the ring passing is complete. The diagram at the end shows a convenient notation to make a time-line of events.

#include  <mpi.h>
#include  <stdio.h>
#include  <stdlib.h>

main(int argc, char **argv){
   MPI_Status status;
   int npes, irank, ierr;
   int i, j, ides, isrc;
   MPI_Comm IWCOMM = MPI_COMM_WORLD;
   int one = 1;

/*  Send message around ring ( 0-> 1-> 2 ... NPES-1-> 0 )*/

   ierr = MPI_Init(&argc, &argv);
   ierr = MPI_Comm_size(IWCOMM, &npes);
   ierr = MPI_Comm_rank(IWCOMM, &irank);

                  /* Determine destinations & sources  */

   ides = (irank+1     )%npes;
   isrc = (irank+npes-1)%npes;
   printf(" rank: %d  ides: %d  isrc: %d\n",irank,ides,isrc);

   /* Start Sending from task 0, and wait (recv) for count. */


   if(irank == 0){
      ierr =  MPI_Send(&one,1,MPI_INT, ides,9,IWCOMM);
      ierr =  MPI_Recv(&j  ,1,MPI_INT, isrc,9,IWCOMM, &status);
   }
   else{
      ierr =  MPI_Recv(&j  ,1,MPI_INT, isrc,9,IWCOMM, &status);
      i=j+1;
      ierr =  MPI_Send(&i  ,1,MPI_INT, ides,9,IWCOMM);
   }

   if(irank == 0) printf("I counted %d processors\n",j);
   ierr = MPI_Finalize();
}

OUTPUT (4 PEs):
 rank: 0  ides: 1  isrc: 3
 rank: 1  ides: 2  isrc: 0
 rank: 2  ides: 3  isrc: 1
 rank: 3  ides: 0  isrc: 2

******Sequence of events for processes 0,1,...,j,k.

      Event  Time -->
      |
      V

      0->1	k)-0									k->)-0
      0)-1	0->)-1, 1->2
      1)-2			1->)-2, 2->3
      2)-3					2->)-3, 3->4
...				...
      j)-k							j->)-k, k->0

      Definitions: 
          x->y   Send    posted on process x; sending   to   process y
          y)-x   Receive posted on process x; expecting from process y
          y->)-x Send/Receive pair completed on process x