The \irecv{} library provides asynchronous communication mode for 
use in chare arrays in the message-passing style. The control flow of a 
message-passing program is broken into two parts (to provide the efficient 
"split phase" structure): nonblocking communication operations and 
iwait with callback functions as continuations. It thus provides a style 
that MPI programmers may find intuitive. This library aids in porting
existing MPI codes to \charmpp{} without using expensive context-switching 
of threads. In this approach, a chare array element is used to
represent a virtual processor. 

There are three functions in \irecv{} library.

\function{void send(buf, size, dest, tag, refno)}
\args{void *buf;}
\args{int size, dest, tag, refno;}
\desc{
Sends message which is pointed by \uw{buf} to another array element whose 
index is specified by \uw{dest} with \uw{tag} and \uw{refno}.
\uw{buf} is a message buffer containing the data to be sent; \uw{size} is 
the total size of the message in bytes. 
Like in {\tt MPI\_send}, the \uw{tag} is used for matching message on 
destination array element. The integer \uw{refno} is a reference number, 
usually the iteration number.
}
  
\function{void irecv(buf, size, source, tag, refno)}
\args{void *buf;}
\args{int size, source, tag, refno;}
\desc{
This function registers \uw{tag} and \uw{buf} with the library. 
When the desired message arrives, it copies the matching message
into the location given by the \uw{buf}.
}

\function{void iwaitAll(f, data, refno)}
\args{recvCallBack f;}
\args{void *data;}
\args{int refno;}
\desc{
This function registers a callback function \uw{f} with the library. This
function is invoked with \uw{data} as its argument when all the previously issued \kw{irecv}s with \uw{refno} as reference number complete.
}

To use the \irecv{} library, first one has to create a chare array, which 
is inherited from class \kw{receiver}. The sender entry methor of the
chare array element prepares the message buffer and calls \kw{send} function 
to send message to another array element; The receiver specifies the 
matching tags and buffer to get the message. After \kw{irecv}, the receiver 
needs to call \kw{iwaitAll} function to wait for all the \kw{irecv} function 
calls to complete. However, \kw{iwaitAll} is a nonblocking function. 
The callback function will be called after the relevant \kw{irecv} calls 
complete.

Here is an example:

\begin{alltt}
     int size = 100;
     for (int i=0; i<size; i++) buf[i] = data[i];
     send(buf, size*sizeof(double), neighbor, tag, iter);
     irecv(buf, size*sizeof(double), neighbor, tag, iter);
     iwaitAll(callfunc, this, iter);
\end{alltt}

and callback function can be declared as:

\begin{alltt}
     void callfunc(void *obj)
     \{
       ... do something with obj
     \}
\end{alltt}
