This section shows the implementation of CSM, a message-passing library. CSM was designed for illustration purposes. So, it is intentionally the simplest possible library that implements message-passing with threads. The basic design shown here can be used to implement any message-passing library, including MPI or PVM. The following function descriptions are from the CSM manual:
void CsmTSend(int pe, int tag, char *buffer, int size)
A message is sent to the given processor pe containing size bytes of data from buffer, and tagged with the given tag. The calling thread continues after depositing the message with the runtime system.
int CsmTRecv(int tag, char *buffer, int size, int *rtag)
Waits until a message with a matching tag is available, and copies it into the given buffer. A wild card value, CsmWildCard, may be used for the tag. In this case, any available message is considered a matching message. The tag with which the message was sent is stored in the location to which rtag points. The number of bytes in the message is returned.
Our implementation buffers messages on the destination processor. To implement this using Converse, two major data structures are needed. First, each processor needs a ``message table'' containing messages that were sent, but for which no CsmTRecv call has been issued yet. Second, each processor needs a ``thread table'' containing threads that are waiting for messages, indexed by the tags that they're waiting for. Given these data structures, the send and receive functions are implemented as follows.
CsmTSend creates a Converse message containing the user data and the tag. It configures the message to invoke the function CsmTHandler. CsmTSend then transmits a copy of this message to the destination processor. When the message arrives, the target processor calls CsmTHandler, passing it a pointer to the message (which contains the user data and tag). CsmTHandler takes the user data and tag, and inserts it into the local message table. It then checks the thread table to see if any thread was waiting for the message. If so, that thread is awakened.
When a thread calls CsmTRecv, it looks in the message table, and if a matching message is already there, it is extracted and returned. If not, CsmTRecv obtains its own thread ID, and inserts itself into the thread table. It then puts itself to sleep. When it wakes up, it knows it has been awakened by CsmTHandler. It retrieves the message from the message table, and returns it.
For the message and thread tables, we used an off-the-shelf table object provided by Converse (the ``message manager''). Thus, our data structures were already available to us. The thread functions were provided, as was the messaging. We had to design the format of the CSM messages (header, then tag, then user data), write the subroutines shown above, and declare and initialize the tables. In all, this took about 100 lines (2 pages) of code. This is interesting, as the message-passing model we implemented is significantly different from the underlying message-driven model of Converse.
Notice that no explicit action was needed to keep CSM from interfering with other libraries also implemented on top of Converse. CSM messages, when they arrive, trigger changes to the CSM data structures. They have no other effect. If a library system does not explicitly monitor the CSM data structures, it will not be aware that a CSM message arrived. In general, two libraries implemented on top of Converse do not notice each other's existence unless explicit action is taken to create interaction. This is in contrast to such systems as MPI, where each independent module must take explicit action (e.g., the creation of new communicator objects, etc) to avoid interfering with other modules.