Traditional MPI runtime systems assume a single-threaded process on every processor, which will block the entire processor on a blocking receive call. By modelling MPI processes with Converse's user-level threads, parallel components written using MPI can be made to co-exist with other components in a Charisma-based parallel application. Adaptive MPI (AMPI), our implementation of MPI on top of Converse, uses Converse's user-level threads to run the component code. AMPI design is similar to our earlier efforts ``Threaded Simple Messaging'' (tSM), ``Parallel Array of Threads'' (PATH), and ``Threaded Message Passing Objects'' (TeMPO). In particular, the PATH library on Converse aimed at providing Chant-like [29] abstraction of a group of threads. One critical difference between these earlier efforts and AMPI is that AMPI was built from scratch with the main objective of providing easier migration path for legacy codes. Therefore, message-passing primitives of AMPI have the same syntax and semantics as MPI, which makes it simpler to adapt existing components written in MPI to run on our component architecture.
There are two issues that need to be addressed in a multi-threaded implementation of MPI. First is the overhead associated with thread context-switching and synchronizations. We have demonstrated that Converse's user-level threads have very low overheads. Even these low overheads can often be compensated for by the cache-performance benefits of a technique called ``overdecomposition'' or ``multipartitioning''. The second issue arises while converting existing MPI components to AMPI. Since the original single threaded MPI component codes need to co-exist with other instances of the component running the same code on the same processor, one has to make sure that the MPI component does not reference any writable global variables.
We discuss these issues in detail next.