Charm++

Charm++ [44] is a parallel message-driven object-oriented language. The basic unit of parallelism in Charm++ is a message-driven object (called a chare), which represents a medium-grained computation. A Charm++ program consists of a collection of chares that interact with each other by calling each other's methods. Unlike a sequential C++ object, a chare in Charm++ has a globally unique identifier, which is used to invoke methods on it from any processor. Methods of a chare that can be invoked from objects on remote processors are called entry methods. Charm++ programs are written in C++ with a few library calls. In addition, each Charm++ object has a published interface, described in Charm++ interface description language (IDL). The translator for Charm++ IDL generates proxy interfaces (proxies) to chares. Proxies simplify remote method invocation by providing a syntax familiar to C++ programmers. The proxy object for a chare contains methods with signatures identical to entry methods of the chare, and are instantiated with a chare handle. The generated code for the method of a proxy object marshals the parameters passed to it in a single contiguous message, and sends the message to the processor that hosts the remote object associated with the proxy. The Converse scheduler on the remote processor then invokes the actual object's entry method after unmarshaling the parameters. This is illustrated in figures 3.1, 3.2, 3.3, and 3.4. The interface to a chare is described in figure 3.1. The Charm++ interface translator generates the proxy class in figure 3.2. The actual chare class is defined in figure 3.3. Figure 3.4 contains code to invoke a chare's entry methods using the proxy object.

{Interface}
chare MyChare {
  entry MyChare();
  entry void EntryMethod1(int);
  entry void EntryMethod2(void);
  entry void EntryMethod3(Message *);
};

Figure 3.1: Chare interface description

{Proxy}
// Generated by Charm++ interface translator
class CProxy_MyChare {
  private:
    // instance data 
    CkChareID cid; // contains chare handle
  public:
    CProxy_MyChare() {
      cid = CkCreateChare(...);
    }
    void EntryMethod1 (int i) {
      MarshallMsg *m = CkCreateMessage(...);
      m->pack(i);
      CkSendMessage(cid, ..., m);
    }
    void EntryMethod2 (void) {
      MarshallMsg *m = CkCreateMessage(...);
      CkSendMessage(cid, ..., m);
    }
    void EntryMethod3 (Message *m) {
      CkSendMessage(cid, ..., m);
    }
};

Figure 3.2: Proxy class to MyChare generated from the interface description

{Chare}
class MyChare : public Chare {
  private:
    // object private data
  public:
    MyChare() { ... } // Constructor
    void EntryMethod1(int i) { ... } // a entry method
    void EntryMethod2(void) { ... } // another entry method
    void EntryMethod3(Message *m) { ... } // another entry method
};

Figure 3.3: Chare Definition

{RMI}
{
  // ....
  CProxy_MyChare * pc = new CProxy_MyChare();
  pc->EntryMethod1(345);
  pc->EntryMethod2();
  pc->EntryMethod3(new Message());
  // ....
}

Figure 3.4: Invoking a chare's method

In addition to chares, Charm++ provides an abstraction for collections of chares, called chare arrays. Chare arrays have a global identifier for the collection of chares of the same class, and each individual chare is addressed within this collection with a unique index. An array element index can be, but is not limited to, a single integer, a pair, or a triplet. In general, any pattern of bits could be used to index an array element. Collective operations such as broadcast and reduction can take place over an entire array efficiently [55]. A chare array is mapped to available processors keeping load balance among processors (assuming unit load for each array element), and simplifies migration for dynamic load balancing. A chare array's interface is described in terms of the interface to the individual chare array element. The Charm++ interface translator generates proxy class definitions similar to that of ordinary chares, so that methods on individual array elements can be remotely invoked. In addition, the translator generates proxy methods that invoke corresponding methods on all the elements of an array (similar to broadcasting a message.) Figure 3.5 shows the interface definition of a one dimensional chare array. Its implementation is shown in figure 3.6. Figure 3.7 shows the two ways of invoking methods on arrays. EntryMethod1 is invoked on an individual array element by explicitly naming it by its index, and EntryMethod2 is invoked on all elements of the chare array.

{Interface}
array [1D] MyArray {
  entry MyArray();
  entry void EntryMethod1(int);
  entry void EntryMethod2(void);
};

Figure 3.5: Chare array interface description

{Chare}
class MyArray : public ArrayElement1D {
  private:
    // object private data
  public:
    MyArray() { ... } // Constructor
    void EntryMethod1(int i) { ... } // a entry method
    void EntryMethod2(void) { ... } // another entry method
};

Figure 3.6: Chare array element Definition

{RMI}
{
  // ....
  int numElements = 25; // number of array elements

  // create the chare array
  CkArrayID aid = CProxy_MyArray::ckNew(numElements);

  // construct a proxy to the chare array
  CProxy_MyArray pa(aid);

  // invoke EntryMethod1 on array element 20
  pa[20].EntryMethod1(768);

  // invoke EntryMethod2 on all elements of the array
  pa.EntryMethod2();
  // ....
}

Figure 3.7: Invoking chare array's methods

A special type of chare array is an array indexed by processor number. This is called a chare group.^3.1 A chare group encapsulates chares exactly equal in number to the available processors, and each processor contains exactly one branch chare of the group. Elements of a chare group are nonmigratable, thus enabling several optimizations in chare group implementation. Chare groups can be used for low-level system tasks such as implementing collective communications, distributed tables [69] etc. In particular, they serve as parallel pathways for data exchange between Charm++ modules [43] as illustrated in section 3.6.

Charm++ can be considered to be an object-based counterpart of Converse, where handler functions are replaced by entry methods that execute in the context of an object. The object context provides encapsulation for entry methods, and maintains state across method invocation unlike Converse handlers. The objects also serve as a useful description of work units to the runtime system, especially the dynamic load balancing subsystem of Converse, since work can easily defined as computations performed by the entry methods of an object, and migration of work can be carried out at the object level. A Charm++ object does not assume ownership of the processor on which it resides, thus allowing the runtime system to concurrently interleave its execution with other components on the same processor. Also, Charm++ allows one to construct parallel components from sequential components with constructs such as chare arrays. Clients of the Charm++ components invoke component services through proxy objects, thus hiding the method of construction of the parallel component. Because of these qualities of Charm++ objects, we have chosen to build Charisma on top of Charm++ rather than directly on top of Converse. The existing interface model of Charm++ however, is an extension of the object model, and causes a number of limitations that prohibit it from possessing all the properties of the ideal interface model that we described earlier.