3.2 CHARM++ Groups and Nodegroups

3.2.1 What are groups and nodegroups used for?

They are used for optimizations at the processor and node level respectively.

3.2.2 Should I use groups?

Probably not. People with an MPI background often overuse groups, which results in MPI-like Charm++ programs. Arrays should generally be used instead, because arrays can be migrated to acheive load balance.

Groups tend to be most useful in constructing communication optimization libraries. For example, all the array elements on a processor can contribute something to their local group, which can then send a combined message to another processor. This can be much more efficient than having each array element send a separate message.

3.2.3 Is it safe to use a local pointer to a group, such as from ckLocalBranch?

Yes. Groups never migrate, so a local pointer is safe. The only caveat is to make sure you don't migrate without updating the pointer.

A local pointer can be used for very efficient access to data held by a group.

3.2.4 What are migratable groups?

Migratable groups are declared so by adding the ``[migratable]'' attribute in the .ci file. They cannot migrate from one processor to another during normal execution, but only to disk for checkpointing purposes.

Migratable groups must declare a migration constructor (taking CkMigrateMessage * as a parameter) and a pup routine. The migration construtor must call the superclass migration constructor as in this example:

class MyGroup : public CBase_MyGroup {
  ...
  MyGroup (CkMigrateMessage *msg) : CBase_MyGroup(msg) { }
  ...
}

3.2.5 Should I use nodegroups?

Almost certainly not. You should use arrays for most computation, and even quite low-level communication optimizations are often best handled by groups. Nodegroups are very difficult to get right.

3.2.6 What's the difference between groups and nodegroups?

There's one group element per processor (CkNumPes() elements); and one nodegroup element per node (CkNumNodes() elements). Because they execute on a node, nodegroups have very different semantics from the rest of Charm++.

Note that on a non-SMP machine, groups and nodegroups are identical.

3.2.7 Do nodegroup entry methods execute on one fixed processor of the node, or on the next available processor?

Entries in node groups execute on the next available processor. Thus, if two messages were sent to a branch of a nodegroup, two processors could execute one each simultaneously.

3.2.8 Are nodegroups single-threaded?

No. They can be accessed by multiple threads at once.

3.2.9 Do we have to worry about two entry methods in an object executing simultaneously?

Yes, which makes nodegroups different from everything else in Charm++.

If a nodegroup method accesses a data structure in a non-threadsafe way (such as writing to it), you need to lock it, for example using a CmiNodeLock.