\subsection{Modules}

\subsubsection{Structure of a \charmpp\ Program}

A \charmpp\ program is structurally similar to a \CC{} program.  Most of a
\charmpp\ program {\em is} \CC{} code \footnote{\bf Constraint: The \CC{} code
cannot, however, contain global or static variables.}. The main syntactic units
in a \charmpp\ program are class definitions. A \charmpp\ program can be
distributed across several source code files.

There are five disjoint categories of objects (classes) in \charmpp:

\begin{itemize}
\item Sequential objects: as in \CC{}
\item Chares (concurrent objects) \index{chare}
\item Chare Groups \index{chare groups} (a form of replicated objects)
\index{group}
\item Chare Arrays \index{chare arrays} (an indexed collection of chares)
\index{array}
\item Messages (communication objects)\index{message}
\end{itemize}

The user's code is written in \CC{} and interfaces with the \charmpp\ system as
if it were a library containing base classes, functions, etc.  A translator is
used to generate the special code needed to handle \charmpp\ constructs.  This
translator generates \CC{} code that needs to be compiled with the user's code.

Interfaces to the \charmpp\ objects (such as messages, chares, readonly
variables etc.) \index{message}\index{chare}\index{readonly} have to be
declared in \charmpp\ interface files. Typically, such entities are grouped
\index{module} into {\em modules}. A \charmpp\ program may consists of multiple
modules.  One of these modules is declared to be a \kw{mainmodule}. All the
modules that are ``reachable'' from the \kw{mainmodule} via the \kw{extern}
construct are included in a \charmpp\ program.

The \charmpp\ interface file has the suffix ``.ci''.  The \charmpp\ interface
translator parses this file and produces two files (with suffixes ``.decl.h''
and ``.def.h'', {\em for each modules declared in the ``.ci'' file}), that
contain declarations (interface) and definitions (implementation)of various
translator-generated entities. If the name of a module is \uw{MOD}, then the
files produced by the \charmpp\ interface translator are named \uw{MOD.decl.h}
and \uw{MOD.def.h}\footnote{Note that the interface file for module \uw{MOD}
need not be named \uw{MOD.ci}. Indeed one ``.ci'' file may contain interface
declarations for multiple modules, and the translator will produce one pair of
declaration and definition files for each module.}.  We recommend that the
declarations header file be included at the top of the header file (\uw{MOD.h})
for module \uw{MOD}, and the definitions file be included at the bottom of the
code for module (\uw{MOD.C}) \footnote{In the earlier version of interface
translator, these files used to be suffixed with ``.top.h'' and ``.bot.h'' for
this reason.}.

A simple \charmpp\ program is given below:

\begin{alltt}
///////////////////////////////////////
// File: pgm.ci

mainmodule Hello \{
  readonly CkChareID mid;
  mainchare HelloMain \{
    entry HelloMain(); // implicit CkArgMsg * as argument
    entry void Wait(void);
  \};
  group HelloGroup \{
    entry HelloGroup(void);
  \};
\};

////////////////////////////////////////
// File: pgm.h
#include "Hello.decl.h" // Note: not pgm.decl.h

class HelloMain: public Chare \{
  public:
    HelloMain(CkArgMsg *);
    void Wait(void);
  private:
    int count;
\};

class HelloGroup: public Group \{
  public:
    HelloGroup(void);
\};

/////////////////////////////////////////
// File: pgm.C
#include "pgm.h"

CkChareID mid;

HelloMain::HelloMain(CkArgMsg *msg) \{
  delete msg;
  count = 0;
  mid = thishandle;
  CProxy_HelloGroup::ckNew(); // Create a new "HelloGroup"
\}

void HelloMain::Wait(void) \{
  count++;
  if (count == CkNumPes()) \{ // Wait for all group members to finish the printf
    CkExit();
  \}
\}

HelloGroup::HelloGroup(void) \{
  ckout << "Hello World from processor " << CkMyPe() << endl;
  CProxy_HelloMain pself(mid);
  pself.Wait();
\}

#include "Hello.def.h" // Include the Charm++ object implementations

/////////////////////////////////////////
// File: Makefile

pgm: pgm.ci pgm.h pgm.C
      charmc -c pgm.ci
      charmc -c pgm.C
      charmc -o pgm pgm.o -language charm++

\end{alltt}

\uw{HelloMain} is designated a \kw{mainchare}. Thus the Charm Kernel starts
execution of this program by creating an instance of \uw{HelloMain} on
processor 0. The HelloMain constructor creates a chare group \uw{HelloGroup},
and stores a handle to itself and returns. The call to create the group returns
immediately after directing Charm Kernel to perform the actual creation and
invocation.  Shortly after, the Charm Kernel will create an object of type
\uw{HelloGroup} on each processor, and call its constructor. The constructor
will then print ``Hello World...'' and then call the \uw{Wait} method of
\uw{HelloMain}. The Wait method calls CkExit() after all group members have
called it (i.e., they have finished printing ``Hello World...''), and the
\charmpp program exits.

\subsubsection{Functions in the ``decl.h'' and ``def.h'' files}

The \texttt{decl.h} file provides declarations for the proxy classes of the
concurrent objects declared in the ``.ci'' file (from which the \texttt{decl.h}
file is generated). So the \uw{Hello.decl.h} file will have the declaration of
the class CProxy\_HelloMain. Similarly it will also have the declaration for
the HelloGroup class. 

This class will have functions to create new instances of the chares and
groups, like the function \kw{ckNew}. For \uw{HelloGroup} this function creates
an instance of the class \uw{HelloGroup} on all the processors. 

The proxy class also has functions corresponding to the entry methods defined
in the ``.ci'' file. In the above program the method wait is declared in
\uw{CProxy\_HelloMain} (proxy class for \uw{HelloMain}).

The proxy class also provides static registration functions used by the
\charmpp{} runtime.  The \texttt{def.h} file has a registration function
(\uw{\_\_registerHello} in the above program) which calls all the registration
functions corresponding to the readonly variables and entry methods declared in
the module.

