CHARM++ Frequently Asked Questions
| Up: Other Manuals |
For answers to questions not on this list, please contact us at ppl@cs.uiuc.edu
Charm++ is a runtime library to let C++ objects communicate with each other efficiently. The programming model is thus very much like CORBA, Java RMI, or RPC; but it is targeted towards tightly coupled, high-performance parallel machines. It uses the ``single program, multiple data'' (SPMD) programming model made popular by MPI.
Charm++ has demonstrated scalability up to thousands of processors, and provides extremely advanced load balancing and object migration facilities.
No.
Charm++ is used to write ``explicitly parallel'' programs-we don't have our own compiler, so we don't do automatic parallelization. We've found automatic parallelization useful only for a small range of very regular numerical applications.
However, you should not have to throw away your serial code; normally only a small fraction of a large program needs to be changed to enable parallel execution. In particular, Charm++'s support for object-oriented programming and high-level abstractions such as Charm++ Arrays make it simpler and more expressive than many other parallel languages. So you will have to write some new code, but not as much as you might think. This is particularly true when using one of the Charm++ frameworks.
Charm++ provides several extremely sophisticated features, such as application-independent object migration, that are very difficult to provide in MPI. If you have a working MPI code but have scalability problems because of dynamic behavior, load imbalance, or communication costs, Charm++ might dramatically improve your performance. You can even run your MPI code on Charm++ unchanged using AMPI.
Yes.
Charm++ supports both shared-memory and distributed-memory machines, SMPs and non-SMPs. In particular, we support serial machines, Windows machines, clusters connected via Ethernet, Myrinet or Infiniband, IBM SP series and BlueGene, Cray XT series, and any machine that supports MPI or SHMEM. We normally do our development on Linux workstations, and our testing on large parallel machines. Programs written using Charm++ will run on any supported machine.
No one has ported Charm++ to a vector supercomputer, but it should be possible.
Yes.
The large, production-quality molecular dynamics application NAMD
is built on Charm++.
The Center for Simulation of Advanced Rockets
has a large physical simulation code built using Charm++.
We have significant collaborations with groups in Materials Science,
Chemistry, and Astrophysics in Illinios, New York, and Washington.
Prof. L.V. Kale, of the Computer Science Department of the University of Illinois at Urbana-Champaign, and his research group, the Parallel Programming Lab. Nearly a hundred people have contributed something to the project over the course of aproximately 15 years; a partial list of contributors appears in the people's page.
Our research group of approximately twenty people are actively engaged in maintaining and extending Charm++; and in particular the Charm++ frameworks. Several other groups are dependent on Charm++, so we expect to continue improving Charm++ indefinitely.
Charm++ is open-source and free for research, educational, and academic use. The University of Illinois retains the copyright to the software, and requires a license for any commercial redistribution of our software. The actual, legal license is included with Charm++ (in charm/LICENSE).
Our mailing list is ppl@cs.uiuc.edu and our Wiki is CharmWiki. We're always glad to get feedback on our software.
See our download page.
The developers of Charm++ routinely use the latest CVS versions, and most of the time this is the best case. Occasionally something breaks, but the CVS version will likely contain bug fixes not found in the releases.
Run the interactive build script ./build with no extra arguments If this fails, email ppl@cs.uiuc.edu with the problem. Include the build line used (this is saved automatically in smart-build.log)
If you have a very unusual machine configuration, you will have to run ./build -help to list all possible build options. You will then choose the closest architecture, and then you may have to modify the associated conf-mach.sh and conv-mach.h files in src/arch to point to your desired compilers and options. If you develop a significantly different platform, send the modified files to ppl@cs.uiuc.edu so we can include it in the distribution.
Run the interactive build script ./build and choose the option for building ``Charm++, AMPI, ParFUM, FEM and other libraries''.
For the net versions, you need to write a nodelist file which lists all the machine hostnames available for parallel runs.
For the MPI version, you need to set up an MPI configuration for available machines as for normal MPI applications.
You need to set up your .ssh/authorized_keys file correctly. Setup no-password logins using ssh by putting the correct host key (ssh-keygen) in the file .ssh/authorized_keys.
Finally, in the .nodelist file, you specify the shell to use for remote execution of a program using the keyword ++shell.
Yes. Some of the known working serial libraries include:
Try
Parallel objects using "Asynchronous Remote Method Invocation":
Entry methods are all the methods of a chare where messages can be sent by other chares. They are declared in the .ci files, and they must be defined as public methods of the C++ object representing the chare.
No! This is one of the biggest differences between Charm++ and most other ``remote procedure call'' systems like CORBA, Java RMI, or RPC. ``Invoke an asynchronous method'' and ``send a message'' have exactly the same semantics and implementation. Since the invoking method does now wait for the remote method to terminate, it normally cannot receive any return value. (see later for a way to return values)
Asynchronous method invocation is more efficient because it can be implemented as a single message send. Unlike with synchronous methods, thread blocking and unblocking and a return message are not needed.
Another big advantage of asynchronous methods is that it's easy to make things run in parallel. If I execute:
Yes. If you want synchronous methods, so the caller will block, use the [sync] keyword before the method in the .ci file. This requires the sender to be a threaded entry method, as it will be suspended until the callee finishes. Sync entry methods are allowed to return values to the caller.
A threaded entry method is an entry method for a chare that executes in a separate user-level thread. It is useful when the entry method wants to suspend itself (for example, to wait for more data). Note that threaded entry methods have nothing to do with kernel-level threads or pthreads; they run in user-level threads that are scheduled by Charm++ itself.
In order to make an entry method threaded, one should add the keyword threaded withing square brackets after the entry keyword in the interface file:
The usual way to get data back to your caller is via another invocation in the opposite direction:
The above example is very non-modular, because b has to know that a called it, and what method to call a back on. For this kind of request/response code, you can abstract away the ``where to return the data'' with a CkCallback object:
There are a few reasons for that:
Each processor executes the following operations strictly in order:
This implies that you can assume that the previous steps has completely finished before the next one starts, and any side effect from all the previous steps are committed (and can therefore be used).
Inside a single step there is no order guarantee. This implies that, for example, two groups allocated from mainchare can be instantiated in any order. The only exception to this is processor zero, where chare objects are instantiated immediately when allocated in the mainchare, i.e if two groups are allocated, their order is fixed by the allocation order in the mainchare constructing them. Again, this is only valid for processor zero, and in no other processor this assumption should be made.
To notice that if array elements are allocated in block (by specifying the number of elements at the end of the ckNew function), they are all instantiated before normal execution is resumed; if manual insertion is used, each element can be constructed at any time on its home processor, and not necessarily before other regular communication messages have been delivered to other chares (including other array elements part of the same array).
A proxy is a local C++ class that represents a remote C++ class. When you invoke a method on a proxy, it sends the request across the network to the real object it represents. In Charm++, all communication is done using proxies.
A proxy class for each of your classes is generated based on the methods you list in the .ci file.
Proxies can be:
This will not compile, because a CProxy_A is not an A. What you want is CProxy_A *ap = new CProxy_A(handle).
You can include the def.h file once you've actually declared everything it will reference- all your chares and readonly variables. If your chares and readonlies are in your own header files, it is legal to include the def.h right away.
However, if the class declaration for a chare isn't visible when you include the def.h file, you'll get a confusing compiler error. This is why we recommend including the def.h file at the end.
Make the global variable ``readonly'' by declaring it in the .ci file. Remember also that read-onlies can be safely set only in che mainchare constructor. Any change after the mainchare constructor has finished will be local to the processor that made the change. To change a global variable later in the program, every processor must modify it accordingly (e.g by using a chare group. Note that chare arrays are not guaranteed to cover all processors)
One can have class-static variables as read-onlies. Inside a chare, group or array declaration in the .ci file, one can have a readonly variable declaration. Thus:
You then refer to the variable in your program as someChare::someGroup.
You can use CkWallTimer() to determine the time on some particular processor. To time some parallel computation, you need to call CkWallTimer on some processor, do the parallel computation, then call CkWallTimer again on the same processor and subtract.
These are just like the standard C++ assert calls in <assert.h>- they call abort if the condition passed to them is false.
We use our own version rather than the standard version because we have to call CkAbort, and because we can turn our asserts off when CMK_OPTIMIZE is defined.
No.
There is no nice library to solve this problem, as some messages might be queued on the receiving processor, some on the sender, and some on the network. You can still:
Quiescence is When nothing is happening anywhere on the parallel machine.
A low-level background task counts sent and received messages. When, across the machine, all the messages that have been sent have been received, and nothing is being processed, quiescence is triggered.
Probably not.
In some ways, quiescence is a very strong property (it guarentees nothing is happening anywhere) so if some other library is doing something, you won't reach quiescence. In other ways, quiescence is a very weak property, since it doesn't guarentee anything about the state of your application like a reduction does, only that nothing is happening. Because quiescence detection is on the one hand so strong it breaks modularity, and on the other hand is too weak to guarentee anything useful, it's often better to use something else.
Often global properties can be replaced by much easier-to-compute local properties. For example, my object could wait until all its neighbors have sent it messages (a local property my object can easily detect by counting message arrivals), rather than waiting until all neighbor messages across the whole machine have been sent (a global property that's difficult to determine). Sometimes a simple reduction is needed instead of quiescence, which has the benefits of being activated explicitly (each element of a chare array or chare group has to call contribute) and allows some data to be collected at the same time. A reduction is also a few times faster than quiescence detection. Finally, there are a few situations, such as some tree-search problems, where quiescence detection is actually the most sensible, efficient solution.
At any given instant, you can call CkMyPe() to find out where you are. There is no reliable way to tell where another array element is; even if you could find out at some instant, the element might immediately migrate somewhere else!
Yes! Most of your computation should happen inside array elements. Arrays are the main way to automatically balance the load using one of the load balancers available.
To do load balancing, you need more than one array element per processor. To keep the time and space overheads reasonable, you probably don't want more than a few thousand array elements per processor. The optimal value depends on the program, but is usually between 10 and 100. If you come from an MPI background, this may seem like a lot.
You can reduce a set of data to a single value. For example, finding the sum of values, where each array element contributes a value to the final sum. Reductions are supported directly by Charm++ arrays, and some operations most commonly used are predefined. Other more complicated reductions can implement if needed.
You can have several reductions happen one after another; but you cannot mix up the execution of two reductions over the same array. That is, if you want to reduce A, then B, every array element has to contribute to A, then contribute to B; you cannot have some elements contribute to B, then contribute to A.
No. You only get load balancing if you explicitly ask for it either at link-time with the +balancer option, or at runtime with the -balancer option.
The migration constructor (a constructor that takes CkMigrateMessage * as parameter) is invoked when an array element migrates to a new processor. If there is anything you want to do when you migrate, you could put it here. However, even if you don't want to do anything, you must create it, as it is called from the code automatically generated by the Charm++ translator, and constructors aren't inherited, so we can't just put a migration constructor in the base class.
The migration constructor should not be declared in the .ci file. Of course the array element will require also at least one regular constructor so that it can be created, and these must be declared in the .ci file.
After sizing and packing a migrating array element, the array manager deletes the old copy. As long as all the array element destructors in the non-leaf nodes of your inheritance hierarchy are virtual destructors, with declaration syntax:
If not using usesAtSync, the load balancer can start up at anytime. There is a dummy AtSync for each array element which by default tells the load balancer that it is always ready. The LDBD manager has a syncer (LBDB::batsyncer) which periodically calls AtSync roughly every 1ms to trigger the load balancing (this timeout can be changed with the +LBPeriod option). In this load balancing mode, users have to make sure all migratable objects are always ready to migrate (e.g. not depending on a global variable which cannot be migrated).
You almost certaintly want to use AtSync directly. In most cases there are points in the execution where the memory in use by a chare is bigger due to transitory data, which does not need to be transferred if the migration happens at predefined points.
They are used for optimizations at the processor and node level respectively.
Probably not. People with an MPI background often overuse groups, which results in MPI-like Charm++ programs. Arrays should generally be used instead, because arrays can be migrated to acheive load balance.
Groups tend to be most useful in constructing communication optimization libraries. For example, all the array elements on a processor can contribute something to their local group, which can then send a combined message to another processor. This can be much more efficient than having each array element send a separate message.
Yes. Groups never migrate, so a local pointer is safe. The only caveat is to make sure you don't migrate without updating the pointer.
A local pointer can be used for very efficient access to data held by a group.
Migratable groups are declared so by adding the ``[migratable]'' attribute in the .ci file. They cannot migrate from one processor to another during normal execution, but only to disk for checkpointing purposes.
Migratable groups must declare a migration constructor (taking CkMigrateMessage * as a parameter) and a pup routine. The migration construtor must call the superclass migration constructor as in this example:
Almost certainly not. You should use arrays for most computation, and even quite low-level communication optimizations are often best handled by groups. Nodegroups are very difficult to get right.
There's one group element per processor (CkNumPes() elements); and one nodegroup element per node (CkNumNodes() elements). Because they execute on a node, nodegroups have very different semantics from the rest of Charm++.
Note that on a non-SMP machine, groups and nodegroups are identical.
Entries in node groups execute on the next available processor. Thus, if two messages were sent to a branch of a nodegroup, two processors could execute one each simultaneously.
No. They can be accessed by multiple threads at once.
Yes, which makes nodegroups different from everything else in Charm++.
If a nodegroup method accesses a data structure in a non-threadsafe way (such as writing to it), you need to lock it, for example using a CmiNodeLock.
A bundle of data sent, via a proxy, to another chare. A message is a special kind of heap-allocated C++ object.
It depends on the application. We've found parameter marshalling to be less confusing and error-prone than messages for small parameters. Nevertheless, messages can be more efficient, especially if you need to buffer incoming data, or send complicated data structures (like a portion of a tree).
You can't pass pointers across processors. This is a basic fact of life on distributed-memory machines.
You can, of course, pass a copy of an object referenced via a pointer across processors-either dereference the pointer before sending, or use a varsize message.
No. You must allocate messages with new.
Yes, or you will leak memory! If you receive a message, you are responsible for deleting it. This is exactly opposite of parameter marshalling, and much common practice. The only exception are entry methods declared as [nokeep]; for these the system will free the message automatically at the end of the method.
No, this will certainly corrupt both the message and the heap! Once you've sent a message, it's not yours any more. This is again exactly the opposite of parameter marshalling.
Variable-length messages can contain arrays of any type, both primitive type or any user-defined type. The only restriction is that they have to be 1D arrays.
No, this will certainly corrupt the heap! These arrays are allocated in a single contiguous buffer together with the message itself, and is deleted when the message is deleted.
Priorities are special values that can be associated with messages, so that the Charm++ scheduler will prefer higher priority messages when choosing a message to deliver. Priorities are respected by Charm++ as much as possible: until there are higher priority messages in the queue, lower priority message will never be delivered. Nevertheless, this is not a guarantee, and a lower priority message can be delivered before a higher priority one.
Messages with priorities are typically used to perform optimizations on the order of the computation.
For integer priorities, the smaller the priority value, the higher the priority of the message. Negative value are therefore higher priority than positive ones. To enable and set a message's priority there is a special new syntax and CkPriorityPtr function; see the manual for details. If no priority is set, messages have a default priority of zero.
The usual way: pup the size(s), allocate the array if unpacking, and then pup all the elements.
For example, if you have a 2D grid like this:
For the automatic allocation described in Automatic allocation via PUP::able of the manual, each class needs four things:
For most system- and user-defined structure someHandle, you want p|someHandle; instead of p(someHandle);
The reason for the two incompatible syntax varieties is that the bar operator can be overloaded outside pup.h (just like the std::ostream's operator<<); while the parenthesis operator can take multiple arguments (which is needed for efficiently PUPing arrays).
The bar syntax will be able to copy any structure, whether it has a pup method or not. If there is no pup method, the C++ operator overloading rules decay the bar operator into packing the bytes of the structure, which will work fine for simple types on homogenous machines. For dynamically allocated structures or heterogeneous migration, you'll need to define a pup method for all packed classes/structures. As an added benefit, the same pup methods will get called during parameter marshalling.
Structured Dagger is a structured notation for specifying intra-process control dependencies in message-driven programs. It combines the efficiency of message-driven execution with the explicitness of control specification. Structured Dagger allows easy expression of dependencies among messages and computations and also among computations within the same object using when-blocks and various structured constructs. See the Charm++ manual for the details.
There is an extra copy involved, because the AMPI message is reusable immediately after the AMPI call returns. Since Charm++ messages are to be handed over to the system, there is an extra copy involved (plus creation of a Charm++ message) while sending.
Wall time.
There are many ways to debug programs written in Charm++:
Currently charmdebug is tested to work only under net- versions. With other versions, testing is pending. The executable is present in java/bin/charmdebug of the Charm++ distribution. To start, simply substitute ``charmdebug'' to ``charmrun'':
Yes, on mpi- versions of Charm++. In this case, the program is a regular MPI application, and as such any tool available for MPI programs can be used. Notice that some of the internal data structures (like messages in queue) might be difficult to find.
It depends on the machine. On the net- versions of Charm++, like net-linux, you can just run the serial debugger:
If the problem only shows up in parallel, and you're running on an X terminal, you can use the ++debug or ++debug-no-pause options of charmrun to get a separate window for each process:
First, make sure the program at least starts to run properly without ++debug (i.e. charmrun is working and there are no problems with the program startup phase). You need to make sure that gdb or dbx, and xterm are installed on all the machines you are using (not the one that is running charmrun). If you are working on remote machines from Linux, you need to run ``xhost +'' locally to give the remote machines permission to display an xterm on your desktop. If you are working from a Windows machine, you need an X-win application such as exceed. You need to set this up to give the right permissions for X windows. You need to make sure the DISPLAY environment variable on the remote machine is set correctly to your local machine. I recommend ssh and putty, because it will take care of the DISPLAY environment automatically, and you can set up ssh to use tunnels so that it even works from a private subnet(e.g. 192.168.0.8). Since the xterm is displayed from the node machines, you have to make sure they have the correct DISPLAY set. Again, setting up ssh in the nodelist file to spawn node programs should take care of that. If you are using rsh, you need to set DISPLAY in /.charmrunrc which will be read at start up time by each node program.
Printouts from different processors do not normally stay ordered. Consider the code:
Though you might expect this code to always print ``cause, effect'', you may get ``effect, cause''. This can only happen when the cause and effect execute on different processors, so cause's output is delayed.
If you pass the extra command-line parameter +syncprint, then CkPrintf actually blocks until the output is queued, so your printouts should at least happen in causal order. Note that this does dramatically slow down output.
Charm++ automatically flushes the print buffers every newline and at program exit. There is no way to manually flush the buffers at another point.
This isn't a bug in the C library, it's a bug in your program - you're corrupting the heap. Link your program again with -memory paranoid and run it again in the debugger. -memory paranoid will check the heap and detect buffer over- and under-run errors, double-deletes, delete-garbage, and other common mistakes that trash the heap.
It's very convenient to do your testing on one processor (i.e., with +p1); but there are several things that only happen on multiple processors.
A single processor has just one set of global variables, but multiple processors have different global variables. This means on one processor, you can set a global variable and it stays set ``everywhere'' (i.e., right here!), while on two processors the global variable never gets initialized on the other processor. If you must use globals, either set them on every processor or make them into readonly globals.
A single processor has just one address space, so you actually can pass pointers around between chares. When running on multiple processors, the pointers dangle. This can cause incredibly weird behavior - reading from uninitialized data, corrupting the heap, etc. The solution is to never, ever send pointers in messages - you need to send the data the pointer points to, not the pointer.
The group it is refering to is the chare group. This error is often due to using an uninitialized proxy or handle; but it's possible this indicates severe corruption. Run with ++debug and check it you just sent a message via an uninitialized proxy.
You are trying to use code from a module that has not been properly initialized.
So, in the .ci file for your mainmodule, you should add an ``extern module'' declaration:
This means that the node program died without informing charmrun about it, which typically means a segmentation fault while in the interrupt handler or other critical communications code. This indicates severe corruption in Charm++'s data structures, which is likely the result of a heap corruption bug in your program. Re-linking with -memory paranoid may clarify the true problem.
Bus Error and Hangup both are indications that your program is terminating abnormally, i.e. with an uncaught signal (SEGV or SIGBUS). You should definitely run the program with gdb, or use ++debug.
No, and there are no plans.
Charm++ supports MPI and uses it as communication library. We have tested on MPICH and LAM, and also some other MPI variants for example MPI-GM, MPI/VMI. Charm++ also has explicit support for SMP nodes in MPI version. Charm++ hasn't been ported to use OpenMP.
Depends. Hopefully, the porting only involves fixing compiler compatibility issues. But porting to new platforms with specific communication libraries usually involve porting Converse threads and writing Converse communication layers. Charm++, which is on top of Converse, should be free of architecture dependent porting issues.
The source is always available, and you're welcome to make it run anywhere. Any kind of UNIX, Windows, and MacOS machine should be straightforward: just a few modifications to charm/src/arch/.../conv-mach.h (for compiler issues) and possibly a new machine.c (if there's a new communication system involved). However, porting to a Lisp machine or VAX would be fairly difficult.
Charm++/Converse has been ported to most UNIX and Linux OS, Windows, and MacOS.
Charm++ itself it fully portable, and should provide exactly the same interfaces everywhere (even if the implementations are sometimes different). Still, it's often harder than we'd like to port user code to new machines.
Many parallel machines have old or weird compilers, and sometimes a strange operating system or unique set of libraries. Hence porting code to a parallel machine can be suprisingly difficult.
Unless you're absolutely sure you will only run your code on a single, known machine, we recommend you be very conservative in your use of the language and libraries. ``But it works with my gcc!'' is often true, but not very useful.
Things that seem to work well everywhere include:
Our suggestions for Charm++ developers are:
Our suggestions for Charm++ developers are:
So use this:
Luckily, C++ support, especially STL support, is improving rapidly. Hopefully soon several of the above features will become widely supported enough to use everywhere.
Fortran compilers ``mangle'' their routine names in a variety of ways. g77 and most compilers make names all lowercase, and append an underscore, like ``foo_''. The IBM xlf compiler makes names all lowercase without an underscore, like ``foo''. Absoft f90 makes names all uppercase, like ``FOO''.
If the Fortran compiler expects a routine to be named ``foo_'', but you only define a C routine named ``foo'', you'll get a link error (``undefined symbol foo_''). Sometimes the UNIX command-line tool nm (list symbols in a .o or .a file) can help you see exactly what the Fortran compiler is asking for, compared to what you're providing.
Charm++ automatically detects the fortran name mangling scheme at configure time, and provides a C/C++ macro ``FTN_NAME'', in ``charm-api.h'', that expands to a properly mangled fortran routine name. You pass the FTN_NAME macro two copies of the routine name: once in all uppercase, and again in all lowercase. The FTN_NAME macro then picks the appropriate name and applies any needed underscores. ``charm-api.h'' also includes a macro ``FDECL'' that makes the symbol linkable from fortran (in C++, this expands to extern ``C''), so a complete Fortran subroutine looks like in C or C++:
This same syntax can be used for C/C++ routines called from fortran, or for calling fortran routines from C/C++. We strongly recommend using FTN_NAME instead of hardcoding your favorite compiler's name mangling into the C routines.
If designing an API with the same routine names in C and Fortran, be sure to include both upper and lowercase letters in your routine names. This way, the C name (with mixed case) will be different from all possible Fortran manglings (which all have uniform case). For example, a routine named ``foo'' will have the same name in C and Fortran when using the IBM xlf compilers, which is bad because the C and Fortran versions should take different parameters. A routine named ``Foo'' does not suffer from this problem, because the C version is ``Foo, while the Fortran version is ``foo_'', ``foo'', or ``FOO''.
Fortran and C have rather different parameter-passing conventions, but it is possible to pass simple objects back and forth between Fortran and C:
| C/C++ Type | Fortran Type |
|---|---|
| int | INTEGER, LOGICAL |
| double | DOUBLE PRECISION, REAL*8 |
| float | REAL, REAL*4 |
| char | CHARACTER |
Converse is the low-level portable messaging layer that Charm++ is built on, but you don't have to know anything about Converse to use Charm++. You might want to learn about Converse if you want a capable, portable foundation to implement a new parallel language on.
drand48 is nonportable and woefully inadequate for any real simulation task. Even if each processor seeds drand48 differently, there is no guarantee that the streams of pseduo-random numbers won't quickly overlap. A better generator would be required to ``do it right'' (See Park & Miller, CACM Oct. 88).
Converse provides a 64-bit pseudorandom number generator based on the SPRNG package originally written by Ashok Shrinivasan at NCSA. For detailed documentation, please take a look at the Converse Extensions Manual on the Charm++ website. In short, you can use CrnDrand() function instead of the unportable drand48() in Charm++.
All the Charm++ core source code is soft-linked into the charm/<archname>/tmp directory when you run the build script. The libraries and frameworks are under charm/<archname>/tmp/libs, in either ck-libs or conv-libs.
cd into the charm/<archname>/tmp directory and make. If you want to compile only a subset of the entire set of libraries, you can specify it to make. For example, to compile only the Charm++ RTS, type make charm++.
This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -white -antialias -local_icons -long_titles 1 -show_section_numbers -top_navigation -address '
November 07, 2009
Charm Homepage' -split 0 manual.tex
The translation was initiated by root on 2009-11-07
November 07, 2009
Charm Homepage