2.3 Running a BigSim Application

To run a parallel BigSim application, CHARM++ provides a utility program called charmrun that starts the parallel execution. For detailed description on how to run a CHARM++ application, refer to the file charm/README in the source code distribution.

To run a BigSim application, one needs to specify the following parameters to charmrun to define the simulated machine size:

1. +vp: define the number of processors of the hypothetical (future) system
2. +x, +y and +z: optionally define the size of the machine in three dimensions, these define the number of nodes along each dimension of the machine (assuming a torus/mesh topology);
3. +wth and +cth: For one node, these two parameters define the number of worker processors (+wth) and the number of communication processors (+cth).
4. +bglog: used so that BigSim logfiles get created
5. +bgcorrect: starts the simulation mode for performance prediction. Otherwise the program runs without doing parallel event simulation for performance prediction of the application.
6. +bgwalltime: used only in simulation mode, when specified, use wallclock measurement of the time taken on the simulating machine to estimate the time it takes to run on the target machine.
7. +bgcounter: used only in simulation mode, when specified, use the performance counter to estimate the time on target machine. This is currently only supported when perfex is installed, like Origin2000.

For example, to simulate a parallel machine of size 64K as 40x40x40, with one worker processor and one communication processor on each node, and use 100 real processors to run the simulation, the command to be issued should be:

./charmrun +p100 ./hello +x40 +y40 +z40 +cth1 +wth1

To run an AMPI program, one may also want to specify the number of virtual processors to run the MPI code by using +vp. As an example,

./charmrun +p100 ./hello +x40 +y40 +z40 +cth1 +wth1 +vp 128000

starts the simulation of a machine of size 40x40x40 with one worker processor in each node, running 128000 MPI tasks (2 MPI tasks on each node), using 100 real processors to run the simulation. In this case, MPI_Comm_size() returns 128000 for MPI_COMM_WORLD. If the +vp option is not specified, the number of virtual processors will be equal to the number of worker processors of the simulated machine, in this case 64000.