2.1 PROJECTIONS API for CHARM++ Applications

2.1.1 Selective Tracing

CHARM++ allows user to start/stop tracing the execution at certain points in time on the local processor. Users are advised to make these calls on all processors and at well-defined points in the application.

Users may choose to have instrumentation turned off at first (by command line option +traceoff - see section 2.3) if some period of time in middle of the applications execution is of interest to the user.

Alternatively, users may start the application with instrumentation turned on (default) and turn off tracing for specific sections of the application.

Again, users are advised to be consistent as the +traceoff runtime option applies to all processors in the application.

• void traceBegin()

  Enables the runtime to trace events (including all user events) on the local processor where traceBegin is called.

• void traceEnd()

  Prevents the runtime from tracing events (including all user events) on the local processor where traceEnd is called.

2.1.2 User Events

PROJECTIONS has a limited ability to visualize traceable user specified events. You can make use of the following API calls to do this. The general steps to do this are:

1. Register an event with an identifying string and either specify or acquire a globally unique event identifier.

2. Use the event identifier to specify trace points in your code of interest to you.

The functions available are as follows:

• int traceRegisterUserEvent(char* EventDesc, int EventNum=-1)

  This function registers a user event by associating EventNum to EventDesc. If EventNum is not specified, a globally unique event identifier is obtained from the runtime and returned.

  EventNum has to be the same on all processors. There are 3 ways to ensure that, and correspondingly 3 ways to call the function:

  1. Call traceRegisterUserEvent on node 0 in main::main without specifying an event number, and store returned event number into a readonly variable.
  2. Call traceRegisterUserEvent on all nodes without specifying an event number. The returned value is the common event number that is agreed on system-wide.
  3. Call traceRegisterUserEvent and specify the event number on processor 0. Doing this on other processors would have no effect. Afterwards, the event number can be used in the following user event calls.

  Eg. traceRegisterUserEvent("Time Step Begin", 10);

  Eg. eventID = traceRegisterUserEvent(``Time Step Begin'');

• void traceUserEvent(int EventNum)

  This function works like a marker, indicating an occurrence of a user-defined point event with the given EventNum.

  Eg. traceUserEvent(10);

• void traceUserBracketEvent(int EventNum, double StartTime, double EndTime)

  This function records an event interval or activity identified by EventNum that lasts the period of time between StartTime and EndTime. Both StartTime and EndTime can be obtained from function call CmiWallTimer() in seconds.

  Eg.

     traceRegisterUserEvent("Critical Code", 20);
     double critStart;  // times of start
     critStart = CmiWallTimer();
     // do the critical code
     traceUserBracketEvent(20, critStart,CmiWallTimer());
  

2.1.3 Function-level Tracing for Adaptive MPI Applications

Adaptive MPI (AMPI) is an implementation of the MPI interface on top of CHARM++. As with standard MPI programs, the appropriate semantic context for performance analysis is captured through the observation of MPI calls within C/C++/Fortran functions. Unfortunately, AMPI's implementation does not grant the runtime access to information about user function calls. As a result, the tracing framework must provide an explicit API for capturing this piece of performance information in addition to MPI calls (which are known to the runtime).

The functions, similar to those used to capture user events, are as follows:

• int _TRACE_REGISTER_FUNCTION_NAME(const char *name);
  

  This function registers an AMPI function name. The tracing framework assigns to function name a unique id and returns it. It is the user's responsibility to ensure that name is unique and consistent.

  This registration function should be called near the start of the application, just after MPI_Init.

• int _TRACE_REGISTER_FUNCTION_ID(const char *name, int idx);
  

  This function registers an AMPI function name to be associated explicitly to the id idx. It is the user's responsibility to ensure that name as well as idx are unique and consistent. The tracing framework returns idx.

  This is an alternative registration function and should be called near the start of the application just after MPI_Init.

• void _TRACE_BEGIN_FUNCTION_NAME(const char *name);
  

  This function tells the tracing framework to record a begin event associated with the registered function name. If this were called in a C/C++ environment with pre-processor support, the line number of the function call will also be recorded.

• void _TRACE_BEGIN_FUNCTION_ID(int idx);
  

  This function tells the tracing framework to record a begin event associated with the registered function indexed by idx. If this were called in a C/C++ environment with pre-processor support, the line number of the function call will also be recorded.

• void _TRACE_END_FUNCTION_NAME(const char *name);
  

  This function tells the tracing framework to record a end event associated with the registered function name.

• void _TRACE_END_FUNCTION_ID(int idx);
  

  This function tells the tracing framework to record a end event associated with the registered function indexed by idx.

AMPI function events captured by the use of this API are recognized by the visualization system and used for special AMPI-specific views in addition to standard CHARM++ entry methods.