MPIIO Configuration

BlueGene MPIIO configuration and performance tuning.

Used by ad_bgl and ad_bglockless ADIO's.

Hints

• bgl_nodes_pset - Specify how many aggregators to use per pset. This hint will override the cb_nodes hint based on BlueGene psets.
  • N - Use N nodes per pset as aggregators.
  • Default is based on partition configuration and cb_nodes.

The following default key/value pairs may differ from other platform defaults.

• key = cb_buffer_size value = 16777216
• key = romio_cb_read value = enable
• key = romio_cb_write value = enable
• key = ind_rd_buffer_size value = 4194304
• key = ind_wr_buffer_size value = 4194304

Environment Variables

• BGLMPIO_COMM - Define how data is exchanged on collective reads and writes. Possible values:
  • 0 - Use MPI_Alltoallv.
  • 1 - Use MPI_Isend/MPI_Irecv.
  • Default is 0.

• BGLMPIO_TIMING - collect timing breakdown for MPI I/O collective calls. Must also compile the library with BGL_PROFILE defined. Possible values:
  • 0 - Do not collect/report timing.
  • 1 - Collect/report timing.
  • Default is 0.

• BGLMPIO_TIMING2 - collect additional averages for MPI I/O collective calls. Must also compile the library with BGL_PROFILE defined. Possible values:
  • 0 - Do not collect/report averages.
  • 1 - Collect/report averages.
  • Default is 0.

• BGLMPIO_TUNEGATHER - Tune how starting and ending offsets are communicated for aggregator collective i/o. Possible values:
  • 0 - Use two MPI_Allgather's to collect starting and ending offsets.
  • 1 - Use MPI_Allreduce(MPI_MAX) to collect starting and ending offsets.
  • Default is 1.

• BGLMPIO_TUNEBLOCKING - Tune how aggregate file domains are calculated (block size). Possible values:
  • 0 - Evenly calculate file domains across aggregators. Also use MPI_Isend/MPI_Irecv to exchange domain information.
  • 1 - Align file domains with the underlying file system's block size. Also use MPI_Alltoallv to exchange domain information.
  • Default is 1.

• BGLOCKLESSMPIO_F_TYPE - Specify a filesystem type that should run the ad_bglockless driver. NOTE: Using romio prefixes (such as "bgl:" or "bglockless:") on a file name will override this environment variable. Possible values:
  • 0xnnnnnnnn - Any valid file system type (or "magic number") from statfs() field f_type.
  • The default is 0x20030528 (PVFS2_SUPER_MAGIC)