Live Webcast 15th Annual Charm++ Workshop

A low-diameter, fast interconnection network is going to be a pre-requisite for building exascale machines. A two-level direct network has been proposed by several groups as a scalable design for future machines. IBM's PERCS topology and the dragonfly network discussed in the DARPA exascale hardware study are examples of this design. The presence of multiple levels in this design leads to hot-spots on a few links for applications where grouping processes together is optimal for minimizing total communication volume. This is especially true for communication graphs with a small number of neighbors per task. Routing and mapping choices can impact the communication behavior of parallel applications. In this paper, we explore intelligent topology aware mappings for different communication patterns on the physical topology to identify cases which minimize link utilization. We also analyze the trade-offs between using direct and indirect routing with different mappings. We use simulations to study communication and overall performance of applications on two-level direct networks since there are no installations of such networks yet. This study raises interesting issues regarding choice of job scheduling, routing and mapping for future machines.