Automating Topology Aware Task Mapping on Large Supercomputers




 Abstract:


Parallel computing is entering the era of petascale machines, which brings enormous computing power with it and new challenges to harness this power efficiently. Machines with hundreds of thousands of processors already exist, connected by complex interconnect topologies. Communication and specifically, network contention is becoming an increasingly important factor affecting overall performance.

The further different messages travel on the network, greater is the chance of resource sharing between messages and hence, of contention. Recent studies on IBM Blue Gene and Cray XT machines have shown that under contention, message latencies can be severely affected. Realizing this fact, application developers have started paying attention to the mapping of tasks to physical processors to minimize contention. Placement of communicating tasks on nearby physical processors can minimize the distance traveled by messages and reduce the chances of contention.

This dissertation proposes algorithms and techniques for automatic mapping of parallel applications to relieve the application developers of this burden. The effect of contention on message latencies is studied in depth to guide the design of mapping algorithms. The {\em hop-bytes} metric is proposed for the evaluation of mapping algorithms and proved to be a better metric than the previously used {\em maximum dilation} metric. The main focus of this dissertation is on developing topology aware mapping algorithms for parallel applications with regular and irregular communication patterns. These algorithms along with pattern matching algorithms for communication graphs of the applications form the automatic mapping framework suitable for most parallel applications.


 PhD Committee:


Prof. Laxmikant V. Kale (Chair and Director of Dissertation Research), Professor of Computer Science and Director, Parallel Programming Laboratory, UIUC
Prof. David A. Padua, Donald Biggar Willett Professor of Computer Science, UIUC
Prof. William D. Gropp, Paul and Cynthia Saylor Professor of Computer Science, UIUC
Dr. Matthew H. Reilly, Co-founder, Vice President of Semiconductor Development and Chief Engineer, SiCortex, Inc.

 Preliminary Examination:


Date: October 30th, 2008 (3:30 pm)
Place: SC 4407
Proposal: [pdf]
Talk: [pptx] [pdf]


 Related Work:


Papers on Topology-aware Mapping
Survey of Interconnect Topologies

 Progress So Far:


Topology Manager (obtains topology information on different machines)
Latency and Contention Benchmark Suite (in MPI) and results on different supercomputers
Application-specific Mapping successes

 Software Downloads:


The TopoManager API
MPI Contention Suite



 Publications:

  1. Abhinav Bhatele and Laxmikant V. Kale, Quantifying Network Contention on Large Parallel Machines, submitted to Parallel Processing Letters (Special Issue on Large-Scale Parallel Processing), 2009 [pdf]
  2. Abhinav Bhatele, Eric Bohm, Laxmikant V. Kale, A Case Study of Communication Optimizations on 3D Mesh Interconnects, To appear in Proceedings of Euro-Par (Topic 13 - High Performance Networks), 2009 [pdf]
  3. Abhinav Bhatele, Laxmikant V. Kale, Nicholas Chen and Ralph E. Johnson, A Pattern Language for Topology Aware Mapping, Workshop on Parallel Programming Patterns (ParaPLOP 2009) [pdf]
  4. Abhinav Bhatele, Laxmikant V. Kale, Sameer Kumar, Dynamic Topology Aware Load Balancing Algorithms for MD Applications, Proceedings of International Conference on Supercomputing, 2009 [pdf]
  5. Abhinav Bhatele, Laxmikant V. Kale, An Evaluative study on the Effect of Contention on Message Latencies in Large Supercomputers, Proceedings of Workshop on Large-Scale Parallel Processing (IPDPS), 2009 [pdf]
  6. Abhinav Bhatele, Laxmikant V. Kale, Benefits of Topology-aware Mapping for Mesh Topologies, Parallel Processing Letters (Special issue on Large Scale Parallel Processing), Vol. 18, Issue 4, Pages 549-566, 2008 [pdf]
  7. Abhinav Bhatele, Laxmikant V. Kale, Application-specific Topology-aware Mapping for Three Dimensional Topologies, Proceedings of Workshop on Large-Scale Parallel Processing (IPDPS), 2008 [pdf]
  8. Abhinav Bhatele, Application-specific Topology-aware Mapping and Load Balancing for three-dimensional Torus Topologies, Master's Thesis, Department of Computer Science, University of Illinois, 2007 [pdf]

 Presentations and Posters:

  1. Abhinav Bhatele, Automating Topology Aware Task Mapping for Large Supercomputers, Doctoral Showcase, SC '09, Portland, OR [pptx] [pdf]
  2. Abhinav Bhatele, Load Balancing and Topology Aware Mapping for Petascale Machines, Scaling to Petascale Summer School, UIUC, Urbana, IL [pptx] [pdf]
  3. Abhinav Bhatele, A Case Study of Communication Optimizations on 3D Interconnects, Euro-Par 2009, Delft, The Netherlands [pptx] [pdf]
  4. Abhinav Bhatele, Dynamic Topology Aware Load Balancing Algorithms for MD Applications, International Conference on Supercomputing (ICS) 2009, New York, NY [pptx] [pdf]
  5. Abhinav Bhatele, An Evaluative Study on the Effects of Contention on Message Latencies in Large Supercomputers, Workshop on Large-Scale Parallel Processing (IPDPS 2009), Rome, Italy [pptx] [pdf]
  6. Abhinav Bhatele, IS TOPOLOGY IMPORTANT AGAIN? - Effects of Contention on Message Latencies in Large Supercomputers, ACM Student Research Competition, SC '08, Austin, TX [pptx] [pdf]
  7. Abhinav Bhatele, Topology Aware Mapping for Performance Optimization of Science Applications, IACAT Seminar, UIUC, Urbana, IL [pptx]
  8. Abhinav Bhatele, Dynamic Topology Aware Load Balancing Algorithms for MD Applications, UK e-Science All Hands Meeting 2008, Edinburgh, UK [ppt]
  9. Abhinav Bhatele, Application-specific Topology-aware Mapping for Three Dimensional Topologies, LSPP (IPDPS '08), Miami, FL, 2008 [ppt]
  10. Abhinav Bhatele, Eric Bohm, Laxmikant V. Kale, Topology Aware Task Mapping Techniques: An API and Case Study (poster), PPoPP 2009 [pdf]
  11. Abhinav Bhatele, Laxmikant V. Kale, Effects of Contention on Message Latencies in Large Supercomputers (poster), ACM Student Research Competition, SC, 2008 [pdf]
  12. Abhinav Bhatele, Laxmikant V. Kale, Automatic Topology-Aware Task Mapping for Parallel Applications Running on Large Parallel Machines (poster), TCPP PhD Forum, IPDPS, 2008 [pdf]


 Resources:


My Official HomePage
Research Page on Topology-aware Mapping

Cray XT3 at PSC (Bigben)
Cray XT4 at ORNL (Jaguar)
Blue Gene/L at IBM T J Watson
Blue Gene/L at LLNL
Blue Gene/P at ANL

Big N Computing

 Other Papers:


Papers on Matrix Mulitplication
Papers on CPMD


 My Work:
  • Considers both topological and cardinality variation
  • Hence considers load imbalance
  • Considers the amount of communication on each link
  • Objective function: Hop-bytes
  • Can do both initial mapping and dynamic movement depending on changes to the communication graph during runtime
  • Currently does not consider the routing rules of the network


Back to Home