The Center for Simulation of Advanced Rockets (CSAR) is one of five university-based research centers funded by the U.S. Department of Energy as part of its Advanced Simulation and Computing (ASCI) program. The principal goal of CSAR is detailed, integrated, whole-system simulation of solid propellant rockets under both normal and abnormal operating conditions.

Our research at PPL is focused on application of object-based parallel programming paradigms for computational science and engineering. Past research in our group allows building multi-paradigm adaptive parallel programs based on the Converse runtime system and Charm++ load-balancing framework. We have adapted this framework to procedural languages such as Fortran 90, and message passing programming paradigms. Detailed performance analysis of Adaptive MPI, an implementation of MPI standard on top of Charm++, has been done, and has been optimized to have minimal overhead as compared to native MPI on several machines. Various components of the integrated rocket simulation codes, ROCFLO, ROCSOLID, and ROCFACE have been ported to Adaptive MPI with minimal changes and with little effort, providing them with automatic load balancing capabilities. The AMPI implementation also enhances the modularity of rocket codes by using separate sets of virtual processors for ROCFLO and ROCSOLID. Based on a similar programming paradigm, a prototype finite element method (FEM) framework has been developed. Although originally designed for finite element applications, it is general enough for unstructured mesh applications. Using the FEM framework, code developers write only sequential code that implements the node and element-specific computations of the FEM solver, while the details of parallelism such as partitioning, communication, and load balancing are taken care of by the framework. A 2-D pressure-driven crack propagation application (originally developed by P. Geubelle et al) has been ported to use this framework.

The multi-partitioned object-based approach has several other benefits besides automatic load balancing and efficient cache reuse. We have developed prototype software components that allow parallel programs to shrink and expand at runtime in order to effectively use inherently dynamic computational resources, such as clusters of workstations (COWs), which are becoming increasingly popular parallel architectures. Also, an efficient checkpointing framework has been developed that allows a parallel program to checkpoint and restart on possibly different number of processors.

We are also developing a Parallel Mesh Adaptation Framework for use with our FEM framework. These codes perform mesh refinement/derefinement and quality improvements in parallel.
 

• Gengbin Zheng

• 05-16    Xiangmin Jiao, Gengbin Zheng, Phillip A. Alexander, Michael T. Campbell, Orion S. Lawlor, John Norris, Andreas Haselbacher, Michael T. Heath,  A system integration framework for coupled multiphysics simulations,  special infrastructure issue Engineering with Computers, 2006
• 05-08    Xiangmin Jiao , Gengbin Zheng , Orion Lawlor , Phil Alexander , Mike Campbell , Michael Heath , Robert Fiedler,  An Integration Framework for Simulations of Solid Rocket Motors,  41st AIAA/ASME/SAE/ASEE Joint Propulsion Conference, July 10--13, 2005, Tucson, Arizona
• 02-05    Orion Lawlor, Milind Bhandarkar, and L. V. Kale,  Adaptive MPI,  PPL Technical Report 02-05, University of Illinois at Urbana-Champaign, 2002.
• 00-04    Eric deSturler, Jay Hoeflinger, L. V. Kale and Milind Bhandarkar,  A New Approach to Software Integration Frameworks for Multi-physics Simulation Codes,  In R. Boisvert and P. Tang (Eds), The Architecture of Scientific Software, IFIP TC2/WG2.5 Working Conference on the Architecture of Scientific Software, Ottawa, Canada, Kluwer Academic Publishers, pp. 87-104, October 2000.
• 00-03    Milind Bhandarkar, L. V. Kale, Eric de Sturler, and Jay Hoeflinger,  Object-Based Adaptive Load Balancing for MPI Programs,  In Vassil Alexandrov et al (Eds), Computational Science -- ICCS2001, Proceedings of the International Conference on Computational Science, San Francisco, CA, Lecture Notes in Computer Science, Vol. 2074, Springer Ver;ag, pp 108-117, May 2001.
• 02-02    Puneet Narula,  An Adaptive Mesh Refinement (AMR) Library Using Charm++,  M.S. Thesis, Department of Computer Science, University of Illinois at Urbana-Champaign, January 2002.
• 00-01    Milind A. Bhandarkar and L. V. Kale,  A Parallel Framework for Explicit FEM,  Proceedings of the International Conference on High Performance Computing (HiPC 2000), Lecture Notes in Computer Science, Vol 1970, Eds: M. Valero and V. K. Prasanna and S. Vajpeyam, Springer Verlag, December 2000, pp. 385-395.

• Center for Simulation of Advanced Rockets