Optimizing the Performance of High Performance Computing Systems

The past decade has witnessed a revolution in the application of high performance computing (HPC) to scientific research, fueled in part by the explosion of data from high-throughput experimental devices. Given the important role that advanced computing plays in science and engineering, it is important to ensure that HPC systems, which are a complex combination of software, processors, memory, networks, and storage systems, are operating at optimal efficiency and that all subcomponents are functioning properly. Unfortunately, and somewhat surprisingly, the current set of resource management tools in HPC do not offer this capability, and HPC centers and their users have been to some extent “flying blind,” without complete information on system behavior and performance.

This presentation will discuss the development of an open source tool chain for the systematic and comprehensive resource management of high performance computing systems, and its incorporation into XDMoD (xdmod.ccr.buffalo.edu), a reporting and analytics framework for resource management that targets meeting the information needs of users, application developers, systems administrators, center directors, and funding managers. A case study of the application of the methodology to XSEDE Ranger and Lonestart4 systems at the University of Texas, and UB’s Center for Computational Research will be included.