Designing Software Libraries and Middleware for Exascale Computing: Opportunities and Challenges

The high-end computing community is aiming to enter exascale-level computing during the next six to eight years. Such systems will consist of millions of processors and accelerators.  This presentation will first focus on the architectural aspects of such exascale computing systems.  Next, we will focus on challenges and opportunities in designing software libraries and middleware for such systems. Both HPC and Enterprise/BigData systems will be targeted.

For HPC systems, we will focus on multiple emerging trends: support for Hybrid MPI+PGAS (OpenSHMEM and UPC) programming models, support for GPGPUs and Intel Xeon Phi, scalable collectives (multi-core-aware, topology-aware and power-aware), non-blocking collectives using offload framework, and schemes for fault-tolerance/fault-resilience. For enterprise/BigData systems, we will focus on RDMA-enabled high-performance and scalable designs of Apache Hadoop (including HDFS, MapReduce, RPC and HBase), Apache Spark and Memcached. Schemes for supporting virtualization with high-performance and RDMA-based WAN communication will also be presented.