Birds of a Feather

Navigating Complexity: Achieving Performance Portability in the Evolving Landscape of Heterogeneous HPC Systems

Authors: David Martin (Argonne National Laboratory (ANL)), Amit Ruhela (Texas Advanced Computing Center (TACC), University of Texas), Nalini Kumar (Intel Corporation), Clayton Hughes (Sandia National Laboratories), Alastair Murray (Codeplay Software Ltd, UK), Venkatatram Vishwanath (Argonne National Laboratory (ANL)), Zhao Zhang (University of Texas), Anna Pietarila Graham (Los Alamos National Laboratory (LANL))

Abstract: With increasing demand for AI in HPC, there has been an explosion in architectures, programming models, and AI frameworks. The already-daunting task of programming for heterogenous systems has become even more challenging. This BoF, organized by the IXPUG but not limited to Intel technology, will focus on portable programming across a wide variety of architectures running a diverse set of HPC, and AI workloads.

This BoF will explore challenges, state-of-the-art approaches, and emergent best practices for programming across heterogeneous systems and novel architectures, identifying common principles and practices that enable development and maintenance of software across sites, architectures, and applications.

Long Description: The Intel eXtreme Performance Computing Users Group (IXPUG) is a worldwide community of hundreds of application users, software developers and HPC center staff that utilize Intel hardware and software technologies (processors, accelerators, memory, software tools and storage middleware) to solve some of the world’s most challenging problems on some of the world’s most powerful supercomputers.

This BoF, organized by the IXPUG, but not limited to Intel technologies, will focus on sharing experiences and expertise in programming across a wide variety of architectures (including AI accelerators), running a diverse set of simulation, data analytics, AI, and HPC-AI workloads. With increasing heterogeneity across the computing landscape over the last few years and community expectations that system diversity will likely intensify over the coming years as AI methods are increasingly explored and incorporated in/around traditional HPC codes, programming models and programming abstractions that enable efficient programming across a variety of computing architectures for a wide range of application workloads have become a necessity for application developers and computational scientists. Different underlying hardware architectures, parallelization strategies, hardware features and primitives make it challenging to develop application software that can efficiently run across machines and sites. This BoF will explore current approaches and best practices for programming heterogeneous systems and novel architectures, with the goal of identifying a unified set of principles and practices that can be leveraged to develop and maintain SW across sites, architectures, and applications. Through invited talks by HPC center researchers and developers of commonly used programming abstractions (e.g. Kokkos, RAJA, SYCL, oneAPI, etc.) and HPC-AI application developers that have real-world experience benchmarking a wide variety of systems, the BoF will provide a forum for researchers, application programmers, HPC center staff and industry experts to share experiences in efficiently programming increasingly heterogeneous systems with increasingly varied workloads. The first half of the BoF will be short presentations, followed by a moderated discussion among the speakers and audience.

This BoF follows productive and well-attended IXPUG BoFs starting from SC14 and continuing through SC22, attracting 80-150 attendees each year. IXPUG has also hosted related BoFs and workshops at ISC High Performance and HPC Asia. The IXPUG steering group has confirmed a diverse group of experts to give short presentations on programming abstractions and models for portability currently in use by developers/users and challenges they foresee in the upcoming years. A moderated discussion will complete the session and allow attendees to interact with speakers and share their own experiences.

Topics covered: - Portable programming models like Kokkos, RAJA, SYCL, OpenACC, - Challenges and best practices with Tensorflow, PyTorch, and High-performance Python - Mapping programming models to architectures - HPC-AI Containers for portability - Tradeoffs between portability and efficiency - Real-world experiences in developing portable code across different architectures

Website: https://www.ixpug.org/events/sc23-bof