Best Paper Awards at EuroMPI and IWOMP

17/12/2024

The HPC team at RWTH Aachen University celebrated a major success in September, bringing home two Best Paper Awards. Semih Burak received the Rusty Lusk Award for the Best Paper at EuroMPI/Australia 2024 for his paper titled “SPMD IR: Unifying SPMD and Multi-value IR Showcased for Static Verification of Collectives”. At the International Workshop on OpenMP (IWOMP 2024), co-located with EuroMPI and also held in Perth, Australia, Jannis Klinkenberg secured the first place with his paper “Towards Locality-Aware Host-to-Device Offloading in OpenMP”. Both EuroMPI and IWOMP are recognized as well-established key events for MPI and OpenMP, two leading parallel programming paradigms widely and extensively utilized on High Performance Computing (HPC) clusters.

The paper by Jannis Klinkenberg et al. “Towards Locality-Aware Host-to-Device Offloading in OpenMP” addresses the optimization of data transfers between host and device memory in OpenMP-supported, heterogeneous computing systems such as CPU-GPU architectures. Nowadays, these systems often comprise multiple sockets and multiple GPUs per compute node introducing performance variability due to differences in memory access across Non-Uniform Memory Access (NUMA) domains. As a result, performance issues frequently occur in these complex systems due to suboptimal offloading strategies and device selections, leading to non-local memory accesses. Existing programming models, such as OpenMP, lack robust features to account for the locality between CPU cores, data, and devices, which limits their ability to make optimal choices for efficient processing.
This work examined offloading performance between CPU cores and GPUs, proposing OpenMP API extensions to prioritize nearby GPUs for faster data transfer. A prototype implementation within the LLVM OpenMP runtime and experiments on two recent heterogeneous architectures with Nvidia and AMD GPUs confirm that the locality-aware approach significantly improves computational efficiency and performance in systems with multiple GPUs.

The paper by Semih Burak et al. “SPMD IR: Unifying SPMD and Multi-value IR Showcased for Static Verification of Collectives” introduces a unifying intermediate representation (IR), called SPMD IR, to enhance the reusability and extensibility of static analysis and optimization within modern compiler systems tailored for HPC. As modern HPC systems consist of increasingly parallel CPU and GPU architectures in multi-node setups, the usage of various parallel programming models (e.g., MPI, SHMEM, NCCL) has been established for making effective usage of the hardware. However, tool developers face challenges in creating adaptable verification and optimization tools that work across models. SPMD (Single Program, Multiple Data) IR addresses these challenges by providing a standardized representation of common parallel operations, such as communication and synchronization, found across SPMD-centric programming models. The paper also describes the mapping from prevailing parallel programming models to the SPMD IR. The IR is further able to represent crucial properties of SPMD programs as a result of an extended multi-value analysis.
Once a tool is ported to the SPMD IR, it enables support for another programming model solely by converting the original representation to the SPMD IR; the implementation of the tool itself does not need to be modified. Among others, the SPMD IR increases the semantic information the compiler has over the program. A prototype implementation in the MLIR/LLVM compiler infrastructure demonstrates SPMD IR’s potential, supporting MPI, SHMEM, and NCCL models (inc. their hybrid combinations), and validating that it enables more flexible and robust compiler toolchains for diverse HPC applications. The SPMD IRs approach was successfully demonstrated and evaluated by the use case of statically verifying collective communication in comparison to state of the art.