Seminar on Efficient Programming of HPC Systems - Frameworks and Algorithms (IN2107)

Prof. Dr. Erwin Laure (TUM and MPDCF)

Motivation

Developing applications for massively parallel High Performance Computing (HPC) systems requires specific programming models and tools to express parallelism and data access/movement. For many years, the predominantly used models have been the Message Passing Interface (MPI) and OpenMP. However, with the increasing complexity of HPC hardware, involving vector units, GPUs, and novel memory and storage technologies, higher-level models and libraries tuned for specific tasks have emerged that provide more powerful and efficient solutions. In addition, algorithmic changes or changes in data representation might be required to fully exploit recent hardware features, like eg. Tensor cores. 

In this seminar we review recent trends in programming HPC systems including accelerators, notably GPUs. 

Topics include, but are not restricted to: 

• High-level frameworks (Kokkos, Alpaka, Cabana, PETSc, etc.)
• Numerical libraries (SLATE, Gingko, heFFTe, etc.)
• Mixed-precision and use of non-IEEE data formats
• Data structures and layouts (AoS, SoA, AoSoA)
• Adaptive Mesh Refinement (AMReX, p4est, etc.)
• Adaptive (task) Parallelism (HPX, StarPU, Charm++, etc.)
• Frameworks for AI (pytorch, tensorflow, etc.)

The students will select published scholarly material, provide a report highlighting and contrasting features, and give a presentation. Tutors will be available to support the students.