Readings and Preparatory Work, HPC II

Readings and Preparatory Work

High Performance Computing II

Readings and Preparatory Work for the SeSE Course, March 2014. The readings and preparatory material listed below give a solid background to the main topics of the course and will prepare you for the week in Umeå. More information may be added during the course.

Students will be acquainted with some of the software listed below during the assignments and homework for the course. It will of course not be possible to cover all the material listed below during the course.

Refresh your parallel programming skills

You should rehearse your MPI- and OpenMP-programming knowledge, either by repeating some of the excercises from the earlier SeSE course or by looking at the tutorials below.

MPI

• Tutorial on MPI: The Message-Passing Interface by William Gropp. A set of exercises for this tutorial is available.
• MPI tutorial from Livermore Computing Center.

OpenMP

• OpenMP tutorial from Livermore Computing Center.

Fundamental parallel algorithms

Basics of parallel linear algebra
Data partitioning and data distribution

The following two articles give an overview of concepts, methods and algorithms for basic problems in parallel numerical linear algebra. To some extent they are overlapping with respect to content.

• Parallel Numerical Linear Algebra (also published in ACTA NUMERICA 1993, pp 111-197)
• Scientific Computing on High Performance Computers (Draft version 2003)

The following article gives an overview of recursive blocked algorithms and hybrid data structures for dense matrix computations, targeting HPC architectures with deep memory hierarchies.

• Recursive Blocked Algorithms and Hybrid Data Structures for Dense Linear Algebra

Templates for the solution of linear systems

• Building Blocks for Iterative Methods is a hyper-text book on iterative methods for solving systems of linear equations.

Templates for dense and sparse eigenvalue problems

• Templates for Algebraic Eigenvalue Problems - A Practical Guide gives a unified overview of the theory, algorithms and practical software for eigenvalue problems. It organizes this large body of material to make it accessible for the first time to experts and non-experts who need to choose the best state-of-the-art algorithms and software for their problems.

Graph partitioning
The following article presents a parallel formulation of the multilevel graph
partitioning and sparse matrix ordering algorithm introduced by Karypis and Kumar.

• A Parallel Algorithm for Multilevel Graph Partitioning and Sparse Matrix Ordering

General HPC library software and tools

Get acquainted with some libraries and tools that will be covered in the course.

Dense linear algebra

• LAPACK Users' Guide, LAPACK website
• ScaLAPACK Users' Guide, ScaLAPACK website
• PBLAS
• BLACS

Sparse linear algebra

• SLEPc Users Manual
• PETSc Users Manual

Parallel graph partitioning

• ParMetis website

Survey of tools for developing HPC applications

• Advanced Computational Software Collection (ACTS)

POSIX Threads

• Pthreads tutorial from Livermore Computing Center.
• Pthreads tutorial from Yolinux.