AMS 530, Principles of Parallel Computing
This course is designed for both academic and industrial scientists interested in
parallel computing and its applications to large-scale scientific and engineering
problems. It focuses on the three main issues in parallel computing: analysis of parallel
hardware and software systems, design and implementation of parallel algorithms, and
applications of parallel computing to selected problems in physical science and engineering.
The course emphasizes hands-on practice and understanding of algorithmic concepts
of parallel computing.
Prerequisite: A course in basic computer science such as operating systems or architecture
or some programming experience.
3 credits, ABCF grading
Offered in Fall semester only.
Text: Professor Deng's Lecture Notes and Handouts
Recommended/Optional Text: "Introduction to Parallel Computing" by Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar, 2nd edition, Pearson Publishing, 2003; ISBN: 978-0-201-64865-2
Learning Outcomes:
1) Demonstrate knowledge of supercomputer architectures:
* Node architecture, central processing units, and accelerators;
* Distributed- and shared-memory;
* Various interconnection networks;
* Network architectures and performance;
* IO subsystem.
2) Demonstrate mastery of software architecture and MPI:
* Communication patterns and protocols;
* Message passing interface (MPI);
* Process creation and management;
* Single-sided communications;
* Collective operations.
3) Demonstrate understanding of performance measurements:
* Speedup, parallel efficiency, overhead, load imbalance ratio;
* Amdahl’s law.
4) Demonstrate mastery of parallel algorithms:
* Algorithm classification according to complexities;
* Problem decomposition;
* Communication handling;
* Amdahl’s law.
5) Demonstrate mastery of applications of parallel computers:
* Linear algebra: matrix multiplication, solution of AX=b, eigenvalues and eigenvectors;
* Basic calculus including high-dimensional integration;
* Solutions of ODEs and systems including molecular dynamics;
* Solutions of a few PDES including wave, diffusion, and Poison equations;
* Fast Fourier transforms;
* Markov chain Monte Carlo methods.