Course Information

Course Title: Parallel Computing

Semester: Spring 2024

Course Rationale

Most of us have learned to program a single microprocessor using a high-level programming language like C/C++, Java, ... This is called sequential programming. We feel very comfortable with this because we, as human beings, think in a sequential way and expect the program statements to be executed in sequence. However, this way of executing a program has dramatically changed. A microprocessor with single core no longer exists in almost all computers we are using today (including but not limited to your tablet, smart watch, and smart phone). Most of our devices are now multicore processors. A multicore processor contains several cores (called CPUs or cores) on-chip.

These cores are executing several programs at the same time, or several pieces of a program at the same time. Many devices also contain accelerators like GPUs.

To make the best use of these chips we need to program them in parallel. Sequential programming, for all platforms from smartphones to supercomputers, is falling out of fashion and taking back-seat to parallel programming.

How to think in parallel? How to write code in parallel to make the best use of the underlying hardware? How is that new hardware different from the traditional one? What will the future be for the software and hardware? This is what this course is about.

Objectives

After this course you should understand the following concepts:

• The need for parallel programming

• The different parallel machine designs

• Challenges of parallel programming

• Different parallel programming models

• MPI

• OpenMP

• CUDA

Course Plan

• What is parallel computing?

• Why do we need parallel computing?

• Why is parallel computing here to stay?

• Basics of parallel hardware

• Basics of parallel programming

• Challenges in parallel programming

• Performance analysis of parallel programs

• Pitfalls in parallel programming

• How are supercomputers programmed?

• MPI for distributed memory machines

• OpenMP for shared memory machines

• What are GPUs?

• How to program GPUs with CUDA?

Course Structure

• This course will have about three homework assignments and three programming assignments.

• There is a midterm exam.

• There is a final exam.

• The programming assignments will be done using parallel programming language on top of Cunder Linux. You can access Linux machines using your CIMS account, as will be shown in class.

Grading/Assessment

Information in this section can include:

● A breakdown of grades:

• Homework assignments 15%

• Labs 30%

• Midterm Exam 25%

• Final Exam 30%

● Grades will be posted on the gradebook of Brightspace.

● Late submission: -10% for each day of late submission, for homework and programming assignments, for up to three days. You have 59 minutes grace period after the assignment deadline. After late policy is enforced. That is, after one hour late, the assignment is viewed one day late.

● You can submit your assignment as many times as you want till the deadline. We will grade the last version submitted.

● After getting the grade, you have one week to argue about the grade. First, contact your assigned grader. If the issue is not resolved, bring it to me.