This is is an introductory course in understanding high performance computing (HPC) organization and the basics of parallel computing. The course is divided into two logical but related parts. (1) Introduction to Computer Systems discusses how a single computer is organized from a programmer's perspective, and the performance implications when developing computational applications; and (2) Parallel computing introduces parallel architectures, and how parallel applications can be designed on such systems. Specific topics that will be covered are:
- Introduction to Computer Systems: Processors, Memory, I/O Devices; Cost, timing and scale (size) models; Program Execution: Process; Virtual Memory; System Calls; Dynamic Memory Allocation. Machine level view of a program; typical RISC instruction set and execution; Pipelining. Performance issues and Techniques; Cost and frequency models for I/O, paging and caching. Temporal and spatial locality. Typical compiler optimizations. Identifying program bottlenecks - profiling, tracing. Simple high level language optimizations - locality enhancement, memory disambiguation. Choosing appropriate computing platforms: benchmarking, cost-performance issues.
- Parallel computing: Parallel Computing: Introduction to parallel architectures and interconnection networks, communication latencies. Program parallelization; task partitioning and mapping; data distribution; message passing; synchronization and deadlocks. Distributed memory programming using MPI/PVM. Shared memory parallel programming. Multithreading.
Govind will be teaching the first part of the course (8-10 weeks) while Yogesh will cover the second part (6-8 weeks).↑
Intended Learning Objectives
At the end of the course, students should have learned the following concepts:
- Understand computer systems, such as processor architecture, memory subsystem and caching, from a programmer's viewpoint to maximize performance of his/her application.
- Understand underlying Operating Systems concepts from a programmer's viewpoint, including process, memory and file system management.
- Understand parallel processing systems, from a programmer's viewpoint, and develop parallel programs.
Collectively, these should help you design and develop high performance applications, and also improve applications' performance.↑
Teaching and Learning Activities
- Lectures: Lectures will form the primary teaching activity, the tentative schedule for which is outlined below. Lecture material will be posted online, and serve as the primary resource. The course text books will supplement the lecture slides. A mailing list is be available for discussions and queries outside the classroom, between students and with the faculty. All students should sign up on the list for official announcements.
- Homeworks: Homework assignments serve the dual purposes of forming a learning activity as well as a means for assessment. These count toward 25% of the total assessment score. Homework will include short answer questions, problem solving and some programming. The students are expected to understand and apply concepts learnt from class lectures and the text book, as well as use online resources, to complete these assignments.
- Exam: There will be three mid-term exams and a final exam for the course. These collectively count toward 75% of the total assessment score.
The total assessment score for the course is based on a 1000 point scale. Of this, the weightage to different activities will be as follows:
|25% Homework||About five homework assignments (250 points total).|
|30% Midterm Exams||Three Mid-term exams of 100 points each.|
|45% Final Exam||One Final exam for 450 points.|
Students must uphold IISc's Academic Integrity guidelines. While these are common sense, it is helpful to review them since failure to follow them will lead to sanctions and penalties. This includes a reduced or failing grade in the course. Severe cases of academic violations will be reported to the Institute and may lead to an expulsion.
Learning takes place both within and outside the class. Hence, discussions between students and reference to online material is encouraged as part of the course to achieve the intended learning objectives. However, while you may learn from any valid source, you must form your own ideas and complete problems and assignments by yourself. All works submitted by the student as part of their academic assessment must be their own. No group work is allowed unless explicitly stated in the assignment.
- Verbatim reproduction of material from external sources (web pages, books, papers, etc.) is not acceptable. If you are paraphrasing external content (or even your own prior work) or were otherwise influenced by them while completing your assignments, projects or exams, you must clearly acknowledge them. When in doubt, add a citation!
- While you may discuss lecture topics and broad outlines of homework problems with others, you cannot collaborate in completing the assignments, copy someone else's solution or falsify results. You cannot use notes or unauthorized resources during exams, or copy from others.
- Classroom Behavior
- Ensure that the course atmosphere, both in the class, outside and on the mailing list, is conducive for learning. Participate in discussions but do not dominate or be abusive. There are no "stupid" questions. Be considerate of your fellow students and avoid disruptive behavior.
|Textbook||Select topics from:|
|Mailing Listfirstname.lastname@example.org | Mailman Info Webpage|
|Prior Course Content||You can find relevant material for the course from past years' webpages of Govind and Sathish|
Teaching & Office Hours
|Lecture||TTh 8AM-930AM, SERC 202|
Schedule is based on two 1.5 hour lectures on TTh 8AM-930AM each week.
|Lecture No.||Date||Topics Covered & Assignments||Slides|
|1||Thu 7 Aug||Introduction||[PPSX] [PDF]|
|2||Tue 12 Aug||Introduction|
|3||Thu 14 Aug||Introduction|
|4||Tue 19 Aug||Basic Computer Organization||[PPSX] [PDF]|
|5||Thu 21 Aug||Basic Computer Organization|
|6||Tue 26 Aug||Pipelining||[PPSX] [PDF]|
|7||Thu 28 Aug||Pipelining|
|8||Tue 2 Sep||Pipelining|
Assignment 1 Posted
|9||Thu 4 Sep||Memory Hierarchy, Part 1||[PPSX] [PDF]|
|10||Sat 6 Sep||Memory Hierarchy|
|11||Tue 9 Sep||Memory Hierarchy, Part 2||[PPSX] [PDF]|
|12||Thu 11 Sep||Mid-term Exam 1|
|13||Tue 23 Sep||Memory Hierarchy, Part 3||[PPSX] [PDF]|
|14||Thu 25 Sep||Process Management, Part 1||[PPSX] [PDF]|
|15||Sat 27 Sep||Performance Tuning and Profiling|
Assignment 2 Posted
|Part A: [PPSX] [PDF]|
Part B: [PPSX] [PDF]
|...||Tue 30 Sep||Cancelled|
|...||Thu 2 Oct||Official Holiday. Mahatma Gandhi's Jayanthi. |
Substitute class TBD.
|16||Tue 7 Oct||Process Management, Part 2|
|17||Thu 9 Oct||Process Management, Part 3|
|18||Tue 14 Oct||Mid-term Exam 2|
|19||Thu 16 Oct||Concurrent Programming, Part 1||[PPSX] [PDF]|
|20||Sat 18 Oct||Concurrent Programming, Part 2|
|21||Tue 21 Oct||Parallelization, Part 1|
|...||Thu 23 Oct||Official Holiday. Deepavali. |
Substitute class TBD.
|22||Tue 28 Oct||Parallel Architectures, Part 1||[PPSX] [PDF]|
|23||Thu 30 Oct||Parallel Architectures, Part 2|
|...||Tue 4 Nov||Official Holiday. Muharram. |
Substitute class TBD.
|24||Wed 5 Nov||Parallelization, Part 2 (Substitute Class)||[PPSX] [PDF]|
|...||Thu 6 Nov||Official Holiday. Guru Nanak's Birthday. |
Substitute class TBD.
|25||Sat 8 Nov||Parallelization, Part 3 (Substitute Class, 10AM)|
|26||Tue 11 Nov||MPI, Part 1|
Assignment 3 Posted
|27||Thu 13 Nov||Mid-term Exam 3|
|28||Fri 14 Nov||MPI, Part 2 (Substitute Class, 830AM)|
|29||Tue 18 Nov||OpenMP, Part 1||[PPSX] [PDF]|
|30||Thu 20 Nov||OpenMP, Part 2|
|31||Tue 25 Nov||File System, Part 1||[PPSX] [PDF]|
|32||Thu 27 Nov||File System, Part 2|
|33||Wed 3 Dec||Final Exam @ 2PM|
This course includes material prepared by Matthew Jacobs and Sathish Vadhiyar from SERC, IISc. Material used from other sources are acknowledged in the slides.