Faculty of Engineering

Subject Code:	ECP4236
Objective:	The objective of the subject is to provide the knowledge and basic applications of parallel processing concepts, parallel environments and architectures, parallel algorithms and parallel programming.
Pre-Requisite:	ECP1026:Algorithm and Data Structures
Credit Hours:	3
Contact Hours:	48 hours
Assessment:	Test/Quiz: 15% Tutorial/Assignment:25% Final Examination: 60%

References:	Barry Wilkinson and Michael Allen, "Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers", Pearson Prentice Hall, 2004. (Textbook) Ananth Grama, Anshul Gupta, George Karypis and Vipin Kumar, "Introduction to Parallel Computing", Addison-Wesley, 2003. (Textbook) Harry F. Jordan & Gita Alaghband, "Fundamentals of Parallel Processing", Prentice Hall, 2003. Peter S. Pacheco, "Parallel Programming with MPI", Morgan Kaufmann Publishers, Inc., 1999. Rajkumar Buyya, "High Performance Cluster Computing: Programming and Applications, Volume 2", Prentice Hall PTR, 1999.

Introduction to Parallel Computing (Part I)
Motivations for parallelism, scope of parallel computing, parallel paradigms, parallel programming environments, physical organization of parallel platforms and computational speed.
Parallel Processing on shared Memory
Shared memory multiprocessors and chip-level multiprocessor (CMP or multi-core), concurrent process creation (UNIX heavyweight process and threads), shared data access, shared memory synchronization (lock, barrier, semaphores, deadlock, etc), POSIX Thread API (Pthreads), OpenMP, sample applications/programs.
Introduction to Parallel Computing (Part II)
Motivations for Interconnection, Communication methods, deadlocks & cluster computing .
Analytical Modeling of Parallel Programs
Basics of message passing programming, performance metrics for parallel systems (execution time, overhead, speedup, efficiency, cost, etc), analytical evaluation of communication operations and parallel programs.
Message Passing Paradigms
Message Passing Interface (MPI), Parallel Virtual Machine (PVM), sample applications/programs.
Parallel Algorithm Design
Partitioning and divide-and-conquer strategies, pipelined computations, embarrassingly parallel computations, other parallel algorithm models (data-parallel, task graph, work pool, master slave, hybrid, etc), sample applications/programs.
Synchronization
Barriers, synchronized computations, sample applications/programs.

At the completion of the subject, students should be able to perform the following tasks:

LO1 - Describe different parallel processing architectures based on relationships between processing elements, instruction sequence, memory and interconnected network (Cognitive – knowledge, level 1) - 7%
LO2 - Identify algorithms, which require parallelization as part of system design or performance enhancement (Cognitive – comprehension, level 2) - 25%
LO3 - Classify shared and distributed memory parallel systems according to their properties and usage models (Cognitive – analysis, level 4) - 15%
LO4 - Design and develop parallel algorithms for shared and distributed memory models (Cognitive – synthesis, level 5) - 48%
LO5 - Evaluate the performance of parallel algorithms designed based on shared and distributed memory models as well as against serial based algorithm designs (Cognitive – evaluation, level 6) - 5%

PO1 - Ability to acquire and apply fundamental principles of science and engineering - 12%
PO3 - Acquisition of technical competence in specialized areas of engineering discipline - 27%
PO4 - Ability to identify, formulate and model problems and find engineering solutions based on a system approach - 37%
PO5 - Ability to conduct investigation and research on engineering problems in a chosen field of study - 3%
PO6 - Understanding of the importance of sustainability and cost-effectiveness in design and development of engineering solutions - 9%
PO8 - Ability to work effectively as an individual, and as a member/leader in a team - 7%
PO11 - Capability and enthusiasm for self-improvement through continuous professional development and life-long learning - 11%