Behrooz Parhami's website banner


Behrooz Parhami's ECE 254B Course Page for Winter 2018

Parallel activities at a racing car's pit stop

Adv. Computer Architecture: Parallel Processing

Page last updated on 2018 March 27

Enrollment code: 13516
Prerequisite: ECE 254A (can be waived, but ECE 154 is required)
Class meetings: MW 10:00-11:30, Phelps 1431
Instructor: Professor Behrooz Parhami
Open office hours: M 12:00-2:00, W 1:00-2:00, HFH 5155
Course announcements: Listed in reverse chronological order
Course calendar: Schedule of lectures, homework, exams, research
Homework assignments: Eight assignments, worth a total of 40%
Exams: No exams this quarter
Research paper: Written research report, worth 60%
Research paper guidlines: Brief guide to format and contents
Policy on academic integrity: Please read very carefully
Poster presentation tips: Brief guide to format and structure (N/A)
Grade statistics: Range, mean, etc. for homework and paper grades
References: Textbook and other sources (Textbook's web page)
Lecture slides: Available on the textbook's web page
Miscellaneous information: Motivation, catalog entry, history

Course Announcements

Megaphone 2018/03/27: The winter 2018 offering of ECE 254B is officially over and all grades have been reported to the Registrar. Research-paper grades have also been reported to students in private e-mail messsages. Four primary criteria (relevance, coverage/breadth, technical depth, and technical accuracy) and four secondary factors (organization, presentation, language, format) were used in evaluating your research papers. I will see several of you in my spring 2018 course ECE 252B (computer arithmetic). Enjoy the rest of your spring break!
2018/03/06: HW8 (last one for the course) has been posted to the homework area below. All but one of the provisional research-paper outlines have been received/reviewed. I responded to two of the submission e-mails with suggestions. In all other cases, the outlines were judged to be sufficient and appropriate. The last research milestone is submission of the final paper by midnight on W 3/21. This final deadline cannot be extended, as I need a few days to read and comparatively evaluate the submitted papers before the grades due date of T 3/27.
2018/02/27: HW7 has been posted to the homework area below. I have received and reviewed all preliminary reference lists for your research papers. In four cases, I responded to the submission e-mails with suggestions. In all other cases, the references were judged to be sufficient and appropriate as preliminary lists. The next research milestone is submission of a provisional outline for your paper by M 3/05.
2018/02/15: HW6 (due on W 2/28) has been posted to the homework area below. I am posting this 6th homework assignment early, so that you can work on it in parallel with your compilation of "Preliminary Refernces" for your research paper, the latter being due by M 2/26.
2018/02/13: HW5 has been posted to the homework area below. Research topic assignments are now complete. The next research milestone is submission of preliminary list of references by M 2/26.
2018/02/08: Fourteen students have already been assigned research topics. One student has not yet submitted his preferences and another one submitted only one topic, which conflicted with other students' choices. Those two students should contact me ASAP. Topics 4, 7, 8, 12 are still available. If you have the time, I recommend attendance at the following seminar on Thursday, February 15, 2018 (4:00 PM, ESB 2001): George Michelogiannakis (LBNL Scientist), "An Architect's Point of View on Emerging Technologies and the Future of Digital Computing."
2018/02/06: HW4 has been posted to the homework area below. Half of the 18 enrolled students now have research topics assigned based on their stated preferences.
2018/02/04: Two of the research topics, #14 (FPGA Accel.) and #18 (new proposal), have arleady been assigned. Please continue sending me your first, second, and third topic preferences or proposal for a new topic (with a summary that defines the topic and a couple of references), so that I can complete the research topic assignemtnts by our 2/12 deadline.
2018/01/30: HW3 has been posted to the homework area below.
2018/01/28: The research section of this page has been updated with the requirements and procedures for winter 2018. Research topic selection deadline is M 2/12, so please get started by familiarzing yourself with the area of research and the designated topics.
2018/01/24: Students who missed the ECE 254B lecture today can take advantage of a repeat lecture on Friday 1/26, 4:30-6:00 PM, in Phelps 1431 (our regular classroom).
2018/01/23: HW2 has been posted to the homework area below. It has been brought to my attention that many graduate students will be attending a fair organized by UCSB Career Services on W 1/24 during our class time. I will explore the possibiity of repeating the 1/24 lecture on F 1/26 and will announce the time and place as soon as I have secured a classroom.
2018/01/16: Welcome to the ECE 254B web page for winter 2018. As of today, 25 students have signed up to take the course. Our schedule will be differrent from the usual, because the quarter is one week shorter and we are still facing emergency conditions in our area, which affect a significant number of students, staff, and faculty. I wll use the lecture slides from 2016 with no modification, but, as will be explained in the first lecture on W 1/17, we will focus on big-data applications as they interact with parallel processing technology.

Course Calendar


Course lectures, homework assignments, and research milestones have been scheduled as follows. This schedule will be strictly observed. In particular, no extension is possible for homework due dates or research milestones. Each lecture covers topics in 1-2 chapters of the textbook. Chapter numbers are provided in parentheses, after day & date. PowerPoint and PDF files of the lecture slides can be found on the textbook's web page.

Day & Date (book chapters) Lecture topic [Homework posted/due] {Special notes}
M 01/15 No lecture: Martin Luther King Holiday
W 01/17 (1) Introduction to parallel processing [HW1 posted]

M 01/22 (2) A taste of parallel algorithms
W 01/24 (3-4) Complexity and parallel computation models [HW1 due] [HW2 posted]

M 01/29 (5) The PRAM shared-memory model and basic algorithms
W 01/31 (6A) More shared-memory algorithms [HW2 due] [HW3 posted]

M 02/05 (6B-6C) Shared memory implementations and abstractions
W 02/07 (7) Sorting and selection networks [HW3 due] [HW4 posted]

M 02/12 (8A) Search acceleration circuits {Research topic selected}
W 02/14 (8B-8C) Other circuit-level examples [HW4 due] [HW5 posted]

M 02/19 No lecture: President's Day Holiday
W 02/21 (9) Sorting on a 2D mesh or torus architectures [HW5 due] [HW6 posted]

M 02/26 (10) Routing on a 2D mesh or torus architectures {Preliminary references due}
W 02/28 (11-12) Other mesh/torus concepts [HW6 due] [HW7 posted]

M 03/05 (13) Hypercubes and their algorithms {Provisional outline due}
W 03/07 (14) Sorting and routing on hypercubes [HW7 due] [HW8 posted]

M 03/12 (15-16) Other interconnection architectures {Instructor/course evaluation surveys}
W 03/14 (17-18) Task scheduling and input/output [HW8 due]

W 03/21 {Research paper due by midnight}

T 03/27 {Course grades due by midnight}

Homework Assignments

Homework image

- Submit solution via e-mail by the given deadline (Wednesday at midnight; no extention)
- Submission must have an attached PDF file of no more than 1 page, with 10-point font
- The PDF file helps me enforce the 1-page limit and can also hold figures, if necessary
- The textual part of the solution must also be included in the body of the e-mail
- Although some cooperation is permitted, direct copying will have severe consequences

Homework 1: Literature review and assessment (due W 2018/01/24, midnight)
Read the following paper and describe its contents in outline form, devoting a couple of sentences to each section. Remember that the PDF file of your submission cannot exceed 1 page, but it can be single-spaced.
M. D. Assuncao, R. N. Calheiros, S. Bianchi, M. A. S. Netto, and R. Buyya (2015), "Big Data Computing and Clouds: Trends and Future Directions," J. Parallel and Disdtributed Computing, Vols. 79-80. pp. 3-15, May.
[A longer version of the paper is available on]

Homework 2: Web-based mini-research project (due W 2018/01/31, midnight)
Data longevity: Big data isn't just "lots of data," although "volume" is in fact one of the so-called "4 Vs" characterizing the big-data universe. Another one of the Vs is "value" (replaced by "varacity" in some sources); rounding up the list are "velocity" (speed of generation and/or change) and "variety" (diversity of formats, including unformatted data). When data is valuable to the enterprise or individual generating or collecting it, protection against loss or theft and preservation over extended periods of time are called for. Encoding and replication provide protection against accidental loss, as discussed in B. Parhami's EBDT article "Data Replication and Encoding." However, we must also be concerned with data loss due to format obsolescence, storage-media obsolescence, and stored-data degradation, that is, we should care about data longevity. Note that storing data in the cloud does not remove any of the problems just cited, as the cloud must also use some form of storage medium to hold the data. All three problems can be dealt with via data refreshing, the frequency of which will depend on the rate of degradation or obsolescence.
With the introduction above, here is your homework assignment. Identify a couple of on-line sources (which you will cite in your report), where useful lives of various storage media are discussed. Prepare a table in which the useful lives, or ranges of lives, are listed, along with whether each medium can be accessed at high speed or is archival in nature. Don't forget that your report should fit in a one-page PDF file, with the text also supplied in the body of the submission e-mail.

Homework 3: Literature review and assessment (due W 2018/02/07, midnight)
Read the following paper and prepare a one-page summary of it, focusing primarily on the data-processing and data-transformation considerations for big data, that is, why parallel processing is an inevitable requirement.
S. Kaisler, F. Armour, J. A. Espinosa, and W. Money, "Big Data: Issues and Challenges Moving Forward," Proc. 46th Hawaii int'l Conf. System Sciences, 2013, pp. 995-1004. [PDF]

Homework 4: Literature review and assessment (due W 2018/02/14, midnight)
Read the following article and prepare a one-page summary of it. Briefly state your opinion on how the article's conclusions may have changed in the 7 years since it was published.
M. Hilbert and P. Gomez, "The World's Technological Capacity to Store, Communicate, and Compute Information," Science, Vol. 332, 2011, pp. 60-65. [PDF]

Homework 5: Explaining big-data's "5 Vs" via analogies (due W 2018/02/21, midnight)
Consider the "5 Vs" of big data (1. Volume; 2. Variety; 3. Velocity; 4. Veracity; 5. Value), which we discussed earlier. Establish a one-to-one mapping between these "5 Vs" and the following five American-English idioms involving straw and hay. Explain your choice of association briefly in each case. Don't worry that the connection isn't direct or very relevant in every case. Just try to make the best possible assignment of a different "V" attribute to each idiom. [Remember the e-mail submission format and the 1-page limit in your PDF file.]
a. Clutching at straws
b. Finding a needle in a haystack
c. Making bricks without straw
d. Straw that broke the camel's back
e. Turning straw into gold

Suggested solution: The solution is not unique, although the connection between gold and value stands out.
Clutching at straws (3. Velocity): As we begin to swim in the ocean of big data, we tend to look around for any sign of helpful information that we can use to navigate the immense space of possibilities and save ourselves from drowning in irrelevant data.
Finding a needle in a haystack (2. Variety): Increased data volume and variety make it difficult to find a desired piece of information or draw needed inferences. Put another way, piles of less valuable data mask the golden nuggets we crave to find.
Making bricks without straw (4. Veracity): Lumps of dirt fall apart without long strands of straw holding them together, so too large data sets need unifiying themes and tools to make the data structures stand up and remain useful under normal usage.
Straw that broke the camel's back (1. Volume): As data volume increases, our storage, communication, and processing capabilities become strained. There is a point beyond which our usual methods turn inapplicable and new strategies are called for.
Turning straw into gold (5. Value): Massaging and transformations are needed to turn raw data into knowledge and insight. Insight gained from correlating and combining data elements increases the value of the data by revealing hidden relationships.

Homework 6: Preparing an annotaved bibliography (due W 2018/02/28, midnight)
Take the top 4 most-relevant/important references among those you will submit on 2/26 as "Preliminary References" for your research paper and turn them into an annotated bibliography for your field of research. An entry of an annotated bibliography includes, in addition to the full referernce citation, a brief description of what the article or source is about. This description differs from an abstract in that it will contain your assessment of its worth or relevance and how it fits into the overall scheme of the field you are studying.
Here is Cornell University's Library Guide to preparing an annotated bibliography.

Homework 7: Preparing Two PowerPoint slides (due W 2018/03/07, midnight)
For several decades now, the world has moved continuously to replace analog data with digital data, given the latter's robustness and affinity with digital computing. We read in the reference for HW4 that the share of digital data increased from near-zero to more than 90% in the period 1986-2007. There are now hints in the literature that in some cases, use of analog data and analog computing may be beneficial, as we grapple with big data and related applications. Find two on-line sources which discuss the advantages of analog data/computation or advocate greater attention to analog in the age of big data, and present a capsule summary of your sources in the form of a single PowerPoint slide. Briefly discuss how analog computing will interface with parallel processing in a second slide. Put your two slides on a single page of a PDF document that you will submit.
[Follow the 1-6-6 rule in each PowerPoint slide: A slide should contain one main idea, 6 or fewer bullet points, and 6 or fewer words per bullet point (but don't take these numbers too literally; they are just guidelines). Avoiding bullet points is even better. Diagrams are always preferable. The slide title should read like a headline about the main idea. For example: "Revenue to Rise 25% for 2018" instead of "Revenue Projection for 2018"]

Homework 8: Literature review and assessment (due W 2018/03/14, midnight)
Read the paper [Tank12] and discuss the following questions in your 1-page PDF submission.
a. How projected increases in data volumes exacerbate privacy and security challenges.
b. Big-data application domains that would suffer most from privacy violations and security breaches.
c. How parallel processing (increased processing power) can help mitigate these problems.
[Tank12] C. Tankard (2012), "Big Data Security," Netwrok Security, Vol. 2012, No. 7, pp. 5-8.

Sample Exams and Study Guides

Answer sheet

Does not apply to the winter 2018 offering of ECE 254B.
The following sample exam using problems from the textbook is meant to indicate the types and levels of problems, rather than the coverage (which is outlined in the course calendar). Students are responsible for all sections and topics, in the textbook and class handouts, that are not explicitly excluded in the study guide that follows the sample exam, even if the material was not covered in class lectures.

Sample Midterm Exam (105 minutes)
Textbook problems 2.3, 3.5, 5.5 (with i + s corrected to j + s), 7.6a, and 8.4ac; note that problem statements might change a bit for a closed-book exam.

Midterm Exam Study Guide
The following sections are excluded from Chapters 1-8 of the textbook to be covered in the midterm exam, including the six new chapters named 6A-C (expanding on Chpater 6) and 8A-C (expanding on Chapter 8):
3.5, 4.5, 4.6, 6A.6, 6B.3, 6B.5, 6C.3, 6C.4, 6C.5, 6C.6, 7.6, 8A.5, 8A.6, 8B.2, 8B.5, 8B.6

Sample Final Exam (150 minutes)
Textbook problems 1.10, 6.14, 9.5, 10.5, 13.5a, 14.10, 16.1; note that problem statements might change a bit for a closed-book exam.

Final Exam Study Guide
The following sections are excluded from Chapters 1-16 of the textbook to be covered in the final exam: All midterm exclusions, plus 9.6, 12.6, 13.5, 15.5, 16.5, 16.6

Research Paper and Presentation

Colored marbles

Our research focus this quarter will be on the topical area of big data, as defined in class and the reading/research assignments for the first three course homeworks. We will be studying the intersection of big data with high-performance computing, that is, how the meteoric spread of big-data applications affects the field of parallel and distributed computing and how research in our field can help solve the challenges brought about by big data.

The "4 Vs" of big data are nicely illustrated in the following IBM infographic. The fourth "V," equated in the image with "veracity," having to do with correctness and accuracy of data, is sometimes replaced by "value" (see the ssecond homework assignment), the data's importance or worth to an enterprise or application. Alternatively, one might view the big-data domain as being defined by "5 Vs."

The '4 Vs' of big data, in an IBM infographic
Here is a list of research paper titles. I am posting the list, even though some of the info is incomplete, in order to give you a head start on choosing a topic. In topic selection, you can either propose a topic of your choosing, which I will review and help you refine into something that would be manageable and acceptable for a term paper, or you can give me your first to third choices among the topics that follow. I will then assign a topic to you by February 12, based on your preferences and those of your classmates. Sample references follow the titles to help define the topics and serve as starting pointa for your study.

1. Hybrid Disk Drives for Big-Data Storage Needs (Assigned to: Chang Lu)
[Wu09] X. Wu and A. L. Narasimha Reddy, "Managing Storage Space in a Flash and Disk Hybrid Storage System," Proc. IEEE Int'l Symp. Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, 2009, pp. 1-4.
Search terms: Flash disk cache

2. GPU-Based Parallel Architectures for Big Data (Assigned to: Barath K. Ramaswami)
[Liu13] Z. Liu, B. Jiang, and J. Heer, "imMens: Real-Time Visual Querying of Big Data," Computer Graphics Forum, Vol. 32, No. 3, 2013, pp. 421-430.
Search terms: GPU-based supercomputing

3. Network Coding for Fast Exchange of Big Data (Assigned to: John Zhou)
[Dima10] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. J. Wainwright, and K. Ramchandran, "Network Coding for Distributed Storage Systems," IEEE Trans. Information Theory, Vol. 56, No. 9, 2010, pp. 4539-4551.
Search terms:

4. Big-Data Allocation and Load-Balancing Methods (Assigned to: Haowen Zhang)
[Wang14] K. Wang, X. Zhou, T. Li, D. Zhao, M. Lang, and I. Raicu, "Optimizing Load Balancing and Data-Locality with Data-Aware Scheduling," Proc. IEEE Int'l Conf. Big Data, 2014, pp. 119-128.
Search terms:

5. Low-Redundancy Methods for Big-Data Protection (Assigned to: Yunxi Li)
[Amja12] T. Amjad, M. Sher, and A. Daud, "A Survey of Dynamic Replication Strategies for Improving Data Availability in Data Grids," Future Generation Computer Systems, Vol. 28, No. 2, 2012, pp. 337-349.
Search terms:

6. RAID in the Era of Cloud-Based Big-Data Storage (Assigned to: Haorui Jiang)
[Gu14] M. Gu, X. Li, and Y. Cao, "Optical Storage Arrays: A Perspective for Future Big Data Storage," Light: Science & Applications, Vol. 3, No. 5, 2014.
Search terms:

7. Structural Representations for Big-Data Storage (Assigned to: No one)
[Sand14] A. Sandryhaila and J. M. F. Moura, "Big Data Analysis with Signal Processing on Graphs: Representation and Processing of Massive Data Sets with Irregular Structure," IEEE Signal Processing, Vol. 31, No. 5, 2014, pp. 80-90.
Search terms:

8. Meeting Reliability Requirements of Big Data (Assigned to: No one)
[Sath13] M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur, "XORing Elephants: Novel Erasure Codes for Big Data," Proc. Symp. VLDB, Vol. 6, No. 5, 2013, pp. 325-336.
Search terms: Reliable distributed systems; Data integrity

9. Search and Query Optimizations for Big Data (Assigned to: Rongjian Li)
[Du92] W. Du, R. Krishnamurthy, and M.-C. Shan. "Query Optimization in a Heterogeneous DBMS," Proc. Symp. VLDB, 1992, Vol. 92, pp. 277-291.
Search terms:

10. The MapReduce Paradigm for Big-Data Parallelism (Assigned to: Zhicheng Zhang)
[Dean10] J. Dean and S. Ghemawat, "MapReduce: A Flexible Data Processing Tool," Communications of the ACM, Vol. 53, No. 1, 2010, pp. 72-77.
Search terms: Data-parallel computing; Divide-and-conquer paradigm

11. Hadoop's Distributed File System for Big Data (Assigned to: Yulin Ou)
[Shva10] K. Shvachko, H. Kuang, S. Radia, and R. Chansler, "The Hadoop Distributed File System," Proc. 26th IEEE Symp. Mass Storage Systems and Technologies, 2010, pp. 1-10.
Search terms: Hadoop; Distributed file systems; Distributed databases

12. Big-Data Longevity and Compatibility Challenges (Assigned to: Brian Young)
[Garf04] T. Garfinkel, B. Pfaff, J. Chow, and M. Rosenblum, "Data Lifetime Is a Systems Problem," Proc. 11th ACM SIGOPS European Workshop, 2004, p. 10.
Search terms: Data preservation; Long-term data archiving

13. Advances in Intelligent Systems with Big Data (Assigned to: Pooja V. Kadam)
[OLea13] D. E. O'Leary, "Artificial Intelligence and Big Data," IEEE Intelligent Systems, Vol. 28, No. 2, 2013, pp. 96-99.
Search terms: Data-driven AI; Large training data-sets

14. FPGA-Based Accelerators for Big-Data Computing (Assigned to: Gokul P. Nallasami)
[Wang15] C. Wang, X. Li, and X. Zhou, "SODA: Software Defined FPGA Based Accelerators for Big Data," Proc. Design Automation & Test Europe Conf., 2015, pp. 884-887.
Search terms: Hardware accelerators; Application-specific coprocessors

15. Big-Data Aspects of Distributed Sensor Networks (Assigned to: Zhaoren Zeng)
[Taka14] D. Takaishi, H. Nishiyama, N. Kato, and R. Miura, "Toward Energy Efficient Big Data Gathering in Densely Distributed Sensor Networks," IEEE Trans. Emerging Topics in Computing, Vol. 2, No. 3, pp. 388-397, 2014.
Search terms: Ad-hoc networks; Data storage in sensor networks

16. Big-Data Challenges of Maps and Auto-Navigation (Assigned to: Jiaheng Tang)
[Shek12] S. Shekhar, V. Gunturi, M. R. Evans, and K. Yang, "Spatial Big-Data Challenges Intersecting Mobility and Cloud Computing." Proc. 11th ACM Int'l Workshop Data Engineering for Wireless and Mobile Access, 2012, pp. 1-6.
Search terms: Geographic information systems; GPS navigation

17. Shared-Memory Models for Big-Data Applications (Assigned to: Yang Zhao)
[Bell18] C. G. Bell and I. Nassi, "Revisiting Scalable Coherent Shared Memory," IEEE Computer, Vol. 51, No. 1, pp. 40-49, January 2018.
Search terms: Memory consistency; Distributed shared memory; Directory-based coherence

18. Edge Computing for Big Data (Assigned to: Leilai Shao)
[Shar17] S. K. Sharma and X. Wang, "Live Data Analytics With Collaborative Edge and Cloud Processing in Wireless IoT Networks," IEEE Access, Vol. 5, pp. 4621-4635, 2017.

Poster Presentation Tips [Does not apply to winter 2018]

Poster format

Here are some guidelines for preparing your research poster. The idea of the poster is to present your research results and conclusions thus far, get oral feedback during the session from the instructor and your peers, and to provide the instructor with something to comment on before your final report is due. Please send a PDF copy of the poster via e-mail by midnight on the poster presentation day.

Posters prepared for conferences must be colorful and eye-catching, as they are typically competing with dozens of other posters for the attendees' attention. Here is an example of a conference poster. Such posters are often mounted on a colored cardboard base, even if the pages themselves are standard PowerPoint slides. In our case, you should aim for a "plain" poster (loose sheets, to be taped to the wall in our classroom) that conveys your message in a simple and direct way. Eight to 10 pages, each resembling a PowerPoint slide, would be an appropriate goal. You can organize the pages into 2 x 4 (2 columns, 4 rows), 2 x 5, or 3 x 3 array on the wall. The top two of these might contain the project title, your name, course name and number, and a very short (50-word) abstract. The final two can perhaps contain your conclusions and directions for further work (including work that does not appear in the poster, but will be included in your research report). The rest will contain brief description of ideas, with emphasis on diagrams, graphs, tables, and the like, rather than text which is very difficult to absorb for a visitor in a very limited time span.

Grade Statistics

Chart All grades listed below are in [0.0, 4.3] (F to A+)
HW1 grades: Range = [C+, A], Mean = 3.16, Median = B
HW2 grades: Range = [B, A], Mean = 3.67, Median = A–
HW3 grades: Range = [B+, A], Mean = 3.65, Median = A–
HW4 grades: Range = [B, A], Mean = 3.63, Median = A–
HW5 grades: Range = [B+, A], Mean = 3.81, Median = A–
HW6 grades: Range = [B+, A], Mean = 3.81, Median = A
HW7 grades: Range = [D+, A], Mean = 3.58, Median = A–
HW8 grades: Range = [B–, A], Mean = 3.78, Median = A–
Research paper grades: Range = [C+, A+], Mean = 3.42, Median = B+
Course letter grades: Range = [B–, A], Mean = 3.47, Median = B+


Image of a reference book

Required text: B. Parhami, Introduction to Parallel Processing: Algorithms and Architectures, Plenum Press, 1999. Make sure that you visit the textbook's web page which contains an errata. Lecture slides are also available there.
Optional recommended book: T. Rauber and G. Runger, Parallel Programming for Multicore and Cluster Systems, 2nd ed., Springer, 2013. Because ECE 254B's focus is on architecture and its interplay with algorithms, this book constitutes helpful supplementary reading.
On-line: N. Matloff, Programming Parallel Machines: GPU, Multicore, Clusters, & More. [Link]
Research resources:
The follolwing journals contain a wealth of information on new developments in parallel processing: IEEE Trans. Parallel and Distributed Systems, IEEE Trans. Computers, J. Parallel & Distributed Computing, Parallel Computing, Parallel Processing Letters, ACM Trans. Parallel Computing. Also, see IEEE Computer and IEEE Concurrency (the latter ceased publication in late 2000) for broad introductory articles.
The following are the main conferences of the field: Int'l Symp. Computer Architecture (ISCA, since 1973), Int'l Conf. Parallel Processing (ICPP, since 1972), Int'l Parallel & Distributed Processing Symp. (IPDPS, formed in 1998 by merging IPPS/SPDP, which were held since 1987/1989), and ACM Symp. Parallel Algorithms and Architectures (SPAA, since 1988).
UCSB library's electronic journals, collections, and other resources

Miscellaneous Information

Motivation: The ultimate efficiency in parallel systems is to achieve a computation speedup factor of p with p processors. Although often this ideal cannot be achieved, some speedup is generally possible by using multiple processors in a concurrent (parallel or distributed) system. The actual speed gain depends on the system's architecture and the algorithm run on it. This course focuses on the interplay of architectural and algorithmic speedup techniques. More specifically, the problem of algorithm design for "general-purpose" parallel systems and its "converse," the incorporation of architectural features to help improve algorithm efficiency and, in the extreme, the design of algorithm-based special-purpose parallel architectures, are dealt with. The foregoing notions will be covered in sufficient detail to allow extensions and applications in a variety of contexts, from network processors, through desktop computers, game boxes, Web server farms, multiterabyte storage systems, and mainframes, to high-end supercomputers.

Catalog entry: 254B. Advanced Computer Architecture: Parallel Processing(4) PARHAMI. Prerequisites: ECE 254A. Lecture, 4 hours. The nature of concurrent computations. Idealized models of parallel systems. Practical realization of concurrency. Interconnection networks. Building-block parallel algorithms. Algorithm design, optimality, and efficiency. Mapping and scheduling of computations. Example multiprocessors and multicomputers.

History: The graduate course ECE 254B was created by Dr. Parhami, shortly after he joined UCSB in 1988. It was first taught in spring 1989 as ECE 594L, Special Topics in Computer Architecture: Parallel and Distributed Computations. A year later, it was converted to ECE 251, a regular graduate course. In 1991, Dr. Parhami led an effort to restructure and update UCSB's graduate course offerings in the area of computer architecture. The result was the creation of the three-course sequence ECE 254A/B/C to replace ECE 250 (Adv. Computer Architecture) and ECE 251. The three new courses were designed to cover high-performance uniprocessing, parallel computing, and distributed computer systems, respectively. In 1999, based on a decade of experience in teaching ECE 254B, Dr. Parhami published the textbook Introduction to Parallel Processing: Algorithms and Architectures (Website).
Offering of ECE 254B in winter 2017 (Link)
Offering of ECE 254B in winter 2016 (PDF file)
Offering of ECE 254B in winter 2014 (PDF file)
Offering of ECE 254B in winter 2013 (PDF file)
Offering of ECE 254B in fall 2010 (PDF file)
Offering of ECE 254B in fall 2008 (PDF file)
Offerings of ECE 254B from 2000 to 2006 (PDF file)