Programming Current and Future Multicore Processors

The reader is likely to be updated.

Background

John L. Hennessy and David A. Patterson, Computer Architecture: A Quantitative Approach, Morgan Kaufmann Publishers, Third Edition, 2002. From Chapter 6 Multiprocessors and Thread-Level Parallelism, pp. 528-664:

1. Introduction (read now)
2. Characteristics of Application Domains (read now)
3. Symmetric Shared-Memory Architectures (read now)
4. Performance of Symmetric Shared-Memory Multiprocessors (skim now; read later)
5. Distributed Shared-Memory Architectures (read now)
6. Performance of Distributed Shared-Memory Architectures (skim now; read later)
7. Synchronization (read now)
8. Models of Memory Consistency: An Introduction (optional)
9. Multithreading: Exploiting Thread-Level Parallelism within a Processor (read now)
10. Crosscutting Issues (optional)
11. Putting It All Together: Sun's Wildfire Prototype (optional)
12. Another View: Multithreading in a Commercial Server (optional)
13. Another View: Embedded Multiprocessors (optional)
14. Fallacies and Pitfalls (read now)
15. Concluding Remarks (read now)
16. Historical Perspective and References (skim now; read later)

Introduction

Herb Sutter, The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software, Dr. Dobb's Journal, 30(3), March 2005. Html.

Herb Sutter and James Larus, Software and the Concurrency Problem, ACM Queue, September 2005, Online PDF for University of Wisconsin only.

Introduction to Parallel Computing LLNL Web Site (html). Reference.

Multicore processors

Poonacha Kongetira, Kathirgamar Aingaran, Kunle Olukotun, Niagara: A 32-Way Multithreaded Sparc Processor, IEEE Micro, March-April 2005, pp. 21-29. Online PDF for University of Wisconsin only.

Lance Hammond, Ben Hubbert, Michael Siu, Manohar Prabhu, Mike Chen, and Kunle Olukotun, The Stanford Hydra CMP, IEEE Micro, March-April 2000, pp. 71-84. Online PDF for University of Wisconsin only.

Luiz Andre Barroso, et al., Piranha: A Scalable Architecture Based on Single-Chip Multiprocessing, Proc. International Symposium on Computer Architecture, June 2000, pp. 282-293. Online PDF for University of Wisconsin only. Reference.

David A. Wood and Mark D. Hill, Cost-Effective Parallel Computing, IEEE Computer, February 1995. Final Scanned PDF or Near-Final Latex PDF. Reference.

Pthreads

POSIX Threads Programming, Web Site (html). Reference.

Steven Cameron Woo, Moriyoshi Ohara, Evan Torrie, Jaswinder Pal Singh, and Anoop Gupta, The SPLASH-2 Programs: Characterization and Methodological Considerations, Proc. International Symposium on Computer Architecture, June 1995. Online PDF for University of Wisconsin only.

John M. Mellor-Crummey and Michael L. Scott, Algorithms for Scalable Synchronization on Shared-Memory Multiprocessors ACM Trans. on Computer Systems. February 1991, pp. 21-65. Online PDF for University of Wisconsin only.

Thomas E. Anderson, Brian N. Bershad, Edward D. Lazowska, Henry M. Levy, Scheduler Activations: Effective Kernel Support for the User-level Management of Parallelism, Proc. Symposium on Operating System Principles, October 1991. Online PDF for University of Wisconsin only.

Atul Adya, Jon Howell, Marvin Theimer, Bill Bolosky, John Douceur, Cooperative Task Management without Manual Stack Management, or Event-driven Programming is not the Opposite of Threaded Programming, Proc. USENIX, June 2002. Online PDF for University of Wisconsin only. Reference.

OpenMP

Leonardo Dagumand and Ramesh Menon, OpenMP: An Industry Standard API for Shared Memory Programming IEEE Computational Science and Engineering, Jan-Mar, 1998. Online PDF for University of Wisconsin only.

LLNL OpenMP Tutorial, Web Site (html).

OpenMP: Simple, Portable, Scalable SMP Programming, Web Site (html). Reference.

Cilk/TBB

Cilk: An Efficient Multithreaded Runtime System, Proceedings of PPoPP'95, 1995, pp. 207-216. Online PDF for University of Wisconsin only.

TBB Tutorial Intel Web Document. Online PDF for University of Wisconsin only.

MapReduce

Luiz Andre Barroso, Jeffrey Dean, Urs Holzle, Web Search For a Planet: The Google Cluster Architecture, IEEE Micro, 23(2):22-28, March-April 2003. Online PDF for University of Wisconsin only. Reference.

Colby Ranger, Ramanan Raghuraman, Arun Penmetsa, Gary Bradski, Christos Kozyrakis, Evaulating MapReduce for Multi-core and Multiprocessor Systems, Proceedings of the 13th Intl. Symposium on High-Performance Computer Architecture (HPCA), February 2007. (PDF)

Locking and Threads

Gray, J., et. al., "Granularity of Locks and Degrees of Consistency in a Shared Database," Readings in Database Systems, pp. 175-193  

Bayer, R. and M. Schkolnick, Concurrency of Operations on B-Trees, Readings in Database Systems, pp. 127-139.

Transactional Memory

Unlocking Concurrency Ali-Reza Adl-Tabatabai, Christos Kozyrakis, Bratin Saha December 2006 ACM Queue, Volume 4 Issue 10

McRT-STM: a high performance software transactional memory system for a multi-core runtime. Bratin Saha, Ali-Reza Adl-Tabatabai, Richard L. Hudson, Chi Cao Minh, Ben Hertzberg, PPOPP 2006: 187-197

Transactional Memory Coherence and Consistency Lance Hammond, Vicky Wong, Mike Chen, Ben Hertzberg, Brian D. Carlstrom, John D. Davis, Manohar K. Prabhu, Honggo Wijaya, Christos Kozyrakis, and Kunle Olukotun Proceedings of the 31st Annual International Symposium on Computer Architecture, München, Germany, June 19-23, 2004.

Kevin E. Moore, Jayaram Bobba, Michelle J. Moravan, Mark D. Hill, and David A. Wood, LogTM: Log-based Transactional Memory, Submitted to Proc. Symposium on High-Performance Computer Architecture, February 2006. Online PDF.

Peter Damron, Alexandra Fedorova, Yossi Lev, Victor Luchangco, Mark Moir, and Daniel Nussbaum, Hybrid transactional memory, Proceedings of the 12th international conference on Architectural support for programming languages and operating systems (ASPLOS), 2006. Online PDF.

Transactional Memory Online, Web Site (html). Reference.

Maurice Herlihy and J. Eliot B. Moss, Transactional Memory: Architectural Support for Lock-Free Data Structures, Proc. International Symposium on Computer Architecture, May 1993. Online PDF for University of Wisconsin only. Reference Only

Albert Chang and Mark F. Mergen, 801 Storage: Architecture and Programming, ACM Trans. on Computer Systems, February 1988. Online PDF for University of Wisconsin only. Concentrate on transactional issues (e.g., Section 3.3). Reference Only

Ravi Rajwar and James R. Goodman, Speculative Lock Elision: Enabling Highly Concurrent Multithreaded Execution, Proc. 34th Intl. Symposium on Microarchitecture, December 2001. Online PDF for University of Wisconsin only. Reference Only

Ravi Rajwar and James R. Goodman, Transactional Lock-Free Execution of Lock-Based Programs, Proc. Architectural Support for Programming Languages and Operating Systems, October 2002. Online PDF for University of Wisconsin only. Reference Only

Virendra J. Marathe and Michael L. Scott, A Qualitative Survey of Modern Software Transactional Memory Systems, University of Rochester Computer Science Department TR 839, June 2004. Online PDF for University of Wisconsin only. Reference Only

Lance Hammond, Vicky Wong, Mike Chen, Brian D. Carlstrom, John D. Davis, Ben Hertzberg, Manohar K. Prabhu, Honggo Wijaya, Christos Kozyrakis, and Kunle Olukotun, Transactional Memory Coherence and Consistency, Proc. International Symposium on Computer Architecture, June 2004. Online PDF for University of Wisconsin only. Reference Only

Lance Hammond, Brian D. Carlstrom, Vicky Wong, Ben Hertzberg, Mike Chen, Christos Kozyrakis, and Kunle Olukotun, Programming with Transactional Coherence and Consistency (TCC), Proc. Architectural Support for Programming Languages and Operating Systems, October 2004. Online PDF for University of Wisconsin only. Reference Only

C. Scott Ananian, Krste Asanovic, Bradley C. Kuszmaul, Charles E. Leiserson, and Sean Lie, Unbounded Transactional Memory, Proc. Symposium on High-Performance Computer Architecture, February 2005. Online PDF for University of Wisconsin only. Reference Only

Ravi Rajwar, Maurice Herlihy, and Konrad Lai, Virtualizing Transactional Memory, Proc. International Symposium on Computer Architecture, June 2005. Online PDF for University of Wisconsin only. Reference Only

Maurice Herlihy, Victor Luchangco, and Mark Moir, Obstruction-Free Synchronization: Double-Ended Queues as an Example Proc. International Conference on Distributed Computing Systems, Online PDF for University of Wisconsin only. Reference Only

Tim Harris, Design Choices for Language-based Transactions, Microsoft Techical Report UCAM-CL-TR-572, August 2003. PDF. Reference Only

Carlstrom et al., Transactional Execution of Java Programs, OOPSLA Workshop on Synchronization and Concurrency in Object-Oriented Languages (SCOOL), October 2005. PDF. Reference Only

Students Present

As students present papers, these paper will be assigned for reading.

"William Thies, Michal Karczmarek, and Saman Amarasinghe. StreamIt: A Language for Streaming Applications. In Proceedings of the 2002 International Conference on Compiler Construction." Online PDF for University of Wisconsin only.

Herlihy, M. A Methodology for Implementing Highly Concurrent Data Structures. PPOPP '90. PDF

N. Hardavellas, I. Pandis, R. Johnson, N. G. Mancheril, A. Ailamaki, and B. Falsafi. Database servers on chip multiprocessors: Limitations and opportunities. In 3rd Biennial Conference on Innovative Data Systems Research (CIDR), 2007. Online PDF for University of Wisconsin only.

David Luebke and Greg Humphreys, "How GPUs Work": PDF

Adl-Tabatabai, A., et al., Compiler and Runtime Support for Efficient Software Transactional Memory, PLDI 2006, pp. 26-37. PDF