--------------------------------------------------------------------
CS 758 Programming Multicore Processors 
Fall 2012 Section 1
Instructor Mark D. Hill
--------------------------------------------------------------------

------------
Threads
------------

OUTLINE

------------------------------


Set up

* Comments on Cilk HW?

* Issues with GPU HW?

* Project ideas?

* GPU review/comments


Reprise of Thread/Process Pictures
----------------------------------

Definitions

   Thread -- PC
   Process -- address space

Basic Programming Models (draw pictures)
  Multitasking:         n x ( 1 thread/process) w/o communication
  Message-Passing       n x ( 1 thread/process) w/ messages
  Shared Memory:        n threads in 1 process (how communcate?)
  Shared Memory':       n x ( 1 thread/process) w/ "System V" shared memory
  Sequential:           1 thread/process with parallelizing software
  Data Parallel         1 thread/process with data parallel ops
  (generalization of SIMD) or n threads in lock-step w/ shared memory
  GPU SIMT (Single Instruction Multiple Threads)
  to first-order        n threads in lock-step w/ shared memory
                        best performance: program as data parallel
                        but threads can diverge -- dividing performance


The Problem with Threads by Edward Lee 2006
--------------------------------------------

Written before GP-GPU computing
Still applies for the state of non-GP-GPU computing

Thread 
* minor syntax extension
* MAJOR semantic extension
* Choice seems to governed by syntax than semantics

Box 1: Nondeterminism and threads
	* non-deteminism by default
	* assumes SC (subtle)
	Big Idea: Thread == sequential execution of a program p_i
		Simplifying, a thread t_i = a finite sequence of instrs ending with halt
				t_i = i_i0, i_i1, i_i2,… halt
		Two programs p_i and p_j are equivalent for a given input if they generate the same sequence of instructions
		That is t_i == t_j

		Enter non-determinism
		Are a pair of programs (p_1,p_2) equivalent to another pair (p'_1,p'_2)???
		Non-deterministic interleaving of instructions makes this intractable

		QED: No theory of equivalence is possible.
	REALLY?
		This theory doesn't even work for programs that don't halt
		Not useful for most programs of interest
		Why does the failure to extend to multithreading mean NO theory is possible?

CORBA and .NET shared name space as well

KEY ARGUMENT:
	Non-determinism is bad (even with message passing)
	Don't use non-determinism for deterministic aims


Java "Synchronized" method not enough
	Observer design pattern
		SetValue
			Change value and notify all listeners
		AddListener
			Update listener list
		Using Synchronized methods can lead to deadlock
			SetValue holds lock, calls valueChanged method, which can do ANYTHING
	
Need SW engineering practice restrict freedom
Design Patterns
	Map-Reduce good for what it does -- see below

Microsoft Dryad

Ptolemy stuff has not generally caught on
Good idea: deterministic by default

P.S.
transformational -- implements a function
reactive


Kahn Process Networks -- a formalize for deteministism

* Nodes connected by unbounded FIFO queues wherein node deterismiscally takes message for
a specific input queue
* Limited expressiveness
* with finite queues, hard to avoid deadlock

Coordination languages?