CS 537
Lecture Notes Part 3
Processes and Synchronization

Contents

• Using Processes
• What is a Process?
• Why Use Processes
• Creating Processes
• Process States
• Synchronization
  • Race Conditions
  • Semaphores
  • The Bounded Buffer Problem
  • The Dining Philosophers
  • Monitors
  • Messages

The text book mixes a presentation of the features of processes of interest to programmers creating concurrent programs with discussion of techniques for implementing them. The result is (at least to me) confusing. I will attempt to first present processes and associated features from the user's point of view with as little concern as possible for questions about how they are implemented, and then turn to the question of implementing processes.

Using Processes

A process is a “little bug” that crawls around on the program executing the instructions it sees there. Normally (in so-called sequential programs) there is exactly one process per program, but in concurrent programs, there may be several processes executing the same program. The details of what constitutes a “process” differ from system to system. The main difference is the amount of private state associated with each process. Each process has its own program counter, the register that tells it where it is in the program. It also needs a place to store the return address when it calls a subroutine, so that two processes executing the same subroutine called from different places can return to the correct calling points. Since subroutines can call other subroutines, each process needs its own stack of return addresses.

Processes with very little private memory are called threads or light-weight processes. At a minimum, each thread needs a program counter and a place to store a stack of return addresses; all other values could be stored in memory shared by all threads. At the other extreme, each process could have its own private memory space, sharing only the read-only program text with other processes. This essentially the way a Unix process works. Other points along the spectrum are possible. One common approach is to put the local variables of procedures on the same private stack as the return addresses, but let all global variables be shared between processes. A stack frame holds all the local variables of a procedure, together with an indication of where to return to when the procedure returns, and an indication of where the calling procedure's stack frame is stored. This is the approach taken by Java threads. Java has no global variables, but threads all share the same heap. The heap is the region of memory used to hold objects allocated by new. In short, variables declared in procedures are local to threads, but objects are all shared. Of course, a thread can only “see” an object if it can reach that object from its “base” object (the one containing its run method, or from one of its local variables.

    class Worker implements Runnable {
        Object arg, other;
        Worker(Object a) { arg = a; }
        public void run() {
            Object tmp = new Object();
            other = new Object();
            for(int i = 0; i < 1000; i++) // do something
        }
    }
    class Demo {
        static public void main(String args[]) {
            Object shared = new Object();

            Runnable worker1 = new Worker(shared);
            Thread t1 = new Thread(worker1);

            Runnable worker2 = new Worker(shared);
            Thread t2 = new Thread(worker2);

            t1.start(); t2.start();
            // do something here
        }
    }

Other names sometimes used for processes are job or task.

It is possible to combine threads with processes in the same system. For example, when you run Java under Unix, each Java program is run in a separate Unix process. Unix processes share very little with each other, but the Java threads in one Unix process share everything but their private stacks.

Why Use Processes

Processes are basically just a programming convenience, but in some settings they are such a great convenience, it would be nearly impossible to write the program without them. A process allows you to write a single thread of code to get some task done, without worrying about the possibility that it may have to wait for something to happen along the way. Examples:

A server providing services to others.

One thread for each client.

A timesharing system.

One thread for each logged-in user.

A real-time control computer controlling a factory.

One thread for each device that needs monitoring.

A network server.

One thread for each connection.

When a new process is created, it needs to know where to start executing. In Java, a thread is given an object when it is created. When it is started, it starts execution at the beginning of the run method of that object.

In Unix, a new process is started with the fork() command. It starts a new process running in the same program, starting at the statement immediately following the fork() call. After the call, both the parent (the process that called fork()) and the child are both executing at the same point in the program. The child is given its own memory space, which is initialized with an exactly copy of the memory space (globals, stack, heap objects) of the parent. Thus the child looks like an exact clone of the parent, and indeed, it's hard to tell them apart. The only difference is that fork() returns 0 in the child, but a non-zero value in the parent.

    #include <iostream.h>
    #include <unistd.h>

    char *str;

    int f() {
        int k;

        k = fork();
        if (k == 0) {
            str = "the child has value ";
            return 10;
        }
        else {
            str = "the parent has value ";
            return 39;
        }
    }

    main() {
        int j;
        str = "the main program ";
        j = f();
        cout << str << j << endl;
    }

    the parent has value 39
    the child has value 10

Some systems require every process to have a parent. What happens when a the parent dies before the child? One possibility is cascading termination. Unix uses a different model. An “orphan” process is adopted by a special process called “init” that is created at system startup and only goes away at system shutdown.

Synchronization

Race Conditions

    void deposit(int amount) {
        balance += amount;
    }

    Load  Reg, balance
    Add   Reg, amount
    Store Reg, balance

    P1 loads 100 into its register
    P2 loads 100 into its register
    P1 adds 10 to its register, giving 110
    P2 adds 20 to its register, giving 120
    P1 stores 110 in balance
    P2 stores 120 in balance

This kind of bug, which only occurs under certain timing conditions, is called a race condition. It is an extremely difficult kind of bug to track down (since it may disappear when you try to debug it) and may be nearly impossible to detect from testing (since it may occur only extremely rarely). The only way to deal with race conditions is through very careful coding. To avoid these kinds of problems, systems that support processes always contain constructs called synchronization primitives.

Semaphores

One of the earliest and simplest synchronization primitives is the semaphore. We will consider later how semaphores are implemented, but for now we can treat them like a Java object that hides an integer value and only allows three operations: initialization to a specified value, increment, or decrement.³

    class Semaphore {
        private int value;
        public Semaphore(int v) { value = v; }
        public void up() { /* ... */ }
        public void down() { /* ... */ };
    }

1. The value is never permitted to be negative. If the value is zero when a process calls down, that process is forced to wait (it goes into blocked state) until some other process calls up on the semaphore.
2. The up and down operations are atomic: A correct implementation must make it appear that they occur instantaneously. In other words, two operations on the same semaphore attempted at the same time must not be interleaved. (In the case of a down operation that blocks the caller, it is the actual decrementing that must be atomic; it is ok if other things happen while the calling process is blocked).

    shared Semaphore mutex = new Semaphore(1);
    void deposit(int amount) {
        mutex.down();
        balance += amount;
        mutex.up();
    }

    Semaphore mutex = 1;

1. The shared Buffer data structure is not screwed up by race conditions in accessing it.
2. Consumers don't try to remove objects from Buffer when it is empty.
3. Producers don't try to add objects to the Buffer when it is full.

    class Buffer {
        private Object[] elements;
        private int size, nextIn, nextOut;
        Buffer(int size) {
            this.size = size;
            elements = new Object[size];
            nextIn = 0;
            nextOut = 0;
        }
        public void add(Object o) {
            elements[nextIn++] = o;
            if (nextIn == size) nextIn = 0;
        }
        public Object remove() {
            Object result = elements[nextOut++];
            if (nextOut == size) nextOut = 0;
            return result;
        }
    }

    shared Buffer b = new Buffer(10);
    Semaphore
        mutex = 1,
        empty = 10,
        full = 0;
    
    class Producer implements Runnable {
        Object produce() { /* ... */ }
        public void run() {
            Object item;
            for (;;) {
                item = produce();
                empty.down();
                mutex.down();
                b.add(item);
                mutex.up();
                full.up();
            }
        }
    }
    class Consumer implements Runnable {
        void consume(Object o) { /* ... */ }
        public void run() {
            Object item;
            for (;;) {
                full.down();
                mutex.down();
                item = b.remove();
                mutex.up();
                empty.up();
                consume(item);
            }
        }
    }

    while (b.empty()) { /* do nothing */ }
    mutex.down();
    /* as before */

There is one more fine point to notice here: Suppose we reversed the down operations in the consumer

    mutex.down();
    full.down();

    class Philosopher implements Runnable {
        int i;  // which philosopher
        public void run() {
            for (;;) {
                think();
                take_forks(i);
                eat();
                put_forks(i)
            }
        }
    }

    Semaphore fork[5] = 1;
    void take_forks(int i) {
        fork[i].down();
        fork[i+1].down();
    }
    void put_forks(int i) {
        fork[i].up();
        fork[i+1].up();
    }

There's a very simple solution: Instead of trying for the right fork first, try for the lower numbered fork first. We will show later that this solution cannot lead to deadlock.

This solution, while deadlock-free, is still not as good as it could be. Consider again the situation in which all philosophers get hungry at the same time and pick up their lower-numbered fork. Both philosopher 0 and philosopher 4 try to grab fork 0 first. Suppose philosopher 0 wins. Since philosopher 4 is stuck waiting for fork 0, philosopher 3 will be able to grab both is forks and start eating.

Dijkstra suggests a better solution. More importantly, he shows how to derive the solution by thinking about two goals of any synchronization problem:

Safety

Make sure nothing bad happens.

Liveness

Make sure something good happens whenever it can.

Safety

For all i, !(state[i]==EATING && state[i+1]==EATING)

Liveness

For all i, !(state[i]==HUNGRY && state[i-1]!=EATING && state[i+1]!=EATING)

With this observation, the solution almost writes itself (See also Figure 2-33 on page 127 of Tanenbaum.)

    Semaphore mayEat[5] = { 0, 0, 0, 0, 0};
    Semaphore mutex = 1;
    final static public int THINKING = 0;
    final static public int HUNGRY = 1;
    final static public int EATING = 2;
    int state[5] = { THINKING, THINKING, THINKING, THINKING, THINKING };
    void take_forks(int i) {
        mutex.down();
        state[i] = HUNGRY;
        test(i);
        mutex.up();
        mayEat[i].down();
    }
    void put_forks(int i) {
        mutex.down();
        state[i] = THINKING;
        test(i==0 ? 4 : i-1);   // i-1 mod 5
        test((i==4 ? 0 : i+1);  // i+1 mod 5
        mutex.up();
    }
    void test(int i) {
        if (state[i]==HUNGRY && state[i-1]!=EATING && state[i+1] != EATING) {
            state[i] = EATING;
            mayEat[i].up();
        }
    }

Although semaphores are all you need to solve lots of synchronization problems, they are rather “low level” and error-prone. As we saw before, a slight error in placement of semaphores (such as switching the order of the two down operations in the Bounded Buffer problem) can lead to big problems. It is also easy to forget to protect shared variables (such as the bank balance or the buffer object) with a mutex semaphore. A better (higher-level) solution is provided by the monitor (also invented by Dijkstra).

If you look at the example uses of semaphores above, you see that they are used in two rather different ways: One is simple mutual exclusion. A semaphore (always called mutex in our examples) is associated with a shared variable or variables. Any piece of code that touches these variables is preceded by mutex.down() and followed by mutex.up(). Since it's hard for a programmer to remember to do this, but easy for a compiler, why not let the compiler do the work?⁵

    monitor class BankAccount {
        private int balance;
        public void deposit(int amount) {
            balance += amount;
        }
        // etc
    }

        Semaphore mutex = 1;

The other way semaphores are used is to block a process when it cannot proceed until another process does something. For example, a consumer, on discovering that the buffer is empty, has to wait for a producer; a philosopher, on getting hungry, may have to wait for a neighbor to finish eating. To provide this facility, monitors can have a special kind of variable called a condition variable.

    interface Condition {
        public void signal();
        public void wait();
    }

1. A semaphore counts the number of excess up operations, but a signal operation on a condition variable has no effect unless some process is waiting. A wait on a condition variable always blocks the calling process.
2. A wait on a condition variable atomically does an up on the monitor mutex and blocks the caller. In other words if c is a condition variable c.wait() is rather like mutex.up(); c.down(); except that both operations are done together as a single atomic action.

    monitor BoundedBuffer {
        private Buffer b = new Buffer(10);
        private int count = 0;
        private Condition nonfull, nonempty;
        public void add(Object item) {
            if (count == 10)
                nonfull.wait();
            b.add(item);
            count++;
            nonempty.signal();
        }
        public Object remove() {
            if (count == 0)
                nonempty.wait();
            item result = b.remove();
            count--;
            nonfull.signal();
            return result;
        }
    }

This strategy, while nice for avoiding certain kinds of errors, is very inefficient. As we will see when we consider implementation, it is expensive to switch processes. Consider what happens when a consumer is blocked on the nonempty condition variable and a producer calls add.

• The producer adds the item to the buffer and calls nonempty.signal().
• The producer is immediately blocked and the consumer is allowed to continue.
• The consumer removes the item from the buffer and leaves the monitor.
• The producer wakes up, and since the signal operation was the last statement in add, leaves the monitor.

To avoid this inefficiency, all recent implementations of monitors replace signal with notify. The notify operation is like signal in that it awakens a process waiting on the condition variable if there is one and otherwise does nothing. But as the name implies, a notify is a “hint” that the associated logical condition might be true, rather than a guarantee that it is true. The process that called notify is allowed to continue. The process awakened by the notify is also allowed to continue, but it cannot precede until it it can regain the mutex on the monitor. By that time, the logical condition may no longer be true. Perhaps the signaling process did something that make it false, or perhaps it notified another process that got the mutex, entered the monitor, and made the logical condition false. Since the logical condition might not be true anymore, the waiter needs to recheck it when it wakes up. For example the Bounded Buffer monitor should be rewritten to replace

    if (count == 10) {
        nonfull.wait();
    }

    while (count == 10) {
        nonfull.wait();
    }

Java has built into it something like this, but with two key differences. First, instead of marking a whole class as monitor, you have to remember to mark each method as synchronized. Every object is potentially a monitor. Second, there are no explicit condition variables. In effect, every monitor has exactly one anonymous condition variable. Instead of writing c.wait() or c.notify(), where c is a condition variable, you simply write wait() or notify(). A solution to the Bounded Buffer problem in Java might look like this:⁶

    class BoundedBuffer {
        private List b = new ArrayList(10);
        private int count = 0;
        synchronized public void add(Object item) {
            while (count == 10) {
                wait();
            }
            b.add(item);
            count++;
            notifyAll();
        }
        synchronized public Object remove() {
            while (count == 0) {
                wait();
            }
            Object result = b.remove(0);
            count--;
            notifyAll();
            return result;
        }
    }

This BoundedBuffer solution is not correct if it uses notify instead of notifyAll. Consider a system with 20 consumer threads and one producer and suppose the following sequence of events occurs.

1. All 20 consumer threads call remove. Since the buffer starts out empty, they all call wait and stop running.
2. The producer thread calls add 11 times. Each of the first 10 times, it adds an object to the buffer and wakes up one of the waiting consumers. The 11th time, it finds that count == 10 and calls wait. Unlike signal, which blocks the caller, notify allows the producer thread to continue, so it may finish this step before any of the awakened consumer threads resume execution.
3. Each of the 10 consumer threads awakened in Step 2 acquires the monitor lock, re-tests the condition count == 0, finds it false, removes an object from the buffer, decrements count, and calls notify. Java makes no promises which thread is awakened by each notify, but it is possible that each notify will awaken one of the remaining 10 consumer threads blocked in Step 1 rather than the producer thread blocked in Step 2.
4. Each of the consumer threads awakened in Step 3 finds that count == 0 and calls wait again.

As another example, here's a version of Dijkstra's solution to the dining philosophers problem using monitors.

    class Philosopher implements Runnable {
        private int id;
        private static DiningRoom dr = new DiningRoom();
        public Philosopher(int id) {
            this.id = id;
        }
        public void think() { ... }
        public void run() {
            for (int i=0; i<100; i++) {
                think();
                dr.dine(id);
            }
        }
    }
    monitor DiningRoom {
        final static private int THINKING = 0;
        final static private int HUNGRY = 1;
        final static private int EATING = 2;
        private int[] state = { THINKING, THINKING, ... };
        private Condition[] ok = new Condition[5];

        private void eat(int p) { ... }
            
        public void dine(int p) {
            state[p] = HUNGRY;
            test(p);
            while (state[p] != EATING)
                try { ok[p].wait(); }
                catch (InterruptedException e) { e.printStackTrace(); }
            eat();
            state[p] = THINKING;
            test((p+4)%5); // (p-1) mod 5
            test((p+1)%5); // (p+1) mod 5
        }

        private void test(int p) {
            if (state[p] == HUNGRY
                && state[(p+1)%5] != EATING
                && state[(p+4)%5] != EATING
            ) {
                state[p] = EATING;
                ok[p].notify();
            }
        }
    }

When a philosopher finishes eating, he calls test for each of his neighbors. Each call checks to see if the neighbor is hungry and able to eat. If so, it sets the neighbor's state to EATING and notify's the neighbor's ok condition in case he is already waiting.

This solution is fairly simple and easy to read. Unfortunately, it is wrong! The problem is that this solution only lets one philosopher at a time eat. The call to eat is inside the monitor method dine, so while a philosopher is eating, no other thread will be able to enter the DiningRoom. We will translate this program into Java, and while we're at it, fix the bug.

This solution also uses two non-Java features: the array ok of Condition variables and the keyword monitor. Java does not have a Condition type. Instead it has exactly one anonymous condition variable per monitor. This non-Java feature is (surprisingly) easy to fix. Just get rid of all mention of the array ok (e.g., ok[p].wait() becomes simply wait()) and change notify() to notifyAll(). Now, whenever any philosopher's state is changed to EATING, all blocked philosophers are awakened. Those whose states are still HUNGRY will simply go back to sleep. (Now you see why we wrote while (state[p] != EATING) rather than if (state[p] != EATING)). This solution is a little less efficient, but not enough to worry about. If there were 10,000 philosophers, and if a substantial fraction of them were blocked most of the time, we would have more to worry about, and perhaps we would have to search for a more efficient solution.

Instead of the keyword monitor, Java lets you decide which methods acquire the monitor lock by marking them synchronized. When a philosopher is deciding who gets to eat, he must be protected from a similar activity by any other philosopher in the room, because the decision requires messing with the shared state array. But an eating philosopher does not require mutual exclusion, so he should not hold the monitor lock while executing eat. Our solution is to break the dine method into two pieces: one piece that grabs the forks and another piece that releases them. The dine method no longer needs to be synchronized.

    class DiningRoom {
        final static private int THINKING = 0;
        final static private int HUNGRY = 1;
        final static private int EATING = 2;
        private int[] state = { THINKING, THINKING, ... };

        private void eat(int p) { ... }
            
        public void dine(int p) {  // Note: public but not synchronized
            grabForks(p);
            eat(p);
            releaseForks(p);
        }

        private synchronized void grabForks(int p) {
            state[p] = HUNGRY;
            test(p);
            while (state[p] != EATING)
                try { wait(); }
                catch (InterruptedException e) { e.printStackTrace(); }
        }

        private synchronized void releaseForks(int p) {
            state[p] = THINKING;
            test((p+4)%5);
            test((p+1)%5);
        }

        private void test(int p) {
            if (state[p] == HUNGRY
                && state[(p+1)%5] != EATING
                && state[(p+4)%5] != EATING
            ) {
                state[p] = EATING;
                notify();
            }
        }
    }

Since shared variables are such a source of errors, why not get rid of them altogether? In this section, we assume there is no shared memory between processes. That raises a new problem. Instead of worrying about how to keep processes from interfering with each other, we have to figure out how to let them cooperate. Systems without shared memory provide message-passing facilities that look something like this:

    send(destination, message);
    receive(source, message_buffer);

Naming

How are destination and source specified? Each process may directly name the other (direct naming), or there may be some sort of mailbox or message queue object to be used as the destination of a send or the source of a receive (indirect naming). The “mailbox” is known by many names in different systems, including channel, pipe, silo, fifo, particularly when receivers are guaranteed to get messages in the same order they were placed into the mailbox. A mailbox is also called a “port”, particularly if only one process is allowed to receive from any one mailbox. In such cases, you can think of each mailbox as a way for messages to get into a process. From the sender's perspective, there isn't much difference between sending to a port and to the associated process, but from the receiver can use multiple ports to help sort messages. For example, it could use different ports for different categories of requests, only receiving messages from categories it is ready to handle.

Some systems allow a set of destinations (called multicast and meaning “send a copy of the message to each destination”) and/or a set of sources, meaning “receive a message from any one of the sources.” A particularly common feature is to allow source to be “any”, meaning that the receiver is willing to receive a message from any other process that is willing to send a message to it.

Synchronization

Does send (or receive) block the sender, or can it immediately continue? One common combination is non-blocking send together with blocking receive. Another possibility is rendezvous, in which both send and receive are blocking. Whoever gets there first waits for the other one. When a sender and matching receiver are both waiting, the message is transferred and both are allowed to continue.

Buffering

Are messages copied directly from the sender's memory to the receiver's memory, or are first copied into some sort of “system” memory in between? If the system does provide space for messages in transit, it there some bound on the space provided?

Message Size

Is there an upper bound on the size of a message? Some systems have small, fixed-size messages to send signals or status information and a separate facility for transferring large blocks of data.

Message-based communication between processes is particularly attractive in distributed systems (such as computer networks) where processes are on different computers and it would be difficult or impossible to allow them to share memory. But it is also used in situations where processes could share memory but the operating system designer chose not allow sharing. One reason is to avoid the bugs that can occur with sharing. Another is to build a wall of protection between processes that don't trust each other. Some systems even combine message passing with shared memory. A message may include a pointer to a region of (shared) memory. The message is used as a way of transferring “ownership” of the region. There might be a convention that a process that wants to access some shared memory has to request permission from its current owner (by sending a message).

Unix is a message-based system (at the user level). Processes do not share memory but communicate through pipes.⁸ A pipe looks like an output stream connected to an input stream by a chunk of memory used to make a queue of bytes. One process sends data to the output stream the same way it would write data to a file, and another reads from it the way it would read from a file. In the terms outlined above, naming is indirect (with the pipe acting as a mailbox or message queue), send (called write in Unix) is non-blocking, while receive (called read) is blocking, and there is buffering in the operating system. In most versions of Unix, the amount of buffering is limited to a few kilobytes (and send blocks if that amount is exceeded) but in some, the amount of space is effectively unbounded. At first glance it would appear that the message size is unbounded, but it would actually be more accurate to say each “message” is one byte. The amount of data sent in a write or received in a read is unbounded, but the boundaries between writes are erased in the pipe: If the sender does three writes of 60 bytes each and the receive does two reads asking for 100 bytes, it will get back the first 100 bytes the first time and the remaining 80 bytes the second time.

Here's a puzzle: A well-known system uses indirect naming, blocking receive and non-blocking send, unlimited buffering of messages, and messages limited to zero bits in size. What is a mailbox in this system called?

Answer: a semaphore! Send is called “up” and receive is called “down”. The messsage and message_buffer arguments are omitted because messages contain no data.

Continued...

¹I'm using the term “private” informally here. The variable tmp is not a field of a class but rather a local variable, so it cannot be declared public, private, etc. It is “private” only in the sense that no other thread has any way of getting to this object.

²Note that this method is deprecated, which means you should never use it!

³ In the original definition of semaphores, the up and down operations were called V() and P(), respectively, but people had trouble remembering which was which. Some books call them signal and wait, but we will be using those names for other operations later.

⁴ Remember, this is not really legal Java. We will show a Java solution later.

⁵ Monitors are not available in this form in Java. We are using Java as a vehicle for illustrating various ideas present in other languages. See the discussion of monitors later for a similar feature that is available in Java.

⁶ We use the Java interface List and class ArrayList rather than the ad hoc Buffer class we defined above.

⁷The Java language specification says that if any threads are blocked on wait() in an object, a notify in that object will wake up exactly one thread. It does not say that it has to be any particular thread, such as the one that waited the longest. In fact, some Java implementations actually wake up the thread that has been waiting the shortest time! Similarly, if more than one process is trying to acquire the monitor lock, there is no guarantee which one will win. In particular, if processes A and B are blocked in wait and process C calls notifyAll, A and B will compete for the lock, and will eventually restart one at a time, but in unspecified order. In fact, process C or some other unrelated process may very well call one of the synchronized methods and grab the monitor lock before A or B.

⁸ There are so many versions of Unix that just about any blanket statement about Unix is sure to be a lie. Some versions of Unix allow memory to be shared between processes, and some have other ways for processes to communicate other than pipes.