CS 334 Programming Languages Spring 2000 Lecture 22

CONCURRENT PROGRAMMING

Classification of machine architectures:

SISD - traditional von Neumann - one processor, one piece of data at a time,

SIMD - currently used supercomputers (typically synchronous)

pipeline (interleave operations) for speed-up or

array-processor - do same thing to all elts of array,

MIMD - most interesting to computer science (asynchronous).

1. Tightly coupled (shared memory)

Difficulties - synchronization - wait for one to finish for another to take output and start.

Generalization of sequential algorithm not necessarily fastest on parallel machine

Contention over resources - reader - writer problem.

2. Distributed computing system (typically no global memory)

Each processor has its own memory but share a common communications channel.

Processes

In one of three states: executing, blocked, or waiting.

Problems:

1. Synchronizing w/ other processes.

2. Communicate data.

With shared memory, communication typically via shared memory.

Problem: Mutual exclusion (reader-writer contention)

Distributed memory, communication via sending messages:
Problem: How to asynchronously send and receive messages?

Need mechanisms in OS to

1. Create and destroy processes.

2. Manage processes by scheduling on one or more processors.

3. Implement mutual exclusion (for shared memory).

4. Create and maintain communication channels between processors (for distributed memory).

Language mechanisms supporting concurrency

Three major mechanisms:

1. Semaphores (for mutual exclusion)

2. Monitors (for mutual exclusion)

3. Message passing (using "tasks")

Focus here on producer/consumer or bounded buffer problem.
Two processes cooperating, one by adding items to a buffer, the other removing items. Ensure not remove when nothing there and not overflow buffer as well.

Text also focuses on parallel matrix multiplication (read on own).

Text also discusses some ways of handling with simple extensions of existing languages:

• Parallel for loops

• library routines for spawning and managing processes.

Coroutines also worth noting (part of Modula-2). Iterators were a special case.

Idea is co-equal routines which pass control back and forth.
E.g., our Modula-2 has library supporting routines:

NewCoroutine(P: PROC; workspaceSize:CARDINAL; VAR q: COROUTINE);

Starts up new coroutine, q, by executing procedure P.

Transfer(VAR from, to: COROUTINE)

Transfers control from one coroutine to another.

Can have multiple coroutines executing same procedure or can all be distinct.

Usually run on single processor.

Can think of as supporting multi-tasking. Good for writing operating systems.

See Modula-2 code in text for bounded buffer problem with coroutines.

Semaphores

Three operations:

    InitSem(S: Semaphore; value: integer);
    Wait(S: Semaphore);
    Signal(S: Semaphore);

InitSem starts up semaphore with an initial (non-negative) value.

Wait(S): If S > 0 then S := S - 1 else suspend self

Signal(S): if processes are waiting, then wake up a process, else S := S + 1;

Think of Wait(S) as claiming a resource so that no one else can get it, while Signal(S) releases the resource.

In order to solve mutual exclusion problem, must ensure that Wait and Signal execute atomically (i.e., cannot be interrupted and no one else can execute at same time).

If start w/S = 1 then protect a critical region by:

    Wait(S);    -- grab token
    {Critical region}
    Signal(S);  -- release token

Solution to bounded buffer:

Suppose also have procedures:

    CreateProcess(p:PROC; workspacesize: CARDINAL);
        Creates nameless process
    StartProcesses;  -- starts all processes which have been created.
    Terminate;  -- stop execution of process

Main program:
CreateProcess(Producer,WorkSize);   -   - create at least one producer
CreateProcess(Consumer,WorkSize);   -- create at least one consumer
BufferStart := 1;  BufferEnd := 0
InitSem(NonEmpty, 0)    -- semaphore w/initial value of  0 to indicate empty
InitSem(NonFull, MaxBuffSize)   -- semaphore w/initial value of  size of buffer
InitSem(MutEx,1)        -- semaphore used for mutual exclusion
StartProcesses
end;

Procedure Producer;
begin
    loop
        read(ch)
        Wait(NonFull);
        Wait(MutEx);
        BufferEnd := BufferEnd MOD MaxBuffSize + 1;
        Buffer[BufferEnd] := ch;
        Signal(MutEx);
        Signal(NonEmpty);
    end loop;
end;

Procedure Consumer;
begin
    loop
        Wait(NonEmpty);
        Wait(MutEx);
        ch := Buffer[BufferStart];
        BufferStart := BufferStart MOD MaxBuffSize + 1;
        Signal(MutEx);
        Signal(NonFull);
        Write(ch)
    end loop
end;

Why is there a MutEx semaphore?

Technically it is not necessary here since Producer only changes BufferEnd, while Consumer only changes BufferStart, but if they both changed a count of the number of items in the buffer would be important to keep them from executing at the same time!

What would go wrong if you changed the order of the two Wait's at the beginning of either Producer or Consumer?

Biggest problem with semaphores is that they are too low level and unstructured. Accidentally reversing order or Wait'ing twice could be disastrous.

Monitors

The idea is to provide an ADT with "condition variables", each of which has an associated queue of processes and suspend (or delay) and continue ops for en and dequeuing processes. Suspend always stops current, continue starts up new if any waiting.

Concurrent Pascal uses monitors. Java "synchronized" similar.

• Continue by process in monitor causes it to be immediately suspended.

• Each queue can only hold one suspended process at a time. (owise error)

type buffer = monitor;

var store: array[1..MaxBuffSize] of char;
    BufferStart, BufferEnd, BufferSize: integer
    nonfull, nonempty: queue;

procedure entry insert(ch: char);
begin
    if BufferSize = MaxBuffSize then delay(nonfull);
    BufferEnd :=BufferEnd mod MaxBuffSize + 1;
    store[BufferEnd] := ch;
    BufferSize := BufferSize + 1;
    continue(nonempty)
end;

procedure entry delete(var ch: char);
begin
   if BufferSize = 0 then delay(nonempty);
   ch := store[BufferStart];
   BufferStart := BufferStart mod MaxBuffSize + 1;
   BufferSize := BufferSize -1;
   continue(nonfull);
end;

begin (* initialization *)
   BufferEnd := 0;
   BufferStart := 1;
   BufferSize := 0
end;

type producer = process (b: buffer);
var ch: char;
begin
    while true do begin
        read(ch);
        b.insert(ch)
    end;
end

type consumer = process(b: buffer);
var ch: char;
begin
    while true do begin
        b.delete(ch);
        write(ch)
    end
end;

var p: producer; q: consumer; b:buffer; begin init b, p(b), q(b) end.

Notice improved structure!

See text for simple way of emulating Semaphores w/ Monitors.