Layers of Logic

14.6

Waiting for Something to Happen

A lock makes threads take turns. It gives one no way to tell another that the thing it was waiting for has arrived. That needs two more methods, and one rule that looks like a typo.

Core20 min read4 exercises
01

Previously on

Section 14.5 gave you a lock, and it does exactly one thing: one thread at a time.

It has no way to say the thing you were waiting for is here.

02

The problem

A queue with one thread putting jobs in and another taking them out. The consumer has to handle an empty queue.

synchronized (lock) {
    while (queue.isEmpty()) {
        // now what?
    }
    Job job = queue.removeFirst();
}

Spinning is the first idea and it is the worst one. An empty loop burns a whole core checking a condition thousands of times a microsecond.

Worse, it can never succeed. The consumer is holding the lock. The producer needs that lock to add anything. The consumer spins forever waiting for something the producer cannot do, and the program stops with one core at a hundred percent.

Sleeping is the second idea and it has the same fault.

synchronized (lock) {
    while (queue.isEmpty()) {
        Thread.sleep(100);
    }
}

Verified: a thread sleeping inside a lock makes everyone else wait for it.

sleeper: got the lock, now sleeping
prober2: waited 322 ms for the lock

sleep does not release the lock. The producer still cannot get in, and now the consumer is asleep as well.

Even if you fix the lock, sleeping is a guess. Sleep 100 ms and you are up to 100 ms late every time. Sleep 1 ms and you are back to burning a core.

What is needed is a thread that steps aside completely, releases the lock while it waits, and is told when something changes.

03

The idea

wait(), notify() and notifyAll() are on Object, alongside the lock from Section 8.3. Every object has all three.

// consumer
synchronized (lock) {
    while (queue.isEmpty()) {
        lock.wait();               // release the lock and step aside
    }
    Job job = queue.removeFirst();
}

// producer
synchronized (lock) {
    queue.add(job);
    lock.notifyAll();              // tell anyone waiting
}

wait() releases the lock. That is the whole difference from sleep, and it is what makes this work at all:

holder: got the lock, now waiting
prober: got in WHILE the holder was waiting
holder: woke up

The waiting thread is in WAITING from Section 14.2, holding nothing, using no processor.

What wait() actually does

  1. Release the lockCompletely, however many times this thread had taken it. The reentrant count is remembered.
  2. Go into the wait setA list on the monitor of threads waiting on this object. The thread now uses nothing at all.
  3. Somebody calls notifyThe thread is moved from waiting into the queue of threads competing for the lock. It is not running yet.
  4. Reacquire the lockIt must win the lock again before it can continue, and other threads may get in first.
  5. Return from wait()With the lock held and the reentrant count restored, on the line after the wait call.

Step four is the one people miss. notify does not hand over the lock. It makes a thread eligible, and there is a gap before it actually runs.

Which is why wait must be in a while loop.

while (queue.isEmpty()) {          // correct
    lock.wait();
}

if (queue.isEmpty()) {             // broken
    lock.wait();
}

Waking up is not a promise that anything happened. Three ways to wake with nothing to do:

Why you wokeWhat is true when you get the lock back
Spurious wakeupno reason. The specification allows itnothing changed at all
notifyAll woke everyoneten threads, one new itemnine of them have nothing to take
Somebody got there firstyou were told, then had to requeue for the lockanother consumer already took it

An if checks once and never again, so the thread continues believing something that has become false. A while re-checks after every wake, which is correct in all three cases.

notifyAll unless you can prove notify is safe.

notify wakes one thread and you do not get to choose which. That is fine when every waiter wants the same thing and any of them will do.

It breaks as soon as there are two kinds of waiter. In a bounded queue, producers wait for space and consumers wait for items, and both wait on the same lock. notify after adding an item can wake a producer, which re-checks, finds no space, and waits again. The consumer was never told and nothing moves.

notifyAll wakes everyone. The ones with nothing to do loop and wait again, and the right one gets through.

04

Under the hood

Going deeper

Two queues per monitor. A wait set for threads that called wait(), and an entry set for threads waiting to acquire the lock.

wait() moves a thread into the wait set and releases the lock. notify moves one from the wait set to the entry set. notifyAll moves all of them. Nothing runs until it wins the lock from the entry set.

That is the whole mechanism, and it explains the gap: being notified only gets you into the queue.

notifyAll on many threads is a thundering herd. All of them wake, all of them compete for one lock, one wins and the rest go straight back to waiting. With two threads it is nothing. With two hundred it is real work.

That is the argument for Condition in Section 14.7, which lets you keep producers and consumers in separate wait sets and wake only the right group.

A missed notification stops everything. notify on an object nobody is waiting on does nothing. It is not remembered:

// consumer                     // producer
if (queue.isEmpty())            queue.add(job);
                                lock.notifyAll();   // nobody waiting yet
    lock.wait();                // now it waits, forever

The producer notified before the consumer waited, so the signal went nowhere. The consumer then waits for a notification that has already happened.

Doing the check and the wait inside one synchronized block is what prevents this. The producer cannot get in between them, because it needs the same lock to add anything.

wait() can be interrupted, and it throws InterruptedException like sleep. Everything from Section 14.3 applies, including that catching it clears the flag.

wait(timeout) gives up eventually. Useful, and it does not tell you which happened: you get back the same way whether you were notified or the time ran out. If you need to know, look at the condition, which you are doing anyway inside the while.

05

What it costs

This is the lowest level tool in the phase and it shows. Producer and consumer have to agree on the lock object, the condition, who notifies and when, and every one of those is a convention with nothing checking it.

The failures are the worst kind. Use if instead of while and the bug appears rarely and under load. Notify before anyone waits and the program stops with no error at all. Both look correct in review.

One wait set for everything is a design limit rather than a mistake. Producers and consumers on the same lock cannot be woken separately, so notifyAll is the only safe choice. Every waiter wakes for every event, whether it concerns them or not.

There is also nothing to look at when it goes wrong. A thread in WAITING tells you it is waiting and not what for, or who was supposed to tell it.

The honest summary is that almost nobody should write this. BlockingQueue does the producer and consumer case correctly and takes about a line, and Section 14.7 has better tools for the rest. What you have gained here is the ability to read the code inside those classes, because this is what they are built from.

06

Check yourself

Answer from memory before opening the explanation. Then compare your rule with the answer and revisit the example if they differ.

  1. `wait()` and `sleep()` both pause a thread. What is the difference that matters?

    Show the answer

    wait() releases the lock. sleep() keeps it.

    Verified. A thread holding a lock and sitting in wait() let another thread straight in. A thread holding the same lock and sitting in sleep() made the other thread wait 322 ms for it.

    That is why sleeping inside a lock is usually a bug. Nobody else can get in, including the thread that was going to produce the thing you are waiting for, so the wait can never end.

    The other difference follows from it. wait() is on Object, because it is about a lock. sleep() is static on Thread, because it is about a thread and no lock is involved.

  2. Why must `wait()` always be called inside a `while` loop and never inside an `if`?

    Show the answer

    Because waking up is not a promise that anything happened.

    Three ways to wake with nothing to do. A spurious wakeup, which the specification allows with no reason given. notifyAll waking ten threads when only one item arrived. And another thread taking the item between your wakeup and your reacquiring the lock.

    An if checks once, before waiting, and never again. So the thread carries on believing a condition that may now be false, and takes from an empty queue.

    A while re-checks after every wake. If the condition is still false it waits again, which is correct in all three cases.

  3. When is `notify()` safe, and why is `notifyAll()` the usual advice?

    Show the answer

    notify() wakes one waiting thread and you do not choose which. It is safe only when every waiting thread is waiting for the same condition and any one of them can handle it.

    The moment there are two kinds of waiter, it breaks. Producers waiting for space and consumers waiting for items are on the same lock, so notify can wake a producer when an item arrived. That producer re-checks, sees no space, and waits again. The consumer was never told, and everything stops.

    notifyAll wakes all of them. The ones with nothing to do go back to waiting, because they are in a while loop.

    It costs more and it cannot deadlock this way, so it is the right default.

07

Exercises

Write each solution in VS Code, predict its result, and then run the checks. Reading an example does not replace implementing it.

4 exercises100 pointsabout 95 minutes

The Layers of Logic VS Code extension runs the checks for exercises marked checked. For a manual exercise, run the program and compare its behaviour with the stated requirements and sample output.
A

Sleeping Inside a Lock

Real work·20 min·20 points

ex-14-6-a

Two ways to pause, and one of them takes the room with it.

Time the second thread in both versions. The numbers are not close, and the difference is the single fact that makes producer and consumer code possible at all.

Then think about where the two methods live. wait is on Object and sleep is on Thread, and once you have seen this experiment that placement stops looking arbitrary.

What your program must do

  • Time how long a second thread waits in each version
  • Print the paused thread's state in both cases
  • Explain which one releases the lock and why that matters
  • Say why the two methods live on different classes
Holding.java
public class Holding {
    static final Object lock = new Object();

    public static void main(String[] args) throws Exception {
        // TODO: a thread that takes the lock and then SLEEPS for 400 ms.
        //       A second thread that tries to take the lock. Time how long it waits.
        // TODO: the same with wait(400) instead of sleep. Time it again.
        // TODO: print the first thread's state in both cases while it is paused
        // TODO: say why wait() lives on Object and sleep() on Thread
    }
}
Hint 1
With sleep the second thread waits the full 400 ms. With wait it gets in almost immediately.
Hint 2
The states differ too. A sleeping thread is TIMED_WAITING and still holds the lock. A waiting thread is also TIMED_WAITING for wait(400) and holds nothing.
Hint 3almost the answer
wait is on Object because it is about a lock, and every object has one. sleep is static on Thread because it is about a thread and no lock is involved.
What this is really testing

Whether you can feel the difference between wait and sleep. Both pause a thread and only one of them lets anybody else in.

B

Break It With an if

Real work·25 min·25 points

ex-14-6-b

Write the broken version first, with three consumers and one item.

An if checks once, before waiting. When the thread wakes, the check has already happened, so it walks straight on and takes from an empty queue. That is not a subtle timing issue: with three consumers and one item it happens on almost every run.

The three reasons at the end are worth writing out in your own words. Only one of them is about notifyAll, and the other two would still be true with a perfect notification scheme.

What your program must do

  • Build the if version with three consumers and one item, and record the failure
  • Fix it with while and confirm the failure is gone
  • Say exactly what the extra consumers did after waking
  • Name the three reasons a thread can wake with nothing to do
WhileNotIf.java
import java.util.*;

public class WhileNotIf {
    static final Object lock = new Object();
    static final LinkedList<Integer> queue = new LinkedList<>();

    public static void main(String[] args) throws Exception {
        // TODO: THREE consumers, each waiting for an item, using `if` not `while`.
        //       One producer that adds ONE item and calls notifyAll.
        //       Predict what happens. Then run it.
        // TODO: change the if to a while and run again
        // TODO: with the if version, catch the exception and say exactly what
        //       the third consumer did wrong
        // TODO: name the three reasons a thread can wake with nothing to do
    }
}
Hint 1
notifyAll wakes all three. One takes the item. The other two return from wait, skip the check because the if already ran, and remove from an empty list.
Hint 2
Expect a NoSuchElementException from the LinkedList, or a null, depending on how you take the item.
Hint 3almost the answer
The three reasons are a spurious wakeup, notifyAll waking more threads than there is work for, and another thread taking the item between your wakeup and your reacquiring the lock.
What this is really testing

Whether the while loop around wait is a ritual or a requirement. Swap it for an if and the bug appears under load, which is the worst possible schedule.

C

The Notification That Arrived Too Early

Hard·25 min·30 points

ex-14-6-c

Force the bad order with sleeps, because waiting for it to happen naturally could take all day.

The consumer checks, finds nothing, and then goes to wait. In between, the producer does everything it was going to do. The signal was sent to an empty room, and the consumer arrives immediately afterwards to wait for a signal that has already been and gone.

The fix is one brace, and the reason it works is worth stating out loud. Holding the lock across both the check and the wait makes them one step as far as the producer is concerned, because the producer cannot act without that lock.

What your program must do

  • Force the notify-before-wait order and show the consumer waiting forever
  • Fix it by putting the check and the wait inside one synchronized block
  • Explain why the fix closes the window, using the lock
  • Say whether a timeout on wait would be a fix or a workaround
TooEarly.java
public class TooEarly {
    static final Object lock = new Object();
    static boolean ready = false;

    public static void main(String[] args) throws Exception {
        // TODO: write it WRONG on purpose: the consumer checks the flag outside
        //       the synchronized block, then enters and waits. The producer sets
        //       the flag and notifies in between. Use sleeps to force the order.
        // TODO: show the consumer waiting forever
        // TODO: fix it by putting the check and the wait in ONE synchronized block
        // TODO: say why the fix works, in terms of who holds the lock
    }
}
Hint 1
notify is not remembered. If nobody is waiting when it is called, it does nothing at all, and the later wait has no idea it already happened.
Hint 2
The fix works because the producer needs the same lock to change the flag. While the consumer holds it, checking and waiting are one uninterruptible sequence from the producer's point of view.
Hint 3almost the answer
wait(timeout) means the thread eventually gives up and re-checks, so it recovers. That is a workaround: it turns a hang into a delay, and the window is still there.
What this is really testing

Whether you can see why the check and the wait must be inside one synchronized block. A notification sent before anyone is waiting is not stored, it is gone.

D

Then Throw It All Away

Real work·25 min·25 points

ex-14-6-d

Write it properly by hand, once, and then delete it.

Both while loops, both notifications, the bound at the top end and the empty check at the bottom. Get it working with two producers and two consumers, because one of each will hide most of the mistakes.

Then replace the whole thing with ArrayBlockingQueue and count what you deleted. The point is not that the library version is shorter. It is that you now know exactly what is inside it, which is the only reason this section existed.

The last question is the one that catches people. Producers and consumers wait on the same lock, so waking one thread means possibly waking the wrong kind.

What your program must do

  • Write a bounded buffer by hand with wait and notifyAll
  • Run it with two producers and two consumers
  • Rewrite it with ArrayBlockingQueue and compare the line counts
  • Say why notify cannot safely replace notifyAll here
ThrowAway.java
import java.util.*;
import java.util.concurrent.*;

public class ThrowAway {
    // TODO: write a bounded buffer BY HAND with wait and notifyAll:
    //   put(item) waits while it is full
    //   take() waits while it is empty
    //   both in while loops, both notifying afterwards

    public static void main(String[] args) throws Exception {
        // TODO: two producers and two consumers against your version. Run it a while.
        // TODO: rewrite the whole thing with ArrayBlockingQueue. Count the lines.
        // TODO: say which parts of your hand written version were the risky ones
        // TODO: why can notify() not be used safely in your version?
    }
}
Hint 1
Two conditions have to be checked in while loops: producers wait while it is full, consumers wait while it is empty. Both notify after changing the buffer.
Hint 2
Use one lock object for everything, and call notifyAll after every change. With a single monitor there is no way to wake only one group.
Hint 3almost the answer
ArrayBlockingQueue replaces the whole class. put and take already block and already handle the bound, and the two Conditions inside it are what Section 14.7 gives you.
What this is really testing

Whether you can recognise what you just built. A bounded producer and consumer queue is about forty lines of careful wait and notify, and one line of BlockingQueue.

08

After the credits

Everything so far has been synchronized, and by now its limits are a list rather than a complaint.

You cannot try for a lock and give up. You cannot wait with a timeout. You cannot be interrupted while waiting for one. You cannot let many readers in while keeping writers out. And you get one wait set, so notifyAll wakes everybody for everything.

Section 14.7 is the package that answers all of it. ReentrantLock with tryLock and a timeout. ReadWriteLock, which lets any number of readers in at once and is a large win on data that is read far more than it is written. Condition, which gives producers and consumers separate wait sets so you can wake only the group that cares. Semaphore, for letting exactly five threads through at a time.

The trade is that synchronized releases its lock for you and every one of these has to be released by hand, in a finally, every time.

Threads you opened in this section

wait and notify will return in Phase XIV. Concurrency