Thread Scheduling

In our introduction to how threads work, we introduced the thread scheduler, part of the OS (usually) that is responsible for sharing the available CPUs out between the various threads. How exactly the scheduler works depends on the individual platform, but various modern operating systems (notably Windows and Linux) use largely similar techniques that we'll describe here. We'll also mention some key varitions between the platforms.

Note that we'll continue to talk about a single thread scheduler. On multiprocessor systems, there is generally some kind of scheduler per processor, which then need to be coordinated in some way. (On some systems, switching on different processors is staggered to avoid contention on shared scheduling tables.) Unless otherwise specified, we'll use the term thread scheduler to refer to this overall system of coordinated per-CPU schedulers.

States

Depending on the system, there are various states that a thread can be in. Probably the two most interesting are:

• runnable, which essentially means "ready to consume CPU"; being runnable is generally the minimum requirement for a thread to actually be scheduled on to a CPU;
• waiting, meaning that the thread currently cannot continue as it is waiting for a resource such as a lock or I/O, for memory to be paged in, for a signal from another thread, or simply for a period of time to elapse (sleep).

Other states include terminated, which means the thread's code has finished running but not all of the thread's resources have been cleared up, and a new state, in which the thread has been created, but not all resources necessary for it to be runnable have been created. Internally, the OS may distinguish between various different types of wait states² (for example "waiting for a signal" vs "waiting for the stack to be paged in"), but this level of granularity is generally not available or so important to Java programs. (On the other hand, Java generally exposes to the programmer things the JVM can reasonly know about, for example, if a thread is waiting to acquire the lock on a Java object— roughly speaking, "entering a synchronized block".)

Next: quanta and switching

On the next page, we continue our description of thread scheduling with a look at thread quanta and switching, and discuss typical thread scheduling algorithms.

---

1. Note that in this description, we're going to assume that the OS performs specifically thread scheduling: that is, the unit that the scheduler "juggles" is the individual user thread rather than a process (or a kernel-level thread that sits halfway between a process and a user thread). Solaris 9 and Linux 2.6 onwards use this threading model, but earlier versions don't. To my knowledge, all versions of Windows use thread-level scheduling.
2. At a very low level, this can lead to some complex interactions such as a thread that was scheduled on to the processor getting preempted "at the last minute" while waiting to have its stack paged in. In other words, the notion that a thread goes from, say, "waiting" to "runnable" in an atomic step is actually a simplification. But it's usually a sufficient simplification from the point of view of the programmer.

---