All topics
7 / 101
multi threading
intermediate

ExecutorService & Thread Pools

Run tasks on a managed thread pool instead of spawning raw threads — bounded resources, reuse, and proper shutdown.

A thread pool is a fixed set of threads with a queue of work in front of them. You hand it tasks; whichever thread is free picks up the next one.

A copy shop with 4 photocopiers and a queue of jobs. Customers drop off jobs, get a claim ticket (Future), and pick up the result when done. Closing time: stop accepting new jobs, finish what's in flight.

Key Concepts

1
ExecutorService is the interface you use to submit that work.
ExecutorService
2
Why not just create a thread per task? Two reasons. Creating a thread is expensive, and each one reserves around a megabyte of memory for its stack. More importantly, nothing stops you: a burst of traffic creates a burst of threads, and the machine falls over.
3
A pool puts a ceiling on both.
4
The three settings, and the order that surprises everyone. You choose a core size, a maximum size, and a queue.
5
Most people assume the pool grows to its maximum under load. It does not. The real order is:
6
  1. fewer threads than the core size   ->  create a thread
  2. queue is not full                  ->  PUT THE TASK IN THE QUEUE
  3. queue is full, room to grow        ->  create a thread
  4. queue is full, at maximum          ->  reject the task
7
Step 2 is the surprise. The pool only grows past the core size when the queue is full.
8
So if your queue is unlimited, step 3 never happens. Your maximum size is a setting that does nothing, and tasks pile up in memory until the process runs out.
9
Which is exactly what the convenient factory methods give you.
  Executors.newFixedThreadPool(n)      n threads, UNLIMITED queue
  Executors.newSingleThreadExecutor()  1 thread,  UNLIMITED queue
  Executors.newCachedThreadPool()      UNLIMITED threads
10
The first two turn overload into an out-of-memory crash. The third turns a traffic spike into thousands of threads. Build the pool yourself with a bounded queue instead.
11
Decide what happens when the queue is full. This is a real choice with four options:
12
  AbortPolicy          throw an exception (the default)
  CallerRunsPolicy     run the task on the thread that submitted it
  DiscardPolicy        drop the new task silently
  DiscardOldestPolicy  drop the oldest queued task
13
CallerRunsPolicy is often the best. The thread that submitted the work has to do it, so it stops submitting for a while. Your producer slows down automatically, which is exactly what you want under overload.
CallerRunsPolicy
14
How many threads? It depends on whether the work uses the processor or waits.
15
For work that computes, use about as many threads as you have processor cores. More threads just switch between each other.
16
For work that waits — a database query, an HTTP call — threads are idle most of the time, so you can have many more. A rough formula: cores x (1 + waiting time / computing time). With 8 cores, and work that waits 90 ms per 10 ms of computing, that is 8 x 10 = 80 threads.
17
Shut it down, or your program will not exit. Pool threads keep the JVM alive after main finishes. shutdown() stops accepting new work and lets the queue drain. shutdownNow() interrupts what is running. Neither waits — awaitTermination does.
mainshutdown()shutdownNow()awaitTermination
18
Name your threads. A thread dump full of pool-1-thread-7 tells you nothing during an incident. Pass a thread factory that names them after the pool's purpose.
pool-1-thread-7
19
What the interviewer is probing.1. "Why use a pool?" Probing: cost and limits. Stalls: "Pools are faster." Moves up: creation cost and about a megabyte of stack each, plus the ceiling that stops a spike creating unlimited threads.
20
2. "Core 2, maximum 10, unlimited queue. When do you get 10 threads?" Probing: the growth order. Stalls: "Under load." Moves up: never — the queue never fills, so the pool never grows.
21
3. "The queue is full. What happens?" Probing: the rejection policy. Stalls: "It waits." *Moves up:* name the four, and why caller-runs gives you natural slowdown.
22
4. "How many threads for work that calls a database?" Probing: the formula. Stalls: "As many as possible." Moves up: cores times one plus the wait-to-work ratio, with a number.
23
5. "The application will not exit." Probing: lifecycle. Stalls: "Call System.exit." Moves up: pool threads keep the JVM alive; shut the executor down and wait for it.
24
  How ThreadPoolExecutor grows (the order surprises people)
25
    1. threads < corePoolSize      -> create a new thread
    2. queue not full              -> QUEUE the task
    3. queue full, threads < max   -> create a new thread
    4. queue full, threads == max  -> rejection policy
26
    => with an unbounded queue, step 3 never happens and
       maximumPoolSize is dead configuration.
27
  What the Executors factories actually give you
28
    newFixedThreadPool(n)     n threads, UNBOUNDED queue
                              -> OOM under sustained overload
    newCachedThreadPool()     0..Integer.MAX_VALUE threads,
                              SynchronousQueue
                              -> thread explosion under a burst
    newSingleThreadExecutor() 1 thread, UNBOUNDED queue
    newVirtualThreadPer
      TaskExecutor()          Java 21+, a new virtual thread per
                              task; the unbounded concern moves to
                              the downstream resource
29
  A production pool, explicitly
30
    new ThreadPoolExecutor(
        8, 32,                       // core, max
        60L, TimeUnit.SECONDS,       // idle keep-alive
        new ArrayBlockingQueue<>(500),          // BOUNDED
        new ThreadFactoryBuilder().setNameFormat("api-%d").build(),
        new ThreadPoolExecutor.CallerRunsPolicy());  // backpressure
31
  Sizing
    CPU-bound     threads ~= cores
    IO-bound      threads ~= cores x (1 + wait/compute)
                  e.g. 8 cores, 90 ms wait / 10 ms compute
                       8 x (1 + 9) = 80 threads
32
  Rejection policies
    AbortPolicy          throws RejectedExecutionException (default)
    CallerRunsPolicy     runs on the caller -> natural backpressure
    DiscardPolicy        silently drops the new task
    DiscardOldestPolicy  drops the oldest queued task
Use cases:Thread PoolAsyncServer