Skip to content

This is general knowledge or an external-engine reference, not an Apollo architecture or implementation claim. See Apollo's architecture status for target terminology and verified implementation boundaries.

Q70: How to Design a High-performance Message Queue? ​

Core Conclusion ​

The core of a high-performance message queue is not the two words "lock-free", but making the relationship between producing, queuing, consuming, and back pressure clear enough.

What really needs to be designed first is:

  • Single producer or multi-producer
  • Single consumer or multi-consumer
  • Whether there are priorities
  • What to do when the queue is full

If these boundaries are unclear, even a highly performant queue implementation will easily get out of control in a real system.

I. First Clarify What Problem the Message Queue Solves ​

Common uses include:

  • Decoupling between threads
  • Smoothing production and consumption rates
  • Asynchronizing expensive tasks
  • Batch processing messages

So a message queue is essentially not just a data structure, but also a flow regulator.

II. Queue Design First Looks at the Concurrency Model ​

The most common models include:

  • SPSC: single producer, single consumer
  • MPSC: multi-producer, single consumer
  • MPMC: multi-producer, multi-consumer

This step is very critical, because different models determine:

  • How much synchronization cost is needed
  • Whether lock-free is worth it
  • How high the implementation complexity gets

Don't default to the most complex model from the start.

III. Bounded or Unbounded Is a Key Trade-off ​

1. Bounded Queue ​

Advantages:

  • Controllable memory
  • Easier to express back pressure

Disadvantages:

  • When full, you must decide whether to drop, block, or degrade

2. Unbounded Queue ​

Advantages:

  • Producers are not blocked immediately

Disadvantages:

  • Easily turns the problem into memory bloat and tail latency

In online services, most critical queues ultimately lean bounded, because "queueing infinitely" usually does not really solve the problem.

IV. Priority and Fairness Must Be Thought Through in Advance ​

Some messages are naturally more important than others, for example:

  • Critical control messages
  • High-value business events
  • High-frequency low-value state updates

If everything is mixed in one FIFO queue, peak hours easily let low-value messages clog the critical path.

But priority also has costs:

  • More complex implementation
  • Easy to starve
  • Harder to debug

V. Back Pressure Strategy Matters More Than Throughput Numbers ​

When consumers cannot keep up, you must be explicit about:

  • Blocking producers
  • Dropping low-priority messages
  • Merging stale messages
  • Degraded handling

This is one of the most important real problems of a queue system.

Many systems don't die from "enqueueing too slowly", but from "too much queued and no one can process it all".

VI. Batch Processing Is Usually Very Valuable ​

For many message queues, batch dequeue and batch processing significantly reduce:

  • Lock overhead
  • Atomic operation cost
  • Context switching

But the batch window cannot be too large, otherwise it increases latency and jitter.

VII. Lock-free Queues Are Not the Default Answer ​

Lock-free structures suit scenarios with clear hotspots and stable concurrency models, but they also bring:

  • Rising code complexity
  • Difficult debugging
  • Memory ordering problems

If the concurrency model is simple, a locked but low-contention implementation may actually be more stable.

VIII. A More Pragmatic Design in Engineering ​

Common practice is:

  • First choose the simplest viable structure according to the concurrency model
  • Keep queues bounded where possible
  • Be explicit about the back pressure strategy when full
  • Support priority or separate queues on critical paths
  • Use batch consumption to reduce scheduling cost

This fits real production needs better than purely chasing theoretical throughput.

IX. Common Misconceptions ​

1. Lock-free Queues Are Definitely Fastest ​

Not necessarily. It depends on the concurrency model, contention pattern, and implementation quality.

2. Unbounded Queues Are Safer ​

More often than not it just turns the problem from blocking into a memory and latency disaster.

3. FIFO Is Enough ​

In peak scenarios, critical control messages and low-value messages usually should not have exactly equal weight.

References ​

  • Materials on concurrent queues, back pressure, and async task scheduling practices

Released under the MIT License