Graceful degradation: how do I isolate non-critical services under load?

Short answer: graceful degradation isn’t a coding technique, it’s a decide-in-advance discipline. Decide what’s critical and what’s nice-to-have before Black Friday arrives; the rest is just flipping flags.

Short answer

The real problem is this: if you start thinking about which service to turn off at 98% CPU, you’ve already lost. The core path (browse a product, add to cart, check out) must always stay up; the “nice-to-haves” must be sacrificeable. I covered how a single slow dependency drags the whole system down in the circuit breaker answer; the issue here is building that isolation at the product level.

Why

1. A crisis is not a decision-making moment. You decide which component is sacrificeable while calm, not at 98% CPU.

2. A slow nice-to-have drags the critical path down. If the recommendations service waits 3 seconds, checkout is the one paying for that wait.

3. An untested kill-switch is a guess. You have no evidence that a button first pressed during a crisis will work.

What to do

1. Separate critical from nice-to-have and label them. Browse, cart, checkout are critical. Recommended products, similar products, reviews are nice-to-have. Put each behind an independently disable-able feature flag.

2. Put a short timeout + static fallback on every nice-to-have service. If recommendations don’t answer in 200ms, return empty or show a cached list.

3. Disable instantly with a kill-switch. A single flag should turn off all nice-to-have components live, with no deploy.

4. Protect the DB with load shedding. Return 503 early for non-critical endpoints; don’t let those requests reach the DB at all.

5. Isolate the core path with a bulkhead. Give critical and non-critical traffic separate connection pools / workers.

Bottom line: I’d build the flags and fallbacks in advance and rehearse the “Black Friday switches” before the day comes. An untested kill-switch is a guess that it’ll work when the crisis hits. You want to run a predefined scenario, not improvise at 98% CPU.