The Backup Generator

Opening Scene

A broadcast facility that genuinely can’t afford to go dark keeps a backup generator on standby — not because the main power is expected to fail often, but because when it does fail, the show still needs to go on. Genuinely critical LLM-dependent systems deserve this same standard of redundancy planning, beyond just handling individual call failures gracefully.

In Plain English

Redundancy and failover planning for LLM systems means maintaining backup options at a systemic level: a secondary model provider ready to take over if the primary one has an extended outage, or a degraded-but-functional fallback mode that keeps a critical feature working, even imperfectly, rather than fully unavailable. This goes beyond the per-call graceful handling covered in Article 8, planning instead for extended, systemic failures rather than individual transient ones.

The Old Way

Before systemic redundancy planning was standard practice for critical LLM-dependent systems, many teams relied entirely on a single provider without a real backup plan:

Many LLM-dependent systems relied entirely on a single model provider, without a genuine, tested plan for an extended outage of that provider.
Per-call retry logic was sometimes treated as sufficient protection, without additional systemic redundancy for outages that outlasted a few retries.
There wasn’t yet a well-established practice of testing failover to a secondary provider before it was actually needed during a real incident.

Systemic redundancy planning emerged specifically once teams experienced extended provider outages that per-call retry logic alone genuinely couldn’t handle.

What’s Changing (and Why AI Is the Reason)

Genuinely critical LLM-dependent systems increasingly maintain a tested secondary provider or degraded fallback mode, not just per-call retry logic.
This connects directly to the incident response and disaster recovery drills covered in Article 16, where failover paths get tested deliberately before they’re actually needed.
Multi-model routing architectures, covered in Article 12, increasingly serve double duty as both a capability strategy and a redundancy strategy.

The Metaphor, Fully Extended

The Broadcast	Redundancy and Failover Concept
A backup generator kept on standby for extended outages	A secondary model provider maintained for extended outages
Not needed often, but essential when the main power actually fails	Not invoked often, but essential when the primary provider has real problems
A degraded emergency broadcast mode, imperfect but functional	A degraded fallback mode that keeps a feature working, even imperfectly
Testing the generator before an actual emergency, not during one	Testing failover before an actual incident, not discovering it fails during one

For Beginners: What to Actually Do

Practice designing a simple fallback mode for a critical LLM-dependent feature, even a degraded one, rather than assuming it will always be fully available.
Learn the difference between per-call retry logic, covered in Article 8, and systemic redundancy planning for extended outages.
Get comfortable with the idea that redundancy is worth planning for even though it’s rarely actually invoked.

For Practitioners and Leaders: The Deeper Layer

Require a tested secondary provider or degraded fallback mode for any genuinely critical LLM-dependent system.
Test failover paths deliberately and regularly, connecting directly to the incident response drills covered in Article 16.
Consider multi-model routing, covered in Article 12, as a strategy that can serve both capability and redundancy purposes simultaneously.

Quick Recap

Systemic redundancy planning maintains backup options at a broader level than individual call retries.
This includes a tested secondary model provider or a degraded-but-functional fallback mode.
Redundancy planning addresses extended, systemic failures, not just transient, individual call failures.
Failover paths need to be tested deliberately before an actual incident, not discovered to fail during one.

Where This Fits in the Series

Article 9 covered systemic redundancy planning. Article 10 turns to a quieter, slower-moving risk: catching model drift before viewers ever notice.