The Wait Before the First Car Arrives

Opening Scene

Even the fastest dispatch system involves some genuine delay between requesting a ride and the car actually pulling up — a gap that’s usually brief, but occasionally noticeable, especially at an unusual hour when fewer drivers are already nearby. Serverless infrastructure has an equivalent gap, called a cold start: the delay between a workload being triggered from zero and it actually being ready to execute.

In Plain English

A cold start is the delay incurred when a serverless function or service must initialize entirely from scratch — allocating resources, loading code and dependencies, establishing connections — before it can begin actually processing a request. Cold starts are usually measured in milliseconds to a few seconds, but for genuinely latency-sensitive workloads, or workloads triggered rarely enough that they’re almost always cold, this delay can matter meaningfully.

The Old Way

Before cold starts were a well-understood, actively managed characteristic of serverless architecture, this tradeoff was less deliberately addressed:

Early serverless offerings sometimes had noticeably longer cold start delays, before providers invested significantly in optimizing initialization speed.
There wasn’t yet a well-established set of practices for designing serverless workloads specifically to minimize or tolerate cold start delay.
Latency-sensitive workloads were sometimes deployed to serverless infrastructure without genuinely accounting for cold start impact on user experience.

Noticeable, unaddressed cold start delay, without deliberate design practices to minimize or tolerate it, is what mature serverless cold-start management directly addresses.

What’s Changing (and Why AI Is the Reason)

Cloud providers and practitioners increasingly use techniques like provisioned concurrency (keeping a minimum number of instances warm) specifically to eliminate cold starts for latency-critical workloads.
This connects directly to the automatic provisioning speed covered in Article 4, since cold start duration is fundamentally a function of how fast that underlying provisioning actually is.
As AI inference increasingly runs on serverless infrastructure, cold start delay has become a particularly significant consideration, since loading a large model into memory can meaningfully extend the initialization delay beyond typical serverless functions.

The Metaphor, Fully Extended

The Taxi Rider	Serverless Data Architecture Concept
A genuine delay between requesting a ride and the car arriving	A genuine delay between triggering a workload and it being ready
Usually brief, but occasionally noticeable at an unusual hour	Usually brief, but occasionally noticeable for rarely-triggered workloads
Keeping some drivers already nearby to shorten the wait	Keeping some instances warm through provisioned concurrency to shorten the wait
A tradeoff worth designing around for time-sensitive trips	A tradeoff worth designing around for latency-sensitive workloads

For Beginners: What to Actually Do

Practice measuring the actual cold start delay for a serverless function you have access to, comparing a cold invocation to a warm one.
Learn the basic factors that influence cold start duration: runtime language, package size, and dependency loading.
Get comfortable with the idea that cold starts are a genuine, quantifiable tradeoff, not a reason to avoid serverless architecture entirely.

For Practitioners and Leaders: The Deeper Layer

Use provisioned concurrency or equivalent warming strategies specifically for latency-critical serverless workloads.
Measure and monitor cold start frequency and duration as a standard part of serverless workload observability.
Account explicitly for model loading time when deploying AI inference on serverless infrastructure, since this can meaningfully extend typical cold start duration.

Quick Recap

A cold start is the initialization delay when a serverless workload spins up entirely from zero.
This delay is usually brief but can matter meaningfully for latency-sensitive or rarely-triggered workloads.
Provisioned concurrency and warming strategies can eliminate cold starts for latency-critical workloads.
AI inference, with its often significant model loading time, faces a particularly notable cold start consideration.

Where This Fits in the Series

Article 5 covered cold starts as serverless architecture’s most visible tradeoff. Article 6 turns to a different kind of variety: different cars for genuinely different kinds of trips.