A Message in a Bottle, One at a Time

Opening Scene

A river looks, from the bank, like one continuous, undifferentiated flow. But it isn’t — it’s countless individual drops of water, each one technically traceable, each one part of the whole only because so many are moving together at once. A river guide who thinks of the river as one big shapeless mass misses something important: individual pieces of debris, individual eddies, individual moments where the water does something specific and identifiable.

Streaming data works the same way. What looks like a continuous flow is actually a sequence of discrete, individual events — and understanding that unit is the foundation for everything else in this series.

In Plain English

An event is a single, discrete record of something that happened — a customer clicked a button, a sensor took a reading, an order was placed — timestamped and self-contained. A stream is a continuous, ordered sequence of these events. Streaming systems don’t process “the data” as one blob the way a batch job might; they process one event at a time, as each one arrives, which is exactly what makes near-immediate reaction possible.

The Old Way

In a batch world, the natural unit of work was the batch itself — a whole file, a whole table’s worth of new rows, processed together as a single job. Individual records mattered, but the system’s fundamental unit of processing was the collection, not the individual item — you waited for a meaningful batch to accumulate before doing anything with it.

Adapting that batch mindset to streaming often produced awkward, underperforming systems: micro-batches so small they approximated event-by-event processing but with all the overhead of the batch model still attached — the equivalent of trying to understand a river by scooping out tiny buckets of water at a time, rather than actually watching individual drops move.

What’s Changing (and Why AI Is the Reason)

True event-at-a-time processing is now genuinely practical at scale. Modern streaming platforms are built from the ground up around the individual event as the unit of processing, rather than approximating it with tiny batches — a real architectural maturity that’s made this the practical default for serious streaming work.
AI is well-suited to reasoning about individual events in context. An AI system evaluating a single event — is this transaction suspicious, does this sensor reading indicate a problem — benefits from seeing that event with its full individual context (timestamp, source, related recent events) rather than as an anonymous row within a larger batch.
Event schemas are getting AI-assisted design help, similar to other schema design covered elsewhere in this site. Defining what a well-formed event actually contains — required fields, structure, self-descriptiveness — benefits from the same AI-assisted schema drafting this site’s data-warehousing-lakehouses topic described for warehouse tables, applied to the individual event as the unit of design instead.

The Metaphor, Fully Extended

River Element	Event Concept
A single traceable drop of water	An individual event
The continuous flow made up of countless such drops	A stream: an ordered sequence of events
Scooping out small buckets to approximate watching individual drops	Micro-batching as an imperfect approximation of true event processing
A guide who tracks each significant piece of debris as it passes	A streaming system processing and reacting to each event individually
A drop’s specific characteristics — size, speed, position — recorded as it passes	An event’s structured fields — timestamp, source, payload

For Beginners: What to Actually Do

Practice identifying, for a system you’re familiar with, what its “event” actually is — the smallest meaningful unit of something happening. That exercise clarifies a lot about how the system should be architected.
Get hands-on with a simple event schema: sketch what fields a “customer clicked a button” event should contain to be genuinely self-descriptive and useful on its own, without needing extra context looked up elsewhere.
Understand micro-batching as a real, sometimes reasonable compromise, not an inherently wrong approach — but recognize its limits compared to true event-at-a-time processing when evaluating a system’s actual real-time capability.
Notice how thinking in individual events, rather than batches, changes how you’d design error handling — one bad event shouldn’t necessarily block the whole stream, the way one bad row in a batch job might halt the entire job.

For Practitioners and Leaders: The Deeper Layer

Evaluate any “streaming” system you’re considering by asking directly whether it processes true individual events or relies on micro-batching under the hood — the distinction matters for both latency and architectural complexity.
Invest real design attention in event schema quality — a well-structured, self-descriptive event pays off throughout the rest of a streaming pipeline, similar to how good warehouse schema design pays off across every downstream query.
AI systems reasoning about individual events benefit from rich, well-structured event context — treat event schema design as directly relevant to AI use case quality, not just an implementation detail for engineers.
Recognize that individual-event error handling requires different thinking than batch error handling — a single malformed event shouldn’t be allowed to halt an entire stream, and your architecture should reflect that from the start.

Quick Recap

An event is a single, discrete, timestamped record of something that happened; a stream is a continuous, ordered sequence of events.
Batch thinking’s natural unit was the collection; streaming’s natural unit is the individual event, and approximating this with micro-batches is a real but imperfect compromise.
Modern streaming platforms process true individual events at scale, and AI systems benefit from reasoning about events with their full individual context.
Good event schema design pays off throughout a streaming pipeline and directly affects the quality of AI systems reasoning over that data.

Where This Fits in the Series

Article 2 covered watching continuously instead of checking periodically. This article covered the actual unit flowing past: the individual event. Article 4 looks at where those events actually go while they’re waiting to be picked up.