Blending Water From More Than One Source

Opening Scene

A water utility sometimes blends supply from multiple sources, a reservoir and a treatment facility, for instance, combining them into one unified, enriched supply before delivery, rather than keeping each source entirely separate and delivering them individually. Stream joins and enrichment in a streaming pipeline perform this exact same kind of deliberate, valuable combination.

In Plain English

Stream joins combine events from two or more streams based on a shared key, producing a single, combined event containing information from both sources. Enrichment adds additional context to a streaming event by looking up related data from a reference source, like adding a customer’s full profile details to a bare transaction event containing only a customer ID. Both techniques let a single, more complete event be available for downstream processing, rather than requiring separate lookups later.

The Old Way

Before stream joins and enrichment were well-established, mature stream processing capabilities, combining data from multiple sources during streaming was often difficult:

Combining data from multiple streaming sources often required collecting and joining the data later, after storage, rather than during the streaming flow itself.
There wasn’t yet a well-established practice of enriching a bare streaming event with additional context in real time, as it flowed through the pipeline.
Downstream consumers sometimes needed to perform their own separate lookups to gather context that could have been attached to the event earlier, during streaming.

Combining and enriching data only after storage, rather than during the streaming flow, is what stream joins and enrichment directly address.

What’s Changing (and Why AI Is the Reason)

Organizations increasingly perform joins and enrichment directly within the streaming pipeline, delivering more complete, immediately usable events to downstream consumers.
This connects directly to the stream processing concepts covered in Article 4, since joins and enrichment are specific, valuable applications of in-motion, continuous data transformation.
As AI models often perform better with richer, more contextual input features, real-time enrichment has become an especially valuable technique specifically for delivering AI systems the additional context they need without requiring separate, added lookup latency.

The Metaphor, Fully Extended

The Water Utility	Cloud-Native Streaming Concept
Blending supply from a reservoir and a treatment facility	Joining events from two or more separate streams
Combining sources into one unified, enriched supply	Combining sources into one enriched, more complete event
Delivering the combined supply, not separate sources	Delivering the combined event, not requiring separate lookups
A deliberate, valuable combination, not an afterthought	A deliberate, valuable transformation, not an afterthought

For Beginners: What to Actually Do

Practice imagining a scenario where combining two separate streams, or enriching one stream with reference data, would genuinely add value.
Learn to distinguish stream joins (combining two streams) from enrichment (adding context from a reference source).
Get comfortable with the idea that performing this combination during streaming avoids requiring separate, later lookups.

For Practitioners and Leaders: The Deeper Layer

Perform joins and enrichment directly within streaming pipelines where doing so delivers genuinely more useful, complete events to consumers.
Connect this practice directly to the broader stream processing concepts covered in Article 4.
Prioritize real-time enrichment specifically for delivering richer, more contextual input features to AI models without added lookup latency.

Quick Recap

Stream joins combine events from multiple streams based on a shared key.
Enrichment adds additional context to an event by looking up related data from a reference source.
Both deliver more complete events during streaming, avoiding the need for separate, later lookups.
AI models benefiting from richer contextual features especially value this real-time enrichment capability.

Where This Fits in the Series

Article 15 covered combining and enriching data during streaming. Article 16 turns to a specific, common application of stream processing: metering usage as it happens.