The Swatch Book Code: Encoding Categorical Features

Opening Scene

The shop’s swatch book has hundreds of fabric samples pinned inside, each with a small reference code — not “a medium-weight navy wool herringbone with a subtle sheen,” but simply “B-114.” The code isn’t the fabric; it’s a compact, consistent reference that lets the Tailor, the supplier, and the order form all agree on exactly which cloth is meant, without retyping a paragraph every time.

In Plain English

Categorical encoding turns non-numeric categories — a product type, a country, a browser, a fabric — into a numeric representation a model can actually process, since most models can only work with numbers. Simple approaches assign each category its own number or column; more sophisticated approaches, called embeddings, place categories in a continuous space where similar categories end up near each other, much like grouping swatch codes so related fabrics sit near each other in the book.

The Old Way

Categories were often converted into arbitrary integer codes with no meaningful relationship between the numbers, similar to numbering swatches in whatever order they arrived with no regard for which fabrics were actually similar.
High-cardinality categories, like “product SKU” with tens of thousands of values, produced enormous, sparse, unwieldy encodings that slowed everything down.
New categories that hadn’t been seen during training routinely broke encoding logic in production, the equivalent of a brand-new fabric arriving with no swatch code assigned at all.

What’s Changing (and Why AI Is the Reason)

Learned embedding techniques, powered by the same neural network advances behind modern AI, can now place categories in a compact space based on genuine similarity in how they behave, rather than arbitrary numbering, so two genuinely similar fabrics land near each other in the swatch book automatically.
Pretrained embeddings for common categories — geography, product taxonomies, common text fields — are increasingly available off the shelf, letting teams borrow a well-organized swatch book rather than building one from scratch every time.
Feature stores increasingly handle unseen categories gracefully through fallback or default encodings, so a brand-new fabric arriving mid-season doesn’t break the whole ordering system the way an unhandled category once broke a live model.

The Metaphor, Fully Extended

Tailoring Element	Categorical Encoding Concept
A fabric sample with no assigned reference code	A raw categorical value a model cannot directly use
A swatch book reference number, like “B-114”	A simple integer or one-hot encoding of a category
Swatches arranged so similar fabrics sit near each other	A learned embedding placing similar categories close in vector space
A brand-new fabric arriving with no code assigned yet	An unseen category appearing in production that wasn’t in training data
A shared house swatch book used by every tailor in the shop	A standardized, reusable encoding scheme shared across a feature store

For Beginners: What to Actually Do

Never feed raw category text directly into a model; always encode it first, even if only as a simple integer mapping.
Watch for high-cardinality categories — thousands of distinct values usually call for embeddings rather than one-hot encoding.
Always plan for unseen categories showing up in production, and define a sensible fallback rather than letting the system fail.
Where available, evaluate a pretrained embedding before building a custom one from scratch.

For Practitioners and Leaders: The Deeper Layer

Standardize categorical encoding schemes across teams so the same category is represented consistently everywhere it’s used.
Budget for embedding infrastructure when categorical features are high-cardinality or behaviorally rich, since simple encodings genuinely underperform there.
Require explicit handling of unseen categories as part of any production feature pipeline’s design review.
Treat well-organized categorical encoding as an accelerant for downstream model quality, not a minor preprocessing detail.

Quick Recap

Categorical encoding turns non-numeric categories into numeric representations a model can use, ranging from simple codes to learned embeddings.
Older approaches relied on arbitrary numbering that broke down at high cardinality or on unseen values.
Learned embeddings and pretrained encodings now place similar categories meaningfully close together, much like a well-organized swatch book.
A good encoding scheme is a shared reference every tailor in the shop can trust, not a private numbering system.

Where This Fits in the Series

Articles 5 through 7 covered the pipeline, aggregation windows, and encoding — the core techniques for shaping a single feature well. Article 8 zooms out to where all these finished features actually live once they’re ready to be reused.