Knowing the Performer's Limits

Opening Scene

Even the most extraordinary improv performer has genuine limits — things no amount of practiced instinct will let them do reliably. Precise, exact arithmetic performed purely in their head, under real time pressure, isn’t the same skill as compelling storytelling, and pretending otherwise sets everyone up for disappointment. Knowing exactly what a remarkable performer is and isn’t reliably good at is just as valuable as appreciating what they can do. The same honest accounting matters for large language models.

In Plain English

Despite genuinely impressive capabilities, LLMs have real, well-documented limitations worth naming plainly: hallucination (Article 11) remains a genuine risk for any factual claim; precise, multi-step arithmetic and exact counting are surprisingly unreliable, since a model is predicting plausible text, not literally executing calculations; genuinely novel, out-of-distribution reasoning that goes well beyond patterns seen during training can fail in surprising ways; and a model has no persistent memory between separate conversations unless a system is specifically built to provide one.

The Old Way

Before LLMs, these specific limitations weren’t relevant in the same way, since earlier systems had entirely different, more narrowly scoped strengths and weaknesses:

Calculators were purpose-built for exact arithmetic, a task LLMs approach quite differently and less reliably by default, since a calculator literally computes while an LLM predicts plausible-looking output.
Rule-based systems, covered in Article 3, were narrowly reliable within their specific, explicitly coded domain, but had no broader flexibility to fall back on outside it.
Earlier AI systems generally had more narrowly scoped, better-understood failure modes, since their capabilities were more limited and explicit to begin with.

LLMs’ genuinely broad, flexible capability makes their specific failure modes less intuitively obvious, and correspondingly more important to understand explicitly.

What’s Changing (and Why AI Is the Reason)

Tool use — letting a model call an actual calculator, code interpreter, or search engine rather than relying purely on its own internal prediction — has become a standard, practical mitigation for several of these specific limitations, covered in more depth in this content library’s dedicated AI agents series.
Growing practical experience deploying LLMs has produced a much clearer, more evidence-based understanding of exactly where these limitations show up, replacing early, more speculative assessments with real, accumulated operational knowledge.
Retrieval-augmented generation, previewed in Article 17, and the hallucination-reduction techniques covered in this content library’s dedicated series both directly address some, though not all, of these core limitations.

The Metaphor, Fully Extended

The Improv Scene	LLM Limitations Concept
Precise mental arithmetic under time pressure, a genuinely different skill from storytelling	Precise calculation, a genuinely different skill from fluent text generation
A performer with no memory of a different show from a different night	A model with no persistent memory across separate conversations by default
Genuinely novel situations outside anything the performer has rehearsed	Genuinely novel reasoning outside patterns present in training data
Bringing in a specialist tool rather than relying on the performer alone	Bringing in an external tool, like a calculator, rather than relying on the model alone

For Beginners: What to Actually Do

Build a genuine, working mental list of LLM limitations — hallucination, unreliable precise arithmetic, no default persistent memory — as practical, everyday knowledge.
Practice using tool-augmented approaches (letting a model call a calculator or code interpreter) for tasks that fall into these known weak spots.
Recognize that acknowledging limitations doesn’t diminish an LLM’s genuine, real capabilities — it makes using those capabilities well actually possible.

For Practitioners and Leaders: The Deeper Layer

Build explicit awareness of these limitations into any deployment involving consequential decisions, connecting directly to the risk management themes covered throughout this series.
Invest in tool-use and RAG-based mitigations specifically for the limitations they directly address, rather than expecting a bigger or better model to solve every weakness on its own.
Set realistic organizational and stakeholder expectations grounded in these genuine, well-documented limitations, rather than in hype-driven assumptions.

Quick Recap

LLMs have real, well-documented limitations: unreliable hallucination risk, unreliable precise arithmetic, and no default persistent memory across conversations.
These limitations follow directly from an LLM’s core mechanism as a next-token predictor, not a calculator or a database.
Tool use and retrieval-augmented generation directly mitigate several, though not all, of these specific weaknesses.
Honest awareness of limitations is what actually makes using LLM capabilities well possible.

Where This Fits in the Series

Article 18 covered an honest accounting of genuine limitations. Article 19 zooms out to the broader ecosystem of tools and techniques this entire series has been building toward throughout.