When the Sights Run Out of Adjustment

Opening Scene

Sometimes an archer’s sights are turned as far as they’ll physically go, and the shot still misses — not because the instruction was unclear, but because the target has moved somewhere the sights simply can’t reach anymore. At that point, no amount of further sight adjustment helps. What’s needed is a genuine change to the archer’s underlying stance and training. Some model behavior problems reach this exact same point with prompting.

In Plain English

Fine-tuning becomes genuinely worth considering when a problem persists despite real, exhausted effort at prompting — when the model consistently lacks a specific capability, style, or domain knowledge that no amount of instruction seems to unlock, or when a task needs to run at high volume with a shorter, cheaper prompt than reliable behavior would otherwise require. These are the honest signals that a problem has outgrown steering and needs an actual change to the model’s underlying weights.

The Old Way

Before these signals were well understood as a deliberate decision point, teams often defaulted to one extreme or the other:

Some teams reached for fine-tuning as a first resort for any stubborn problem, without genuinely testing whether better prompting could have resolved it instead.
Other teams avoided fine-tuning almost entirely, even for problems that had genuinely outgrown what prompting alone could fix, accepting persistently poor performance rather than investing in the deeper change.
There wasn’t yet a well-established, honest checklist for recognizing when a problem had genuinely crossed this threshold.

Recognizing these specific signals as a deliberate decision point — not a vague gut feeling — represents real, accumulated practical wisdom from teams who’ve made this call both well and poorly.

What’s Changing (and Why AI Is the Reason)

Practitioners increasingly recognize specific, checkable signals — persistent capability gaps despite exhausted prompting, or prompt-length costs at scale — as the genuine threshold for considering fine-tuning.
This connects directly to the cost comparison covered in Article 10, since one common trigger for fine-tuning is that a long, complex prompt becomes genuinely expensive to run at high volume.
As fine-tuning tooling has become more accessible, the real question has shifted from “can we fine-tune” to “have we genuinely exhausted prompting first” — the exact discipline covered in Article 4.

The Metaphor, Fully Extended

The Archer	Signals That Prompting Has Run Out
Sights turned as far as they’ll physically go	A prompt genuinely exhausted through real, deliberate iteration
A target that’s moved somewhere the sights can’t reach	A capability gap no amount of instruction seems to close
Needing a real change to stance and training, not another sight tweak	Needing an actual change to the model’s weights, not another prompt tweak
Recognizing this threshold honestly, not defaulting to one extreme	Recognizing the threshold through specific, checkable signals, not a vague feeling

For Beginners: What to Actually Do

Practice building an honest checklist of what “genuinely exhausted prompting” looks like for a specific problem, before assuming it needs fine-tuning.
Learn to distinguish a true capability gap (the model doesn’t know how) from an instruction gap (the model wasn’t told clearly).
Get comfortable estimating a task’s prompt length and volume, to check whether cost is a genuine factor pushing toward fine-tuning.

For Practitioners and Leaders: The Deeper Layer

Establish a specific, checkable threshold for approving fine-tuning projects, tied to genuinely exhausted prompting effort, not just persistent frustration.
Track prompt-length and volume costs explicitly, since this is one of the most concrete, measurable triggers for considering fine-tuning.
Build organizational memory of past fine-tuning decisions — which were justified and which weren’t — to sharpen this threshold over time.

Quick Recap

Fine-tuning becomes worth considering when a problem persists despite genuinely exhausted prompting effort.
Key signals include persistent capability gaps and prompt-length costs that become significant at scale.
These should be specific, checkable signals, not a vague sense that prompting “isn’t working.”
The real decision point has shifted from “can we fine-tune” to “have we genuinely exhausted prompting first.”

Where This Fits in the Series

Article 5 covered recognizing when prompting has genuinely run out. Article 6 turns to what fine-tuning actually does to a model once that threshold is crossed.