A Full Rebuild vs. a Small Adjustment

Opening Scene

Rebuilding an archer’s entire stance from the ground up is one way to change their shot. But sometimes a much smaller, more targeted adjustment — just the grip, or just the release timing — achieves nearly the same improvement with a fraction of the retraining effort. Fine-tuning has this exact same distinction: fully retraining every one of a model’s parameters, or adjusting only a small, targeted subset of them.

In Plain English

Full fine-tuning updates every parameter in a model, requiring significant compute and memory. Parameter-efficient fine-tuning methods, such as LoRA, instead train a small number of additional parameters while leaving most of the original model frozen, achieving much of full fine-tuning’s benefit at a fraction of the cost. This tradeoff — between the thoroughness of a full rebuild and the efficiency of a targeted adjustment — is a genuinely practical decision most fine-tuning projects now face.

The Old Way

Before parameter-efficient methods matured into a widely available, practical option, fine-tuning generally meant one thing only:

Fine-tuning essentially meant full fine-tuning, updating every parameter in the model, regardless of how small the target adjustment actually needed to be.
This made fine-tuning genuinely expensive in compute and memory, putting it out of reach for many teams and use cases that could have benefited from a smaller, more targeted change.
There wasn’t yet a well-established, efficient middle ground between prompting’s zero-weight-change and full fine-tuning’s every-weight-change.

Parameter-efficient fine-tuning emerged specifically to fill this gap, once it became clear many tasks didn’t genuinely need every parameter updated to see real improvement.

What’s Changing (and Why AI Is the Reason)

Parameter-efficient methods like LoRA have made fine-tuning accessible to significantly more teams and use cases, by training a small additional set of parameters rather than the full model.
This connects directly to the cost comparison covered in Article 10, since parameter-efficient methods meaningfully shift fine-tuning’s cost profile closer to prompting’s.
Choosing between full and parameter-efficient fine-tuning has become its own deliberate decision, based on how deep a behavioral change the task genuinely requires.

The Metaphor, Fully Extended

The Archer	Full vs. Parameter-Efficient Fine-Tuning
Rebuilding the entire stance from the ground up	Full fine-tuning: updating every parameter in the model
A smaller, targeted adjustment to just the grip or release	Parameter-efficient fine-tuning: training a small additional parameter set
Achieving much of the same improvement for a fraction of the effort	Achieving much of full fine-tuning’s benefit at a fraction of the compute cost
Choosing the depth of adjustment based on what the shot actually needs	Choosing the fine-tuning method based on how deep a change the task actually needs

For Beginners: What to Actually Do

Practice explaining, in your own words, the basic difference between full fine-tuning and parameter-efficient methods like LoRA.
Learn to recognize when a task’s needed behavioral change is likely small enough for a parameter-efficient approach to suffice.
Get comfortable with the idea that fine-tuning isn’t a single monolithic technique, but a spectrum of methods with real cost and thoroughness tradeoffs.

For Practitioners and Leaders: The Deeper Layer

Default to parameter-efficient methods for most fine-tuning projects, reserving full fine-tuning for cases genuinely requiring deeper behavioral change.
Track the real compute and cost savings parameter-efficient methods provide, informing the broader cost comparison covered in Article 10.
Build organizational familiarity with both approaches, since the right choice depends on the specific depth of change a task requires.

Quick Recap

Full fine-tuning updates every parameter in a model; parameter-efficient methods train a small additional subset while freezing the rest.
Parameter-efficient methods like LoRA achieve much of full fine-tuning’s benefit at a fraction of the compute cost.
This tradeoff has become a genuinely practical decision most fine-tuning projects now face.
Parameter-efficient methods have made fine-tuning accessible to significantly more teams and use cases.

Where This Fits in the Series

Article 9 covered full versus targeted fine-tuning approaches. Article 10 looks directly at what all of this actually costs, compared to prompting.