When to fine-tune: form, not facts

Here is the single most important idea in the whole course, and the one most often gotten wrong: fine-tuning is for behavior and form; RAG is for knowledge and facts. Internalize this, and you'll make good customization decisions. Miss it, and you'll waste effort fine-tuning to add facts — which fine-tuning does badly and can even make worse.

What fine-tuning is genuinely good for (behavior/form):

• Consistent style, tone, or voice — making the model reliably sound a certain way.
• Strict output format — reliably producing a specific structure (a particular JSON shape, a fixed template).
• Following a narrow task reliably — teaching a specific skill or behavior the base model does inconsistently.
• Correcting instruction-following failures — when the model can do the task but doesn't do it reliably from prompting alone.
• Latency and cost — fine-tuning a smaller model to match a bigger general model's quality on a narrow task, so you can serve it cheaper and faster.

In all of these, you're shaping how the model behaves, not what facts it knows.

What fine-tuning is the WRONG tool for — adding knowledge. Facts that the model needs but doesn't have — especially fresh or changing facts (a document written last week, today's inventory, your proprietary data) — belong in RAG, which injects the relevant information at inference time. Fine-tuning to add knowledge is problematic for two reasons:

• It doesn't reliably teach new facts. Research (Gekhman et al., 2024) found that LLMs learn fine-tuning examples containing new knowledge slowly.
• **It can increase hallucination.** The same research found that as models do learn new knowledge through fine-tuning, they become more prone to hallucinate with respect to their pre-existing knowledge. So forcing facts in via fine-tuning can actively make the model less reliable.

This is why "fine-tune the model on our company docs so it knows them" is usually a mistake — RAG does that job better, cheaper, and more updatably.

A supporting idea — the "superficial alignment hypothesis." Research (LIMA, 2023) suggests that almost all of a model's knowledge is acquired during pretraining, and that instruction tuning / alignment mainly teaches format and style. This reinforces the framing: fine-tuning largely surfaces and shapes what's already there; it's not an efficient way to pour in new knowledge.

The decision rule:

• **Is your need about how the model behaves** (style, format, reliability on a task, latency/cost)? → Fine-tuning may help.
• **Is your need about what the model knows** (facts, especially changing ones)? → Use RAG, not fine-tuning.
• Both? → Combine them (Module 1's spectrum): RAG for knowledge, fine-tuning for behavior. They're complementary.

The mindset: the most important framing in model customization is fine-tuning is for form, RAG is for facts. Fine-tuning genuinely helps with style, output format, reliable task-following, and serving a cheaper small model on a narrow task — all matters of behavior. It's the wrong tool for adding knowledge: research shows it teaches new facts poorly and can increase hallucination, so use RAG for knowledge instead. When in doubt, ask "is this about how the model behaves or what it knows?" — behavior points to fine-tuning, knowledge points to RAG, and often you combine both. Getting this distinction right is what separates effective customization from wasted effort.