When a language model doesn't do what you need, two techniques get reached for: retrieval-augmented generation and fine-tuning. They're often framed as competitors, but they solve different problems, and choosing well starts with knowing which problem you actually have.
What each one does
Retrieval-augmented generation (RAG) supplies knowledge at query time. It searches a knowledge base for the passages relevant to a question and inserts them into the prompt, so the model answers from material it was never trained on. The model's weights don't change; the context does.
Fine-tuning changes the model itself. It continues training on your examples so that new behavior — a format, a tone, a task pattern — is baked into the weights. Nothing needs to be retrieved at query time because the behavior is now part of the model.
Knowledge vs behavior
The cleanest way to choose is to ask whether you need the model to know something or to behave a certain way. RAG is for knowledge: facts, documents, and data that change and must stay current. Fine-tuning is for behavior: consistent formatting, a specialized style, or a task the base model does clumsily.
This is why they're complements more often than alternatives. A support assistant might be fine-tuned to follow a house style and structure, while using RAG to pull the current policy documents it answers from. Behavior is trained in; knowledge is retrieved.
Cost, freshness, and effort
RAG keeps knowledge current by design: update the documents and the answers update, no retraining required. Its cost is architectural — you maintain a retrieval pipeline, and every query carries the retrieved passages as extra input tokens, which you can model with the LLM API cost calculator.
Fine-tuning front-loads effort into preparing training data and running the training, and its knowledge is frozen at training time — new facts require retraining. What it buys is efficiency at inference: behavior is built in, so prompts can be shorter and no retrieval step is needed.
How to choose
Reach for RAG when the problem is knowledge that is large, private, or changing, and when you need answers to cite sources. Reach for fine-tuning when the problem is consistent behavior the base model struggles with, and when you have quality examples of the behavior you want. When you need both current knowledge and specialized behavior, use them together.
The takeaway
RAG and fine-tuning are not the same tool pointed at the same job. RAG adds knowledge at query time and keeps it fresh; fine-tuning changes behavior and bakes it in. Ask whether you need the model to know more or to act differently — and don't be surprised when the answer is both.