Key takeaways

Not every task people call “training” requires changing weights. Document search, prompt instructions, or an API call are often enough.

Knowledge ≠ behavior ≠ tools ≠ external memory. A base model handles language; SFT reshapes style and format; RAG injects current facts; agents act through systems.

LoRA is a cheap adaptation technique, not a separate kind of knowledge. It answers “how do we change weights for less money,” not “what should we teach.”

RAG does not teach a model to “remember PDFs.” It attaches external sources for the duration of an answer. For policies, prices, and document versions, that is usually better than fine-tuning a corpus into the weights.

A mature system is a hybrid. Pretraining / SFT / preference set capability and behavior; RAG holds facts; tools perform actions; evaluation keeps quality honest.

What “training a model” actually means

A common mistake: “if the LLM must know our documents, we have to fine-tune it on those documents.” That idea produces projects like “upload 100,000 PDFs into fine-tuning” — costly, hard to refresh, and nearly opaque when something goes wrong.

The mechanisms are fundamentally different:

1. Pretraining — building a base language model on a huge corpus.
2. Continued pretraining — further training on a specialized domain corpus.
3. SFT (supervised fine-tuning) — teaching the model to follow instructions and formats.
4. Preference optimization (RLHF, DPO, and related methods) — preferring better answers.
5. LoRA / PEFT — a technical way to change a small slice of parameters.
6. RAG — retrieval over an external store without changing weights.
7. Tool use / agents — calling APIs, SQL, files, and business services.

In short: training changes parameters. RAG and tools usually do not change parameters — they change context and what the system can do.

That matches how enterprise architecture already describes the model as an interface to knowledge, not the company’s memory, and how engine choice comes after use cases and the security boundary.

Teams often collapse these layers into one backlog item: “train on our data.” That wording hides three separate decisions — whether you need new capabilities in the weights, whether you need fresher facts at request time, and whether you need the model to take actions. Treating them as one decision is how budgets disappear into fine-tuning jobs that should have been an index plus access control.

Level 1. Pretraining — the base model

Pretraining is the main training run on a massive text collection. Simplified:

texts → tokenization → neural net → next-token prediction → loss → weight update

At each step the model sees a prefix (“In 1863 the Act was published …”) and learns to predict the next token. After billions of such steps, internal representations of language, syntax, concepts, styles, and a large number of facts emerge. That is not a lookup table of “fact → value”: knowledge is distributed across parameters.

Why the stage is expensive: token volume, model size, GPU/TPU clusters, wall time, checkpoint storage, distributed training. Specific prices and sizes age quickly — the principle matters more: building a new base LLM from scratch makes sense for a handful of organizations. Everyone else usually gets more value from taking a ready base and adapting it at a lower level.

Pretraining also shapes what later stages can and cannot fix. A model that never saw enough technical prose in a language will struggle with domain continued pretraining; a model with weak instruction-following after post-training will need more SFT than one that already behaves like an assistant. You inherit the base’s strengths and blind spots.

In short: pretraining gives the model general capability. It is a foundation, not a corporate knowledge base.

What model weights are

Parameters (weights) are numbers inside the network layers. During training they move toward lower prediction error. Intuitively:

W_new ≈ W_old − learning_rate × gradient

When people say “we trained the model,” they almost always mean: we changed those numbers (all of them or a subset). When they say “we wired up RAG,” the weights usually stay the same: only the text the model sees in the request context changes.

The practical takeaway: if information must update daily, storing it in weights is like carving a policy into stone. If you need stable behavior (“always answer with JSON schema X”), changing weights — or strong SFT / instructions — can be appropriate. The knowledge-versus-behavior boundary is covered in more detail in when fine-tuning is actually needed.

Weights also explain why “unlearning” is hard. Removing a bad fact from a corpus and re-indexing is routine for RAG. Removing the same fact from a trained parameter set is closer to surgery: you may need more training, careful evaluation, and you still may not prove the knowledge is gone. That asymmetry alone should push volatile content out of the weights whenever possible.

Level 2. Continued pretraining — domain adaptation

A general-purpose base model may poorly “feel” the language of medicine, law, petrochemicals, a holding company’s internal jargon, or a narrow tech stack. Continued pretraining keeps training on a specialized corpus:

Base LLM → domain corpus → continued pretraining → Domain-adapted LLM

Unlike SFT, the goal here is closer to shifting the model’s internal distribution of domain language and facts, not forcing a rigid “for this question, answer like this.” SFT teaches behavior and format; continued pretraining teaches the model to speak the industry’s language over a large volume of text.

When to look this way: a huge private corpus, rare terminology, weak zero-shot performance on the domain. When not to: “we have 500 policies” — that is a RAG and corpus-prep problem, not another pretraining run.

Continued pretraining also carries operational cost that teams underestimate. You need enough clean domain text to move the distribution, compute to run the job, and a plan for how often the domain-adapted checkpoint is refreshed. If the “domain” is mostly living documents with owners and revision dates, those documents belong in retrieval, not in another multi-week training cycle.

Level 3. SFT — behavior and instructions

After pretraining, a model can continue text, but it does not necessarily follow instructions in a stable way. SFT trains on pairs of “instruction → desired answer”:

Question: Explain how RAG works.
Good answer: …

Typical uses: corporate tone, structured JSON, classification, code generation in a house style, support dialogues, sticking to a conclusion template.

SFT does not replace a current document store. It makes answers more predictable in form and style. A practical example of narrow SFT / QLoRA for classification before retrieval appears in the write-up on fine-tuning a small local model.

Dataset quality dominates SFT outcomes. A few thousand carefully reviewed instruction–answer pairs often beat a noisy dump of chat logs. Label leakage, contradictory gold answers, and examples that encode outdated policy into “desired behavior” all show up later as product bugs that look like model defects.

In short: SFT tells the model “do it this way”; RAG says “here are fresh facts for this answer.”

LoRA and PEFT — cheaper ways to change weights

LoRA and other PEFT methods are often confused with a separate “knowledge level.” They are primarily a technique: instead of updating all parameters, you train small adapter matrices on top of a frozen base.

Base model (frozen)
        +
   LoRA adapter
        ↓
 Specialized model

Pros: fewer trainable parameters and less VRAM, easier storage and switching of adapters, multiple specializations on one base. The wording trap: LoRA does not answer “what to teach” — only “how to change the model more cheaply.” A normal combination is SFT + LoRA, preference + LoRA, and so on.

In production, adapters also become deployment units. You can ship a classifier adapter, a drafting adapter, and a tone adapter without maintaining three full fine-tunes. You still need versioning, evaluation, and a story for how adapters are promoted — the cheaper training method does not remove MLOps, it just shrinks the artifact.

Level 4. Preference optimization — “this answer is better”

After SFT you can teach the model to prefer some answers over others. Classic RLHF: a human (or preference policy) chooses between variants A and B; the model shifts toward B’s behavior. Modern alternatives such as DPO simplify the pipeline, but the idea is the same:

SFT says “do it this way”; preference optimization learns from comparisons that “this answer is better than that one.”

Where it fits: safety, politeness, product policy compliance, lower toxicity, preference for short or structured answers. Where it is a poor substitute: live prices, policy versions, personal data from internal systems — that remains RAG and ACL territory, not preferences.

Preference data is also a governance surface. Whoever labels “better” is encoding product values. Without clear guidelines, labelers optimize for fluency or agreeableness while the business needed strict refusal behavior or terse technical tone. Treat preference sets like product specs: documented, reviewed, and regression-tested when policies change.

Level 5. RAG — knowledge outside the model

RAG (retrieval-augmented generation) is the key layer for applied systems. The model does not need to “remember” corporate documents in its weights. Instead:

question → retrieval → relevant chunks → context → LLM → answer (+ citations)

For information that changes often — instructions, prices, regulations, catalogs, wiki — RAG is usually more sensible than fine-tuning: sources are versioned, permissions are checked before delivery, citations stay auditable. Why enterprises need that boundary is covered in why enterprises need RAG. How to build a production path (parsing, hybrid search, rerank, evaluation) is in production RAG engineering and enterprise RAG architecture.

Important: RAG is not “just a vector DB.” A working chain includes document parsing, chunking, metadata, embeddings, keyword / hybrid search, access filters, reranking, context assembly, citation, and evaluation. If retrieval handed over the wrong fragment, a strong LLM will not reliably “fix the source” — it will confidently err on bad context.

Failure modes cluster early in the pipeline. Broken PDF tables, overlapping chunks that split definitions from exceptions, missing ACL filters, and prompt injection buried in uploaded files all corrupt the answer before generation starts. Investing in retrieval quality usually pays more than swapping the generator model.

In short: fine-tuning changes weights; RAG attaches external memory for the duration of the answer.

Level 6. Tools and agents

The next layer is not only reading knowledge but acting:

LLM ├── RAG ├── SQL ├── REST API ├── files ├── calculations └── business services

A request like “show last month’s sales and explain the drop” may need SQL, a calculation, documents from RAG, and only then prose. That is already LLM + tools + orchestration + permissions + state, not “a chat with a model.” Failures here are more often about rights, idempotency, and observability than about “insufficient training.”

Related topics on the site: MCP and APIs for LLMs, routing and gateways in LLM routing taxonomy.

Agents amplify both capability and blast radius. A tool that can create tickets or mutate records needs the same controls you would put on a service account: scoped credentials, dry-run modes where possible, audit logs, and clear stop conditions. “The model decided” is not an access-control policy.

Comparing the levels

The table deliberately places LoRA next to training methods: it is how, not why. RAG and tools sit apart: they extend the system without rewriting parameters.

Practical scenario: enterprise assistant

Suppose you have tens of thousands of documents, APIs, policies, code, tickets, databases, and wiki. Automatically “training the model on everything” is a bad framing: documents go stale, removing knowledge from weights is hard, citation is weak, ACL breaks, updates are expensive.

A working architecture more often looks like this:

                 ┌─────────┐
                 │   LLM   │
                 └────┬────┘
          ┌───────────┼───────────┐
          ↓           ↓           ↓
         RAG        Tools       SQL/API
          │           │           │
   Knowledge Base  Internal    Databases

• RAG — active policies and wiki with role-based filtering.
• SFT / LoRA — a stable draft format or an intent classifier.
• Tools — create a ticket, run a calculation, read metrics.
• Evaluation — regression on your scenarios, not only a public leaderboard (LLM quality testing).

Walk through one user journey. An engineer asks for the current change-management steps for a production deploy. Retrieval should return the approved procedure for their region and role, not a 2022 draft that still lives in a shared drive. If the assistant must open a change ticket afterward, that is a tool call with the user’s identity, not a paragraph the model invents. If every answer must start with risk classification codes in a fixed schema, that is where SFT or a strong structured-output path earns its keep — not by stuffing the entire wiki into weights.

How to choose an approach

A simple decision tree:

The main mistake is trying to solve everything with fine-tuning: “we have 100,000 PDFs, let’s train the model on them.” Documents change; stale knowledge in weights has no delete button; citation and access are poorly controlled; updates are expensive. The alternative: LLM + RAG + ACL + reranking + citations + evaluation.

The opposite mistake: expecting miracles from RAG without corpus engineering. Bad parsing, chunking, retrieval, tables in PDFs, prompt injection in documents — all of that breaks the answer before generation. Depth on that layer lives in the RAG guides linked above; here the point is the boundary of responsibility.

When you are unsure, start by writing the failure you fear most. If the failure is “answered with yesterday’s price,” you have a knowledge-freshness problem. If it is “ignored the JSON schema,” you have a behavior problem. If it is “could not create the ticket,” you have a tools problem. Naming the failure mode usually names the layer.

Evaluation — system maturity

Building does not end at generation. You need separate measurements:

• Retrieval: recall and precision of the hit list (Recall@K, Precision@K, MRR, NDCG — as orientation, not universal norms).
• Generation: factuality, grounding in context, citation correctness, answer relevance.
• System: latency, token cost, refusal rate, regressions after a model or index change.

A public model ranking is someone else’s sample. A product needs its own scenario set. A map of check types is in LLM quality testing; a regression loop is in AI eval harness.

Evaluation also prevents layer confusion. If groundedness fails while retrieval metrics look fine, you have a generation or prompt problem. If recall is weak, swapping the chat model will not help. If tool success rate drops after a gateway change, that is orchestration, not “the base model got worse.” Instrument each layer so regressions point at the right owner.

Architecture of a mature AI system

The final formula is not “RAG beats fine-tuning,” but:

Choosing between training the model and an external knowledge base is an architectural decision, not a technology contest.

                  AI SYSTEM
                      │
        ┌─────────────┼─────────────┐
        ↓             ↓             ↓
    Model          Knowledge       Tools
  pretraining         RAG        APIs / SQL
  SFT / LoRA        Search        Agents
  DPO/RLHF         Sources

Alongside that sit guardrails, ACL, observability, audit, and evaluation. The architect’s question is: which part of the task should live in model parameters, which in external knowledge, and which in tools and business logic?

If you need a production cut — RAG, agents, MCP, or an LLM gateway with budgets — see the AI implementation service.

FAQ

Conclusion

Training an LLM is not one lever — it is a stack of decisions. Pretraining and post-training shape capability and behavior; RAG keeps facts outside; tools enable action; evaluation keeps the system honest. In a code lab this looks like careful assembly of a path, not a magical “pour PDFs into the weights.”

Further reading in the cluster: why LLMs appeared, why enterprises need RAG, when fine-tuning is needed, production RAG engineering, LLM quality testing.