Prompting changes the instructions and context sent with a request. Retrieval supplies current or private evidence at request time. Training changes model behavior through an additional learning process. They solve different failures. If the model receives the right facts but ignores an output constraint, inspect the prompt and validation. If the facts never arrive, repair ingestion or retrieval. If a stable, frequent behavior remains poor after clean instructions and evidence, training may be worth evaluating against a fixed baseline. No option removes the need for held-out tests.
Make this comfortable
Prompting, retrieval, or training: choose the failing layer
Decision in practice
A parts-support assistant repeatedly gives an old replacement rule. The team first finds that the current rule is absent from the retrieved set; rewriting the prompt cannot supply a document that was never fetched. After the index is repaired, most cases pass. A small formatting defect remains and is fixed with a schema. The team does not train a model on copies of the policy, because that would put time-bound facts in weights and make a later policy change harder to trace. The release packet records which layer changed and the cases it affected.
Failure A: correct evidence present, wrong field shape -> schema and prompt.
Failure B: current policy absent -> ingestion and retrieval.
Failure C: repeated behavior defect after clean inputs -> compare training with baseline.
For each: freeze cases, cost, and acceptance checks.Performance and operating cost
Prompt edits are usually quick to test; retrieval requires an index and freshness controls; training adds data preparation, evaluation, and lifecycle cost. Comparing them requires cost per accepted case, not a claim that one technique is always cheaper. If N policy updates occur, embedding current facts in model weights can add repeated retraining work. Keep private and time-sensitive facts in governed data stores when a request-time lookup can supply them. Diagnose the input path before paying to change model behavior.
Common Mistakes
- Do not ask a prompt to invent missing source facts.
- Do not use training as a substitute for current policy retrieval.
- Do not call a formatting problem a knowledge problem.
Connected lessons
prompt engineering
tutorial