Inside language models · Module 1: Inside language models in practice
Prompting, retrieval and fine-tuning
Learning outcome: Choose the kind of change a task actually needs.
Prompting supplies instructions and examples at use time. Retrieval supplies selected external information. Fine-tuning changes model parameters using a training process. These approaches solve different problems and may be combined. A style problem does not automatically require training; a changing policy is not automatically a good training target.
Diagnose the failure first. Missing current facts point towards a source access problem. Repeated format errors suggest checking instructions, examples and validation. A persistent specialised behaviour may justify investigating training, but that also requires representative data, held-out evaluation and maintenance. Never assume a trained model will reliably reproduce every supplied record or remove the need for evidence.
Worked example
A fictional assistant writes accurate answers in the wrong template and also quotes last year's travel policy. Those are two problems. The team tests a clear response schema for the template and retrieves the approved current policy for the facts. It measures the results before considering the additional work of fine-tuning.
Try it yourself
Classify three failures: missing current opening hours, inconsistent headings and a specialised classification task with many labelled examples. Suggest a first intervention and a way to test it.
You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.
Compare your approach
Use a current source for opening hours. Test instructions and validation for headings. For classification, compare a simple baseline and examples before considering training. Different failures need different evidence of improvement.
Check your understanding
An assistant lacks this week’s approved timetable. What should you investigate first?
- Training on last year’s timetable
- Access to the current timetable
- A more enthusiastic system prompt
Show the answer and explanation
Answer: Access to the current timetable. The missing item is current information. Retrieve or supply the approved timetable and check the answer against it. A style instruction or obsolete training example cannot establish this week’s facts.
Workbook
Download the course workbook to record your answer and compare it after the knowledge check.