AI Research Scientist · Pretraining & Fine-Tuning Objectives
Supervised fine-tuning of language models
Supervised fine-tuning of language models
Supervised fine-tuning
Supervised fine-tuning (SFT) continues training a pretrained language model on labeled input-output pairs, using a supervised loss that pushes the model’s output toward a specific target response — unlike pretraining, which predicts the next token from raw, unlabeled text.Formatting a raw pair for SFT
- 1Extract the instructionIsolate the raw prompt: ‘Explain photosynthesis in one sentence.’
- 2Attach the target responsePair it with the exact desired output, e.g. ‘Plants convert light into chemical energy.’
- 3Wrap in a fixed templateStore both as {prompt, completion} so the loss has one clear target span to match.
Recall check from the same lesson
You could get the same instruction-following capability by simply continuing next-token pretraining on more raw text that happens to contain instructions and answers, without ever building labeled input-output pairs.
Review the explanation
Answer: False. Next-token pretraining has no notion of a designated target response — it just predicts whatever token follows in raw text. Without labeled pairs marking which span is the intended completion for a given instruction, there is no supervised signal steering the model toward producing a correct response; that labeled structure is exactly what SFT adds.
Sources
One sitting · 20–30 minutes
A focused session on your AI Research Scientist interview
LearnBench starts from what you already know — skip what you have, master what you’re missing.
Start now