Practical AI training for everyone
Module 1
9 min read
Most people learn prompting by collecting phrases. Act as an expert. Take a deep breath. Think step by step. Some of these help, some did once and no longer do, and none of them explain why a prompt that worked on Tuesday produced something useless on Thursday. The explanation is worth more than the collection.
A language model takes the text you supply and produces what plausibly follows it. That is the whole mechanism. When you write a good prompt, you are not persuading anything. You are constructing a piece of text whose most plausible continuation happens to be the answer you want.
This reframes every technique in this course. Asking for a role works because text that begins like an expert document continues like one. Asking for reasoning steps works because a text containing working is more likely to continue with a correct conclusion than one that jumps straight to an answer. Nothing is being convinced. The distribution of likely next words is being shaped.
The model does not pick the single most likely next word every time. It samples from a distribution. That randomness is deliberate, since always taking the top choice produces flat repetitive text, and it has one consequence you must build around: a prompt is not a function. Running it once tells you what it can do, not what it will do.
Almost every frustration people report with these tools traces back to this. A prompt is judged on one impressive result, deployed, and then behaves differently on the fourth attempt. It was never reliable. It was sampled once, favourably.
The same prompt, three runs
"Summarise this report in three bullets." Run one returns three bullets. Run two returns four, one of which is a heading. Run three returns three bullets, but the third editorialises about what the company should do next. Nothing changed except the sample. Any downstream process expecting exactly three factual bullets is now broken twice out of three times, and the person who wrote the prompt believes it works.
Four things, in order. How to construct a prompt from parts that each do a job. How to diagnose which part is failing when the output is nearly right. How to constrain output so a machine or a person downstream can rely on it. And how to measure all of that with a test set, so that "it works" becomes a number you can defend.
Try it now
Take a prompt you use regularly and run it five times on the same input, without changing a word. Save all five outputs side by side. The differences between them are the real quality of that prompt, and most people have never once looked at this.
A few quick questions. You'll see an explanation after each one.
Question 1 of 5
Why does the same prompt produce different answers on different runs?