Table of Contents
- 1. Overview
- 2. Learning Outcomes
- 3. Before You Begin
- 4. Retrieval Practice
- 5. Opening Scenario
- 6. Core Lesson Content
- 7. Process Walkthrough
- 8. Worked Example
- 9. Guided Learning Activities
- 10. Exercises
- 11. Homework Problems
- 12. Mini‑Project
- 13. Self‑Check Quiz
- 14. Assignment 1 Connection
- 15. Key Takeaways
- 16. Key Terms
- 17. Looking Ahead
- 18. Sources & Further Reading
1. Overview
In Lesson 2.1, you learned how generative models are built from training data and how they encode patterns in parameters. Now, in Lesson 2.2, you will dive into the generation process itself — how a model turns a prompt into a fluent response.
You will explore tokens and tokenization — the way text is broken into units that the model can process. You will learn about next‑token prediction, the fundamental mechanism by which language models produce extended text. You will examine how probabilities guide the selection of each token, and why this process is neither completely random nor a guarantee of truth.
By the end of this lesson, you will be able to explain why the same prompt can produce different outputs, why high‑probability tokens can still lead to false statements, and why fluency is not the same as factual accuracy. This understanding is essential for the responsible use of generative AI, which you will explore further in Lesson 2.3 and Unit 3.
2. Learning Outcomes
By the end of this lesson, you will be able to:
- Define token and tokenization at a conceptual level.
- Explain why a token is not necessarily a complete word.
- Describe how next‑token prediction can produce extended text.
- Explain how probabilities influence generated outputs.
- Distinguish probabilistic generation from unrestricted randomness.
- Explain why the same prompt may produce different outputs.
- Distinguish linguistic likelihood from factual correctness, source support, and confidence.
- Explain why repeated outputs agreeing with each other does not constitute independent verification.
3. Before You Begin
- Assigned reading: Chapter 2, sections on tokenization and generation.
- Prior knowledge: Complete Lesson 2.1, including the concepts of training, inference, parameters, and the distinction between training data and user context.
- Optional tool: You may use an approved generative AI system for some activities. A supplied‑output alternative is always available.
- Privacy reminder: Use only synthetic, public, or generalised information. Do not enter personal, confidential, or proprietary content.
- Academic integrity: All work should be your own. Document any AI use according to course policy.
- Preparation time: Approximately 15 minutes for reading and setup.
This lesson is aligned with Chapter 2. Where specific definitions, examples, or figures from the textbook are required but not yet available, a production placeholder appears. All such items are recorded in the instructor and production notes.
4. Retrieval Practice
Before we explore tokens and probabilities, let's activate the concepts from Lesson 2.1 that are essential for understanding this lesson.
Retrieval Question 1
Recall: What is the difference between training and inference? Why does the model's training data not update during inference?
Show suggested response
Answer: Training is the phase where the model's parameters are adjusted using a large dataset. Inference is the phase where the trained model processes a user's input and produces an output. During inference, the model's parameters are frozen — they are not updated. The training data is used only during training to create the model; it is not consulted during inference. Instead, the model applies the patterns it learned during training to the new input.
Retrieval Question 2
Recall: What are model parameters (weights), and why is it misleading to describe them as "stored facts"?
Show suggested response
Answer: Model parameters are numerical values that encode statistical patterns learned from training data. They are not stored sentences, conscious memories, or explicit rules. Describing them as "stored facts" is misleading because the model does not store documents or individual facts; it stores relationships among tokens and concepts in a high‑dimensional space. The model does not "know" which document a particular pattern came from.
Retrieval Question 3
Recall: Why does extensive training on a large dataset not guarantee that the model's outputs are complete, current, or accurate?
Show suggested response
Answer: Training data has limitations: gaps, errors, imbalances, and a cutoff date. The model learns patterns from whatever is in the data, including mistakes and biases. It does not have access to new information after the training cutoff. It also does not "remember" sources, so it cannot verify its own claims. Therefore, a model can produce fluent but incorrect or outdated information.
5. Opening Scenario
Scenario: Two students, Maria and Jamal, are working on a group project. They each use the same generative AI system and enter the exact same prompt: "Explain the difference between machine learning and deep learning in three sentences." Maria receives a response that emphasises the role of neural networks, while Jamal receives a response that focuses on the amount of data required. Both responses are grammatically correct and plausible, but they are different.
Maria says: "Something is wrong — the system gave different answers to the same question." Jamal replies: "Maybe it's just random, or maybe it's learning from our previous questions."
Questions for you:
- Is the variation necessarily a sign of malfunction? Why or why not?
- What factors might influence the differences between the two responses?
- How should Maria and Jamal evaluate the outputs for their project?
Show guided analysis of the scenario
1. Is variation necessarily a sign of malfunction?
No. Variation in outputs is a normal characteristic of most generative AI systems. The model does not produce a single deterministic answer; it generates responses based on probabilities, and small differences in the internal state (or even system settings like temperature) can lead to different token sequences. Variation does not mean the system is broken.
2. Factors that might influence differences:
- Probabilistic selection: The model may assign high probabilities to several possible next tokens, and the sampling process may choose different ones.
- System settings: Parameters like temperature or top‑p can affect variability.
- Conversation history: If the system retains previous interactions, the context may differ.
- Random seed: Some systems use a random seed for generation; different seeds yield different outputs.
- Model version: The system might be updated or load‑balanced across different instances.
3. How should they evaluate the outputs?
They should treat both outputs as drafts that require verification. They should check the claims against authoritative sources (e.g., textbooks, academic articles). They should also compare the two responses to identify common ground and areas of disagreement. They should not assume that either response is completely correct simply because it sounds confident.
Takeaway: Variation is expected and does not indicate a problem. However, it highlights the importance of human oversight: the outputs are provisional and must be verified.
6. Core Lesson Content
6.1 Tokens and Tokenization
Generative language models do not process text as whole words or sentences. Instead, they divide text into smaller units called tokens. The process of dividing text into tokens is called tokenization.
A token is a unit of text that the model can process. Tokens may represent:
- Common words (e.g., "the", "cat")
- Parts of words (e.g., "un", "ing", "tion")
- Punctuation (e.g., ".", ",", "?")
- Numbers, symbols, or other characters
- Whitespace (in some tokenizers)
Tokenization is system‑specific: different models use different tokenizers. A tokenizer determines how text is split and what each token represents. There is no universal tokenizer.
Important: A token is not necessarily a word. For example, the word "unbelievable" might be tokenized as "un", "believ", "able" or as a single token, depending on the tokenizer.
"Tokens are the same as words."
Correction: While some tokens may be whole words, many tokens are sub‑word units, punctuation, or other characters. Tokenization varies across systems, and a single word may be split into multiple tokens. This has implications for context length and processing efficiency.
[INSERT DEFINITION OF TOKEN AND TOKENIZATION FROM CHAPTER 2]
[INSERT TEXTBOOK EXAMPLE SHOWING HOW A SENTENCE IS TOKENIZED]
Why tokenization matters:
- Context capacity: Models have a limited context window measured in tokens, not words. A long word may consume multiple tokens, reducing the amount of text the model can consider.
- Multilingual variation: Tokenization can differ greatly across languages. Some languages have many sub‑word tokens, which can affect performance.
- No universal tokenizer: You cannot assume that a token count corresponds to a word count across different systems.
The following are conceptual token divisions — they are not meant to replicate any particular tokenizer. They illustrate how text can be split.
- English: "The cat sat on the mat."
[“The”] [“ cat”] [“ sat”] [“ on”] [“ the”] [“ mat”] [“.”] - Compound word: "unbelievable"
[“un”] [“believ”] [“able”](possible split) - Punctuation: "Hello, world!"
[“Hello”] [“,”] [“ world”] [“!”] - Number: "123.45"
[“123”] [“.”] [“45”](or as a single token) - Multilingual: "Bonjour tout le monde" (French)
[“Bon”] [“jour”] [“ tout”] [“ le”] [“ monde”](example only)
6.2 Next‑Token Prediction: How Generation Works
The core mechanism of a language model is next‑token prediction. Given a sequence of tokens (the prompt and any preceding generated tokens), the model estimates which token is most likely to come next.
- Input: The model receives a sequence of tokens (the prompt and any previously generated tokens).
- Probability distribution: The model assigns a probability (or score) to every token in its vocabulary. These probabilities indicate how likely each token is to be the next one, given the context.
- Selection: The system selects a token based on the probabilities — often using a sampling method (e.g., choosing the most likely token, or randomly sampling from the distribution).
- Append: The selected token is appended to the sequence.
- Repeat: The process repeats until a stopping condition is met (e.g., reaching a maximum length, generating an end‑of‑text token, or encountering a stop instruction).
This iterative process can produce long, coherent passages, even though the model only predicts one token at a time. The coherence emerges from the statistical patterns encoded in the model's parameters.
"The model plans the entire response in advance."
Correction: The model generates text token by token, without a pre‑planned outline. It does not "think" about the whole response before starting. Each token is chosen based on the tokens that came before it. This is why generated text can sometimes start strongly but then drift or become inconsistent.
6.3 Probability, Variability, and Truth
The next‑token selection is governed by probabilities, but these probabilities are not the same as truth, confidence, or correctness.
- Linguistic likelihood vs. factual correctness: A token may be highly probable because it frequently appears in that context in the training data, even if the resulting statement is false.
- Confidence vs. accuracy: The model does not have a built‑in confidence meter for factual claims; it only has probability scores for tokens.
- Variability: Because the model may sample from the probability distribution, the same prompt can produce different outputs. This is a feature, not a bug, but it means that outputs are not deterministic.
- High probability ≠ reliability: A highly probable token sequence can still be misleading, inaccurate, or fabricated.
The model does not select tokens completely at random. The probabilities are shaped by the patterns learned from training data, so the outputs are constrained to be linguistically plausible and contextually relevant. However, the selection process does involve stochasticity (randomness) in many systems, which leads to variability. This is why we say the generation is probabilistic — it is guided by learned probabilities, not pure chance.
Likely language is not necessarily true language. A model can produce a sequence of tokens that is grammatically perfect, fluent, and convincing, but that is factually wrong. The model has no way to verify the truth of what it generates; it only knows what is statistically plausible.
Therefore, never assume that a fluent, confident‑sounding output is accurate. Always verify.
6.4 Why Repeated Outputs Are Not Independent Evidence
If you ask the same prompt multiple times and get similar answers, you might be tempted to think that the answer is more reliable. However, this is a dangerous assumption.
- Same underlying model: All outputs come from the same model, which has the same learned patterns. Repeated generations do not provide new evidence; they just sample from the same distribution.
- Not independent: The outputs are correlated because they are generated by the same system. They do not constitute independent verification.
- Consistency ≠ correctness: The model may consistently produce the same wrong answer because it learned a common misconception from its training data.
Suppose you ask a model: "What is the capital of Australia?" The model might consistently answer "Sydney" (incorrect) because it learned a common error. Repeating the question 10 times will still yield "Sydney" most of the time. The consistency does not make the answer correct. You must verify against an authoritative source.
- Tokens are the units of text that a model processes; they are not always words.
- Tokenization is system‑specific and affects context capacity.
- Next‑token prediction is the iterative process of generating text.
- Probabilities guide token selection, but they are not a measure of truth.
- Generation is probabilistic, not deterministic nor purely random.
- Variability is normal; repeated outputs do not constitute independent verification.
7. Process Walkthrough: From Prompt to Generated Output
This walkthrough traces how a language model turns a user's prompt into a complete response, focusing on tokenization, next‑token prediction, and probability.
- User enters a prompt. Example: "Explain why the sky is blue."
- Tokenization. The prompt is divided into tokens by the system's tokenizer. For instance, the sentence might be tokenized as:
[“Explain”] [“ why”] [“ the”] [“ sky”] [“ is”] [“ blue”] [“?”] - Initial context. The tokenized prompt forms the initial context. The model has no other information unless additional context is supplied.
- First token prediction. The model computes a probability distribution over all tokens in its vocabulary for the next token. The highest‑probability tokens might include "because", "due", "since", etc.
- Token selection. The system selects one token, e.g., "because". This selection may be the most probable token (greedy) or sampled from the distribution (stochastic).
- Append and repeat. "because" is appended to the sequence. The model now predicts the next token given the longer sequence. This continues, generating tokens one by one.
- Stopping condition. The process stops when a maximum length is reached, an end‑of‑text token is generated, or another stop condition is met.
- Output. The final sequence of tokens is returned as the generated response.
Simplification note: This walkthrough omits many technical details (e.g., attention mechanisms, sampling algorithms). It is intended to build an accurate conceptual model.
Failure points: Tokenization errors, low‑quality probabilities, misleading patterns, and accumulation of errors can all lead to poor outputs. Human review is essential.
8. Worked Example: Generating a Brief Explanation
Human purpose: A student wants a simple, conversational explanation of why the sky is blue.
AI application: A large language model is used to generate a one‑paragraph answer.
Prompt: "Explain why the sky is blue in one sentence for a young child."
Tokenization (conceptual): The prompt is tokenized into something like: [“Explain”] [“ why”] [“ the”] [“ sky”] [“ is”] [“ blue”] [“ in”] [“ one”] [“ sentence”] [“ for”] [“ a”] [“ young”] [“ child”] [“.”]
First token prediction: After processing the prompt, the model assigns high probabilities to tokens like "The", "Sun", "Light", "Because". Suppose it selects "The".
Subsequent predictions: The model continues, generating "sky", "appears", "blue", "because", "of", "the", "way", "sunlight", "scatters", "in", "the", "atmosphere", ".".
Provisional output:
"The sky appears blue because of the way sunlight scatters in the atmosphere."
Capability demonstrated: The model produced a grammatically correct, clear, and concise sentence that conveys a common scientific explanation.
Limitation: The explanation is simplified and may not include details about Rayleigh scattering or the role of wavelengths. It is also unverified — the student should check with a trusted source.
Verification need: The student should compare the output with a science textbook or educational website to ensure accuracy and completeness.
Responsible human action: The student reads the output, checks it against a known source, and decides whether to use it in their work. They may revise it to add more detail or cite the source.
Preliminary conclusion: The model can provide a useful draft, but the output is provisional and requires human review.
9. Guided Learning Activities
- Purpose: To practise thinking about how text might be divided into tokens.
- Learning outcome: Propose plausible token divisions and explain uncertainty.
- Estimated time: 15–20 minutes.
Instructions: For each of the following, propose one or more possible token divisions. State explicitly that your divisions are conceptual and may not match any particular tokenizer. Discuss why tokenization is uncertain.
- "I can't believe it!" (with punctuation)
- "anti‑inflammatory" (a compound word)
- "123,456.789" (a number with punctuation)
- "Élan" (a word with an accent)
- "Hello, how are you?" (a simple sentence)
Expected product: A list of conceptual token divisions with notes on potential variations.
Show suggested token divisions and discussion
- "I can't believe it!" — Possible tokens:
[“I”] [“ can”] [“’t”] [“ believe”] [“ it”] [“!”]or[“I”] [“ can’t”] [“ believe”] [“ it”] [“!”](depending on whether "can't" is a single token or split). - "anti‑inflammatory" — Could be
[“anti”] [“‑”] [“inflammatory”]or[“anti”] [“inflamm”] [“atory”]etc. - "123,456.789" — Could be
[“123”] [“,”] [“456”] [“.”] [“789”]or a single token. - "Élan" — May be tokenized as
[“É”] [“lan”]or[“Élan”]depending on the tokenizer's handling of accents. - "Hello, how are you?" —
[“Hello”] [“,”] [“ how”] [“ are”] [“ you”] [“?”]
Discussion: Tokenization is not standardised. Different tokenizers may split words differently, especially with punctuation, compound words, numbers, and special characters. The example shows that multiple divisions are plausible. Always treat tokenization as system‑specific.
- Purpose: To experience how different token choices lead to different outputs.
- Learning outcome: Explain how next‑token prediction shapes the final text.
- Estimated time: 15–20 minutes.
Instructions: You are given the start of a sentence: "The patient's symptoms include …"
Consider three possible next‑token choices: (a) "fever", (b) "headache", (c) "fatigue".
For each choice, imagine the continuation that might follow. How would the final sentence differ? Which direction might lead to a factual‑sounding statement, a cautious statement, or an unsupported claim?
Expected product: A brief explanation of how each choice could lead to a different overall message.
Show model analysis
Choice a: "fever" — The continuation might be "fever, chills, and night sweats." This could lead to a factual‑sounding statement about possible infections.
Choice b: "headache" — The continuation might be "headache, nausea, and sensitivity to light." This could suggest a neurological or migraine‑related context.
Choice c: "fatigue" — The continuation might be "fatigue, muscle aches, and loss of appetite." This could point to a viral illness.
Key insight: The initial token choice shapes the entire subsequent generation. The model does not "know" which direction is correct; it simply follows statistical patterns. The final output may be plausible but unsupported, and it may vary significantly with the first token chosen.
- Purpose: To distinguish linguistic likelihood from factual correctness.
- Learning outcome: Identify why probability and truth are not the same.
- Estimated time: 10–15 minutes.
Instructions: Read the following pairs. For each pair, identify which statement is more linguistically likely (i.e., which sounds more natural) and which is factually correct (based on common knowledge). Then explain why they may differ.
- "The capital of France is Paris." vs. "The capital of France is London."
- "Water boils at 100°C." vs. "Water boils at 200°C."
- "The Earth is flat." vs. "The Earth is round."
Show suggested answers
- "The capital of France is Paris." is both linguistically likely and factually correct. "London" is linguistically plausible but factually incorrect.
- "Water boils at 100°C" is both likely and correct. "200°C" is less common and factually incorrect.
- "The Earth is round" is both likely and correct, though "flat" might be less likely in modern text but could appear in historical contexts.
Explanation: In these examples, the linguistic likelihood and factual correctness often align because the training data contains accurate information. However, this is not always the case. For less common facts or for topics where the training data contains errors, the model may produce a fluent but incorrect statement. Probability is a statistical measure of what appears in the training data, not a measure of truth.
- Purpose: To observe and compare multiple generated outputs for the same prompt.
- Learning outcome: Explain why outputs may vary and why agreement does not constitute independent verification.
- Estimated time: 20–30 minutes.
Instructions: Use an approved AI system (or supplied outputs) to generate three different responses to the same prompt: "Describe the benefits of exercise in one paragraph."
If you are using supplied outputs, you will find three sample responses below. Compare the responses in terms of wording, structure, examples, assumptions, factual claims, uncertainty, and omissions.
Then, answer: Does the fact that the responses are similar or different tell you anything about the reliability of the information? Why or why not?
Supplied outputs (alternative):
- Response 1: "Exercise offers numerous benefits, including improved cardiovascular health, stronger muscles, and better mental well‑being. Regular physical activity can reduce the risk of chronic diseases such as diabetes and obesity."
- Response 2: "Physical activity is beneficial for both body and mind. It helps maintain a healthy weight, enhances mood, and boosts energy levels. It also contributes to better sleep and a stronger immune system."
- Response 3: "Engaging in regular exercise can lead to a longer, healthier life. It strengthens the heart, builds endurance, and supports cognitive function. Moreover, it can be a social activity that improves emotional resilience."
Show model analysis
Comparison: All three responses are similar in tone and content — they all mention physical and mental benefits. However, the specific examples and phrasing differ. Response 1 focuses on disease prevention, Response 2 emphasises mood and energy, and Response 3 includes cognitive and social benefits.
Assumptions: All assume that exercise is generally beneficial and that the benefits are well‑established.
Factual claims: All claims are broadly supportable, but they are general and lack specifics (e.g., no mention of duration or intensity).
Omissions: None mention potential risks or contraindications.
Variability: The variability shows that the model can generate different yet plausible paragraphs. This is normal and not a sign of malfunction.
Reliability: The similarity across responses does not establish reliability. All three could contain generic, unverified claims. To confirm, you would need to check against authoritative health sources.
10. Exercises
These exercises help you apply the concepts. Each includes a self‑assessment option.
- Learning outcome: Propose plausible token divisions and explain uncertainty.
- Estimated time: 10 minutes.
- Task: For the following sentence, propose at least two different conceptual token divisions. Explain why tokenization is not fixed.
"She said, 'Hello, world!'"
Show sample solution
Possible division 1: [“She”] [“ said”] [“,”] [“ “] [“Hello”] [“,”] [“ world”] [“!”] [“’”]
Possible division 2: [“She”] [“ said”] [“,”] [“ “] [“Hello”] [“,”] [“ world”] [“!”] (including spaces as separate tokens)
Explanation: Tokenization depends on the specific tokenizer. Some tokenizers treat punctuation and spaces as separate tokens; others combine them. The quote marks may be handled differently. There is no single "correct" division.
- Learning outcome: Trace a simplified next‑token sequence.
- Estimated time: 10 minutes.
- Task: You are given the start of a sequence: "The quick brown fox jumps over the …"
List three plausible next tokens. For each, describe how the continuation might differ.
Show sample answer
Plausible next tokens: "lazy", "fence", "river".
- If "lazy" is chosen: "The quick brown fox jumps over the lazy dog." (classic phrase).
- If "fence" is chosen: "The quick brown fox jumps over the fence and runs away."
- If "river" is chosen: "The quick brown fox jumps over the river, but fails to reach the other side."
Insight: The chosen token determines the direction of the sentence. The model does not plan ahead; it reacts to each token as it is selected.
- Learning outcome: Distinguish probability from factual reliability.
- Estimated time: 10 minutes.
- Task: A model generates the following text: "The tallest mountain in the world is Mount Everest, which stands at 29,029 feet."
Is this statement likely to be linguistically probable? Is it factually correct? What additional verification would you perform?
Show sample answer
Linguistic probability: High. This is a common fact that appears frequently in training data.
Factual correctness: Correct. Mount Everest is the tallest mountain above sea level, and 29,029 feet is the accepted height (though it can vary slightly).
Additional verification: Even though it is correct, you should still verify with an authoritative source, such as a geological survey or encyclopedia, especially if the information is important. Also note that the height may be measured differently (e.g., snow cap vs. rock height).
- Learning outcome: Explain why repeated outputs may vary and why agreement does not constitute independent verification.
- Estimated time: 10 minutes.
- Task: A student runs the same prompt three times and gets the same output. They conclude that the answer is reliable. Explain why this conclusion is not justified.
Show sample answer
The model generates outputs based on its learned parameters, which are fixed during inference. Multiple runs do not consult new sources or provide independent evidence; they simply sample from the same underlying distribution. If the model has learned an error (e.g., a common misconception), it will consistently reproduce that error. Consistency across runs indicates that the model is internally consistent, but it does not indicate external validity. To verify, you must check against authoritative, independent sources.
11. Homework Problems
- Learning outcome: Define token and tokenization, and explain why tokens are not always words.
- Estimated time: 8 minutes.
- Task: Write a short paragraph (4–6 sentences) explaining what tokens are and why it is important to understand that a token is not the same as a word. Use an example to illustrate.
Show sample answer
Tokens are the units of text that a language model processes. They may be whole words, parts of words, punctuation, or other symbols. For example, the word "unbelievable" might be tokenized as "un", "believ", and "able" by some tokenizers, while others might treat it as a single token. This matters because context windows are measured in tokens, not words, and tokenization varies across systems. Therefore, one cannot assume that a token count equals a word count, and one cannot generalise tokenization rules from one model to another.
- Learning outcome: Explain conceptually how next‑token prediction can produce extended text.
- Estimated time: 10 minutes.
- Task: Imagine you are teaching a friend how a language model generates a response. Write a dialogue (6–8 exchanges) in which you explain that the model predicts one token at a time, and that it does not plan the entire response in advance. Use a concrete example.
Show solution design and hints
Hint 1: Start with the friend asking, "How does the model write a whole paragraph?"
Hint 2: Explain that the model starts with the prompt, then predicts the next token, appends it, and repeats.
Hint 3: Use an example like "The weather today is …" and show how the next token could be "sunny", "rainy", etc., and how that choice shapes the rest.
Solution design: The dialogue should correct the misconception that the model has a pre‑written answer. Emphasise that generation is incremental and probabilistic.
- Learning outcome: Distinguish probability from factual reliability.
- Estimated time: 10 minutes.
- Task: A model assigns a high probability to the token "ate" after the prompt "The cat ___ the mouse." The model is more likely to select "ate" than "saw" because "ate" is more common in that context. Explain why a high probability for "ate" does not necessarily mean that the cat actually ate the mouse in a specific story. What does the probability reflect?
Show sample answer
The probability reflects the statistical likelihood of the token "ate" given the context "The cat ___ the mouse" based on the patterns learned from the training data. It does not reflect any knowledge of a specific story; it reflects that in many texts, cats are described as eating mice. The model has no way to know whether, in the particular context being generated, the cat actually ate the mouse — it is just producing the most probable continuation. Therefore, probability is not a measure of truth or factual accuracy.
- Learning outcome: Explain why the same prompt may produce different outputs.
- Estimated time: 10 minutes.
- Task: A user complains that an AI assistant gave different answers to the same question. Write a brief, polite response explaining why this can happen and what the user should do to verify the answers.
Show sample answer
Thank you for your question. Generative AI systems often produce different responses to the same prompt because they generate text probabilistically — they may sample from a distribution of possible next tokens, which can lead to variation. This is a normal characteristic, not a malfunction. To get a reliable answer, I recommend that you compare the responses, check the claims against authoritative sources, and verify any critical information. The model's outputs are drafts that require human review.
- Learning outcome: Distinguish fluency from factual correctness.
- Estimated time: 8 minutes.
- Task: Provide an example of a statement that is highly fluent but factually incorrect. Explain why the model might generate such a statement.
Show sample answer
Example: "The capital of the United States is New York City." This sentence is grammatically correct and fluent, but it is factually incorrect — the capital is Washington, D.C. The model might generate this if its training data contained frequent references to New York as an important city, leading to a statistical pattern that sometimes predicts "New York" in contexts about important U.S. cities. The model has no way to verify the fact; it only knows that "New York" is a plausible completion in some contexts.
- Learning outcome: Recognise that tokenisation may vary across languages.
- Estimated time: 8 minutes.
- Task: Why might a model perform differently on texts in languages that have many compound words or inflections? How could tokenisation affect context capacity?
Show sample answer
Languages with many compound words or inflections may have longer words that are tokenized into multiple sub‑word tokens. This means that a sentence in such a language may require more tokens than a similar sentence in English, reducing the amount of text that fits into the context window. Additionally, if the tokenizer is not well‑optimised for that language, the model may produce lower‑quality outputs. Thus, tokenization differences across languages can affect both performance and capability.
- Learning outcome: Explain how context influences token predictions.
- Estimated time: 10 minutes.
- Task: Given the prompt "The bank is …" what are two very different continuations that depend on context? How would the model's token predictions differ based on surrounding context? (Assume you can provide additional context before generating.)
Show sample answer
Without context, "The bank is …" could continue as "closed" (financial institution) or "eroding" (river bank). If the preceding context mentions "money" or "finance," the model is more likely to predict tokens related to a financial bank. If the context mentions "river" or "water," it will predict tokens related to a river bank. The probabilities shift dramatically based on the surrounding words, demonstrating how context shapes generation.
- Learning outcome: Apply concepts to an unfamiliar context.
- Estimated time: 12 minutes.
- Task: A journalist uses a generative AI to draft an article about a recent scientific breakthrough. The journalist runs the same prompt three times and receives three slightly different drafts. The journalist is concerned that the AI is inconsistent. Write a short memo to the journalist explaining why variability is normal, what factors might cause it, and how to evaluate the drafts responsibly.
Show sample answer
Memo:
Variability in generative AI outputs is a normal feature of probabilistic systems. The model does not produce a single deterministic answer; it generates text by sampling from a probability distribution, which can lead to slight differences. Factors include random seed, system settings (e.g., temperature), and even minor differences in how the prompt is processed.
To evaluate the drafts: treat each as a draft, not a final product. Compare them for common themes and notable discrepancies. Verify key claims and data against authoritative sources. Check that the drafts do not introduce errors or fabrications. Ultimately, you as the journalist are responsible for the accuracy of the final article. The AI is a tool to assist, not to replace your judgment.
12. Mini‑Project: Token‑to‑Text Generation Analysis
- Purpose: To analyse a simplified generation sequence, exploring how token choices shape meaning, variability, and the need for verification.
- Learning outcomes: Explain how next‑token prediction works conceptually; distinguish probability from truth; identify the importance of human verification.
- Estimated effort: 60–90 minutes.
Instructions: Select a simple, low‑risk prompt of your choice (or use the supplied prompt). Analyse the generation process conceptually.
Supplied prompt: "Describe a healthy breakfast in one sentence."
Required components:
- Initial prompt: Write the prompt you are using.
- Conceptual token divisions: Propose a tokenization of the prompt.
- Three plausible next‑token paths: After the prompt, list three possible first tokens and briefly describe how each might lead to a different sentence.
- Resulting differences in meaning: Show how the three paths produce different meanings.
- One path that produces unsupported certainty: Choose one continuation that might sound overly confident or make a strong claim without evidence.
- One path that preserves uncertainty: Choose a continuation that is more cautious or conditional.
- Explanation of why probability is not evidence: Explain that even a high‑probability token does not guarantee the statement is true.
- Human‑verification requirement: State what a human would need to check in the generated output.
- One AI Paradox opportunity‑risk pair: Connect the generation process to an opportunity and a risk.
Deliverable: A structured analysis (text‑based) that addresses all nine components.
- Use only synthetic, public, or generalised information.
- Do not use personal, confidential, or proprietary content.
- If you use an AI tool to assist, document your use.
Show self‑review checklist
- All nine components are addressed.
- Token divisions are conceptual and not claimed to be exact.
- Three different paths are clearly described.
- Unsupported certainty and cautious paths are identified.
- The explanation of probability vs. truth is clear.
- Human‑verification requirement is practical.
- Opportunity‑risk pair is relevant.
Show Hint 1: Choosing a prompt
Choose a prompt that is open‑ended enough to allow different continuations. Avoid prompts that are too specific or have a single correct answer.
Show Hint 2: Token divisions
You can divide the prompt into tokens conceptually. For example, "Describe a healthy breakfast in one sentence." might be divided as [“Describe”] [“ a”] [“ healthy”] [“ breakfast”] [“ in”] [“ one”] [“ sentence”] [“.”]. Remember, this is just an example; actual tokenization may differ.
Show Hint 3: Opportunity‑risk pair
Consider the opportunity of generating quick, varied ideas vs. the risk that the generated ideas are nutritionally inaccurate or over‑simplified.
Show solution architecture and partial example
Solution architecture:
- State the prompt.
- Show conceptual tokenization.
- List three possible next tokens (e.g., "A", "Eggs", "Oatmeal").
- For each, write a one‑sentence continuation.
- Highlight differences.
- Identify one path that is overly confident (e.g., "A healthy breakfast is essential for everyone.") and one that is cautious (e.g., "A healthy breakfast often includes whole grains and protein.").
- Explain that probability does not validate the claims.
- State verification steps: check nutritional guidelines.
- Opportunity: quick meal ideas; Risk: misleading dietary advice.
Partial example:
Prompt: "Describe a healthy breakfast in one sentence."
Tokenization (conceptual): [“Describe”] [“ a”] [“ healthy”] [“ breakfast”] [“ in”] [“ one”] [“ sentence”] [“.”]
Next token paths:
1. "A" → "A healthy breakfast includes fruits and whole grains."
2. "Eggs" → "Eggs with vegetables and whole‑grain toast make a healthy breakfast."
3. "Oatmeal" → "Oatmeal topped with berries and nuts is a healthy breakfast."
...
13. Self‑Check Quiz
These questions are ungraded and include explanatory feedback.
Question 1
What is a token in the context of a language model?
Show answer and explanation
Correct answer: B.
Explanation: Tokens are the basic units processed by the model. They are not necessarily whole words; they can be sub‑word units, punctuation, or other characters.
Question 2
Why is tokenization system‑specific?
Show answer and explanation
Correct answer: B.
Explanation: There is no universal tokenizer. Each model may use a different tokenization method, vocabulary size, and rules for splitting text.
Question 3
How does a language model generate extended text?
Show answer and explanation
Correct answer: B.
Explanation: Generation is iterative: the model predicts the next token given the current sequence, appends it, and repeats until a stopping condition is met.
Question 4
What role do probabilities play in generation?
Show answer and explanation
Correct answer: B.
Explanation: The model assigns a probability to each possible next token based on the learned patterns. These probabilities guide token selection, but they do not guarantee correctness.
Question 5
Why is probabilistic generation not the same as unrestricted randomness?
Show answer and explanation
Correct answer: B.
Explanation: The probabilities come from the model's training, so the outputs are statistically plausible. The generation is probabilistic but not random in the sense of being unconstrained; it follows the patterns learned from the data.
Question 6
If you ask the same prompt multiple times and get similar answers, what can you conclude?
Show answer and explanation
Correct answer: B.
Explanation: Repeated outputs come from the same model and are not independent evidence. The model may consistently produce an error. Always verify against authoritative sources.
Question 7
Which statement about linguistic likelihood and factual correctness is true?
Show answer and explanation
Correct answer: C.
Explanation: A model can produce highly probable token sequences that are factually wrong, because probability is based on training data patterns, not on truth.
Question 8
Why might the same prompt produce different outputs in a generative AI system?
Show answer and explanation
Correct answer: B.
Explanation: Variability is a normal consequence of probabilistic generation. Factors like random seed, temperature, and sampling methods can produce different outputs for the same input.
14. Assignment 1 Journal Connection
As you work through this lesson, consider these journal prompts for Assignment 1.
- Mental model shift: Before this lesson, what did you think it meant for a model to "generate" text? How has your understanding changed after learning about tokens and next‑token prediction?
- Probability vs. truth: Why is it important to distinguish between what is linguistically likely and what is factually correct? Give an example from your own experience or a hypothetical one.
- Variability: How does the probabilistic nature of generation affect your trust in AI outputs? What steps will you take to verify information?
- Question for Unit 3: Based on what you have learned about tokenization and probability, what is one question you want Unit 3 to address about using AI responsibly?
Show reflection‑planning checklist
- I have reviewed the lesson content and my notes.
- I have identified at least one change in my mental model about generation.
- I have articulated the distinction between probability and truth.
- I have reflected on variability and verification.
- I have formulated at least one question for Unit 3.
15. Key Takeaways
- Tokens are the units of text processed by a model; they are not always words and tokenization is system‑specific.
- Next‑token prediction is the iterative process that generates text; the model does not plan the entire response in advance.
- Probabilities indicate the model's statistical estimate of what token is likely next, but they are not a measure of truth, confidence, or correctness.
- Generation is probabilistic — guided by learned patterns but not deterministic; variation is normal.
- High probability does not guarantee factual accuracy; a fluent, confident‑sounding output can be false.
- Repeated outputs from the same model do not constitute independent verification.
- Always verify generated content against authoritative sources and treat it as provisional.
16. Key Terms
- Token
- A unit of text processed by a language model; may be a word, part of a word, punctuation, or other character.
- Tokenization
- The process of dividing text into tokens, which varies across systems.
- Next‑token prediction
- The iterative process by which a language model generates text, predicting one token at a time based on the preceding context.
- Probability distribution (over tokens)
- The set of probabilities assigned to each possible next token, indicating how likely each is given the current context.
- Sampling
- A method of selecting a token from the probability distribution, which may involve randomness.
- Greedy decoding
- A method of selecting the most probable token at each step, producing a deterministic output.
- Stochastic (probabilistic) generation
- Generation that involves randomness in token selection, leading to variability.
- Linguistic likelihood
- The probability of a token sequence based on patterns in the training data, not necessarily indicating factual truth.
- Factual correctness
- The alignment of a statement with verifiable reality, which the model cannot directly assess.
- Variability
- The tendency of a probabilistic system to produce different outputs for the same input.
- Independent verification
- Checking information against an external, authoritative source, not just relying on the model's own consistency.
17. Looking Ahead
In this lesson, you learned how generative models produce text through tokenization and next‑token prediction, guided by probabilities. You also learned why probability is not truth and why variability is normal.
In Lesson 2.3: Prompts, Context Windows, and Model Outputs, you will explore how prompts, system instructions, conversation history, and context windows shape generation. You will learn about the limitations of context and why information in the prompt is not automatically verified.
The concepts you have learned — tokenization, prediction, and probability — are foundational for understanding why context matters and why you cannot assume that a model "knows" or "understands" the information it processes.
18. Sources & Further Reading
- Primary: COMP 3270 Textbook, Chapter 2: How Generative AI Works (sections on tokenization and generation).
- Supporting: COMP 3270 Appendix A: Generative AI Glossary.
- Supporting: COMP 3270 Appendix D: Responsible‑Use Toolkit.
- External (if available): Current authoritative technical documentation on tokenization and language model inference.
Specific definitions, examples, and figures from Chapter 2 will be inserted once the textbook content is available. All unresolved items are recorded in the instructor and production notes.