COMP 3270 / Unit 2 / Unit Home / Lesson 3

Lesson 2.3 · Week 2 · Reading: Chapter 2

Prompts, Context Windows, and Model Outputs

Explore how prompts, instructions, conversation history, and context windows shape what a model can consider — and why context is not automatically verified.

Suggested time: 100–120 minutes Continues Unit 2 Builds on Lesson 2.2

Table of Contents

1. Overview

In Lesson 2.2, you learned how generative models produce text through tokenization and next‑token prediction, guided by probabilities. Now, in Lesson 2.3, you will explore how prompts, instructions, conversation history, and context windows shape the output of a generative AI system.

You will learn what a context window is, why it is limited, and how its size affects the model's ability to consider information. You will examine how different types of context — missing, excessive, inaccurate, or conflicting — can influence the quality and reliability of the output. Crucially, you will understand why information supplied in a prompt or retrieved from a source is not automatically verified, and why a generated response does not reflect durable memory or human understanding.

This lesson prepares you for Lesson 2.4, where you will explore retrieval, tools, multimodal AI, and the distinction between a model and a complete AI system. It also sets the stage for Unit 3, where you will learn to assess the suitability and risks of AI use in more depth.

Responsible‑use reminder: Do not enter personal, confidential, proprietary, privileged, or security‑sensitive information into any public or unapproved AI system. Use only synthetic, public, or generalised information for all activities.

2. Learning Outcomes

By the end of this lesson, you will be able to:

  1. Describe how prompts, instructions, context, and conversation history influence generation.
  2. Define context window and explain its practical limitations.
  3. Identify how missing, irrelevant, excessive, inaccurate, or conflicting context may affect an output.
  4. Distinguish information supplied in context from independently verified evidence.
  5. Explain why a larger context window does not automatically produce better output.
  6. Diagnose conversation or context drift and describe when to restart or restate a task.
  7. Identify technical questions that must be considered when evaluating an AI‑supported activity.

3. Before You Begin

📖 Textbook integration note

This lesson is aligned with Chapter 2. Where specific definitions, examples, or figures from the textbook are required but not yet available, a production placeholder appears. All such items are recorded in the instructor and production notes.

4. Retrieval Practice

Before we dive into prompts and context, let's activate key concepts from Lessons 2.1 and 2.2.

Retrieval Question 1

Recall: What is the difference between training and inference? Why does this distinction matter when we talk about prompts and context?

Show suggested response

Answer: Training is the phase where the model's parameters are adjusted using a large dataset. Inference is the phase where the trained model processes a user's input (prompt) and produces an output. During inference, the model's parameters are frozen — they are not updated. This matters because prompts and context are supplied during inference and do not change the model's underlying knowledge. They only influence the generation process by shaping the immediate input to the model.

Retrieval Question 2

Recall: Why is fluency not a reliable indicator of truth? How does this relate to the information you might supply in a prompt?

Show suggested response

Answer: Fluency reflects the model's ability to produce likely language patterns, not its ability to verify facts. The model can generate fluent, confident‑sounding text even if the content is false. This means that even if you supply correct‑looking context in the prompt, the model may not treat it critically; it may incorporate it as if it were true. You, as the user, are responsible for verifying the accuracy of both the context you supply and the output the model generates.

Retrieval Question 3

Recall: In Lesson 2.2, you learned that the same prompt can produce different outputs because of probabilistic generation. How might this variability interact with context to affect output consistency?

Show suggested response

Answer: Probabilistic generation introduces variability even with the same prompt. When you add context, the variability may be reduced because the context narrows the range of plausible next tokens. However, variability is not eliminated; the model still samples from a probability distribution, so outputs may differ even with identical context. Context can constrain the generation, but it does not make it deterministic.

5. Opening Scenario

📄 The Policy Summary Conundrum

Scenario: A human resources manager, Alex, asks an AI system to summarise the company's remote work policy. Alex provides the following information in the prompt:

  • An excerpt from an older version of the policy (from 2020).
  • A note about a recent update that changed the eligibility criteria.
  • A few sentences from a team meeting that suggested flexible hours.
  • An instruction: "Focus on the latest rules, but also mention the old ones."

The AI generates a summary that includes outdated information, incorrectly states that flexible hours are mandatory, and omits the new eligibility criteria. Alex is puzzled: "I gave the system all the right information — why did it get it wrong?"

Questions for you:

  1. What assumptions is Alex making about how the AI uses the supplied context?
  2. How might conflicting, incomplete, or excessive context affect the output?
  3. What should Alex do to get a more accurate summary?
Show guided analysis of the scenario

1. Alex's assumptions:

  • Alex assumes the AI will critically compare and integrate all pieces of context.
  • Alex assumes the AI will prioritise the most recent information.
  • Alex assumes that the AI can resolve conflicts between different sources.

2. How conflicting, incomplete, or excessive context can affect the output:

  • Conflicting information: The model may mix outdated and updated rules, producing a contradictory summary.
  • Incomplete information: If the new eligibility criteria are not fully described, the model may fill in gaps with assumptions from its training data.
  • Excessive information: Too much context may overwhelm the model's limited context window, causing it to drop or downweight important details.
  • Conflicting instructions: The instruction to "focus on latest rules, but also mention old ones" may lead to an unclear or muddled response.

3. What Alex should do:

  • Provide only the most current, authoritative source (e.g., the latest policy document) as context.
  • Use a clear, unambiguous instruction, e.g., "Summarise the current remote work policy based on this document. Do not include outdated rules."
  • Verify the output against the original policy document.
  • Consider using a structured prompt that asks for specific sections (e.g., eligibility, hours, exceptions).

Takeaway: Context is not automatically processed perfectly by the model. The quality, clarity, and relevance of the context you supply directly influence the output. You must treat the output as provisional and verify it against authoritative sources.

6. Core Lesson Content

6.1 Prompts, Instructions, and Conversation History

The prompt is the primary input you provide to a generative AI system. It may include:

How prompts influence generation

The prompt and any included context become the initial sequence of tokens that the model uses to begin generation. The model conditions its next‑token predictions on this entire sequence. Therefore, the phrasing, structure, and content of your prompt can significantly shape the output.

Conflict among instructions: If the user prompt, system instruction, and supplied context contain conflicting directives, the model may produce an output that is inconsistent, or it may prioritise one instruction over others in unpredictable ways.

⚠️ Common misconception

"The system understands my instructions exactly as I intended."
Correction: The model processes instructions as patterns of tokens, not as human‑like understanding. Ambiguity, vagueness, or implicit assumptions can lead to outputs that differ from your intent. Clarity and specificity in your prompt are essential.

6.2 Context Windows: Definition and Limitations

A context window is the maximum number of tokens that the model can consider at one time. It includes the prompt, system instructions, conversation history, and any supplied or retrieved context.

Key points about context windows
  • Limited capacity: The context window is finite (e.g., a few thousand to over a hundred thousand tokens, depending on the model).
  • Tokens, not words: The window is measured in tokens, which may be words or sub‑word units. A long document may exceed the window.
  • Oldest information may be dropped: In many systems, when the context window is full, the oldest tokens are discarded to make room for new ones. This means early parts of a long conversation or document may no longer influence later responses.
  • Long context does not guarantee better output: Even with a large window, the model may not attend evenly to all parts; it may focus on the beginning or end, or it may lose track of details.
  • Varies by system and version: Exact context window sizes are product‑specific and change over time. Always check the current documentation.
⚠️ Common misconception

"A larger context window means the model can remember everything and produce perfect answers."
Correction: While a larger window allows more information to be considered, the model's ability to use that information effectively is limited by its architecture. It may still miss details, be confused by long‑range dependencies, or attend to the wrong parts. Quality of context matters more than quantity.

[INSERT VERIFIED CONTEXT‑WINDOW DIAGRAM]
Figure 1. Simplified illustration of a context window containing prompt, system instructions, conversation history, and supplied context. (Placeholder — a verified diagram will be inserted in the production version.)

6.3 How Context Quality Affects Output

The model's output is heavily influenced by the quality, relevance, and accuracy of the context you provide. Different types of context issues can lead to different types of errors.

Table 1. Types of context issues and their potential effects
Issue Description Potential effect on output
Missing information Necessary details are not provided. The model may fill gaps with assumptions, leading to incomplete or inaccurate output.
Inaccurate information Context contains factual errors. The model may reproduce the errors as if they were correct.
Excessive information Too much context for the window, or too much irrelevant detail. The model may omit critical details, become confused, or produce a rambling response.
Irrelevant information Context not related to the task. The model may include irrelevant content or be distracted.
Ambiguous information Vague or unclear context. The model may interpret it differently than intended, leading to off‑target output.
Conflicting information Contradictory pieces of context. The model may produce inconsistent or contradictory output, or may arbitrarily pick one source.
Outdated information Context that is no longer current. The output may reflect outdated policies, facts, or assumptions.
Unlabelled sources Context provided without indicating which source it came from. The model cannot attribute information, making verification difficult.
Hidden instructions Instructions embedded in context that are not explicitly stated as instructions. The model may inadvertently follow unintended directives.
Assumptions Context that relies on unstated assumptions. The model may accept those assumptions, leading to output that is valid only under those assumptions.
🧠 Critical insight

Information in a prompt is not automatically verified. The model treats the context you supply as part of its input. It does not fact‑check or question that context. If you provide inaccurate or misleading information, the model is likely to incorporate it into its output. This means that the responsibility for the accuracy of the output — and the context you supply — rests entirely with you.

6.4 Conversation Drift and Context Degradation

In multi‑turn interactions, the conversation history becomes part of the context. Over time, the task or focus may shift, or the context window may fill up and cause older messages to be dropped. This can lead to conversation drift.

Example of conversation drift

Turn 1: User: "Summarise this article about climate change."
Turn 2: AI: "The article discusses rising temperatures…"
Turn 3: User: "What about the economic impacts?"
Turn 4: AI: "Economic impacts include…"
Turn 5: User: "Can you give me a list of key policies?"
Turn 6: AI: "Key policies include…"

After several turns, the AI may lose track of the original article and start generating generic information, rather than staying focused on the specific source. This is drift. To recover, the user may need to restate the original task or provide the article again.

📋 Core concepts recap
  • Prompts include instructions, context, and conversation history.
  • Context windows are limited and affect which information the model can use.
  • The quality of context — missing, inaccurate, excessive, conflicting, etc. — directly shapes output.
  • Context is not verified by the model; you are responsible for accuracy.
  • Conversation drift can occur as context accumulates; restating or restarting may be necessary.

7. Process Walkthrough: How Context Flows Through a Model

This walkthrough traces how the input (prompt + context) is processed and how the context window limits what the model can consider.

Step‑by‑step conceptual flow
  1. User composes input. The user writes a prompt and may include system instructions, task instructions, and additional context (documents, conversation history, etc.).
  2. Tokenization. The entire input (prompt + context) is tokenized into a sequence of tokens.
  3. Context window check. The total number of tokens is compared to the model's maximum context window. If the input exceeds the window, the system may truncate it (e.g., discard the oldest tokens) or reject it.
  4. Model processing. The model processes the token sequence (up to the window limit) and generates a response using next‑token prediction.
  5. Output. The model returns the generated tokens as a response.
  6. Human review. The user reviews the output, verifies claims, and decides whether the context was used appropriately.

Failure points: If the context is too long, important information may be cut off. If the context is ambiguous or conflicting, the model may generate misleading output. If the conversation drifts, the original intent may be lost.

[INSERT VERIFIED CONTEXT‑TO‑OUTPUT PROCESS DIAGRAM]
Figure 2. Simplified flow of context through a generative AI system. (Placeholder — a verified diagram will be inserted in the production version.)

8. Worked Example: Context Quality Audit

Scenario: Drafting a customer service email

Human purpose: A customer support agent needs to draft a response to a customer who complained about a delayed shipment.

AI application: A generative AI system is used to draft the email based on provided context.

Provided context:

  • Customer email: "My order #12345 was supposed to arrive on Monday, but it's now Friday and I haven't received it. This is unacceptable."
  • Company policy: "Standard shipping takes 3‑5 business days. We apologise for delays and offer a 10% discount on the next order."
  • Order status: "Order #12345 was shipped on Tuesday, expected delivery is Saturday."
  • Irrelevant note: "The weather in the warehouse is cold."
  • Conflicting instruction: "Be firm with the customer, but also be apologetic."

Generation process: The model tokenises the prompt and context, and generates a draft email.

Provisional output (simplified):

"Dear Customer, Thank you for your email. We understand your frustration. Your order was shipped on Tuesday and is expected to arrive on Saturday. We apologise for the delay and will offer a 10% discount on your next order. We also note that the warehouse is cold, which may have affected shipping. Please let us know if you have further questions."

Capability demonstrated: The model incorporated most of the relevant context and produced a coherent draft.

Limitations:

  • The irrelevant "cold warehouse" note was included, which is unprofessional and irrelevant.
  • The conflicting instruction ("be firm but apologetic") resulted in a mixed tone.
  • The model did not explicitly apologise for the delay beyond a generic statement.

Verification need: The agent should remove the irrelevant note, clarify the apology, and ensure the discount offer aligns with company policy.

Responsible human action: The agent edits the draft, removes the irrelevant information, strengthens the apology, and verifies the delivery date with the shipping provider.

Preliminary conclusion: The model can produce a useful draft, but the output must be reviewed and refined by a human. The quality of the context directly affects the output quality.

9. Guided Learning Activities

Activity 1: Context Component Identification
  • Purpose: To distinguish between different types of inputs in a prompt.
  • Learning outcome: Classify content as system instruction, user instruction, conversation history, supplied source, retrieved context, irrelevant information, or potentially untrusted content.
  • Estimated time: 15 minutes.

Instructions: Read the following prompt package and classify each piece of content according to the categories above.

Prompt package:

  • System instruction: "You are a knowledgeable assistant that provides concise, factual answers."
  • User instruction: "Summarise the attached policy document in 100 words or less."
  • Conversation history: (previous turn) "What is the company's policy on sick leave?"
  • Supplied source: The text of the policy document (paragraphs).
  • Retrieved context: (none in this example).
  • Irrelevant information: "The weather is nice today." (user added as a note).
  • Potentially untrusted content: A quote from an anonymous blog included in the prompt.

Expected product: A classification of each item with a brief justification.

Show model analysis
  • System instruction: "You are a knowledgeable assistant…" — This is a system‑level directive that guides behaviour.
  • User instruction: "Summarise the attached policy…" — This is the specific task instruction.
  • Conversation history: The previous question about sick leave — This is part of the ongoing interaction.
  • Supplied source: The policy document — This is factual context provided by the user.
  • Retrieved context: None — no external retrieval in this example.
  • Irrelevant information: "The weather is nice today." — This is not relevant to the task and may distract the model.
  • Potentially untrusted content: The anonymous blog quote — This is not from a verified source and could contain misinformation.

Key insight: Being aware of what you include in the prompt helps you identify potential sources of error and verify the quality of the context.

Activity 2: Context Quality Audit
  • Purpose: To identify how different types of context issues can affect output.
  • Learning outcome: Identify missing, irrelevant, inaccurate, excessive, or conflicting context.
  • Estimated time: 15–20 minutes.

Instructions: You are given a synthetic context package for a task: "Generate a one‑paragraph summary of the benefits of a plant‑based diet for a health newsletter."

Context package:

  • A study from 2015 on plant‑based diets and heart health (relevant but outdated).
  • A recent (2023) meta‑analysis showing benefits for diabetes prevention (relevant, current).
  • A long, detailed article about vegan cooking techniques (mostly irrelevant to health benefits).
  • A statement: "Some experts believe that plant‑based diets may lack certain nutrients." (relevant but could be misleading without context).
  • An instruction: "Highlight the benefits, but also mention the risks."
  • A note: "The newsletter audience is health‑conscious adults." (relevant).

For each item, classify it as necessary, potentially useful, irrelevant, outdated, conflicting, or missing (if something important is absent). Then, predict how each might affect the output.

Show model analysis
  • 2015 study: Outdated. May lead to outdated recommendations.
  • 2023 meta‑analysis: Necessary and current.
  • Vegan cooking article: Irrelevant. May cause the model to include cooking tips instead of health benefits.
  • Nutrient deficiency statement: Potentially useful but could be misleading if not balanced. May cause an overemphasis on risks.
  • Instruction to mention risks: Could conflict with the main request to highlight benefits, leading to a mixed message.
  • Audience note: Useful for tailoring the tone.
  • Missing: No specific mention of environmental benefits or cost considerations, which might be expected.

Predicted effects: The model may produce a summary that includes outdated data, mentions cooking techniques, overemphasises nutrient risks, and omits environmental benefits. The human user should edit to prioritise recent evidence, remove irrelevant content, balance risks and benefits, and add missing information.

Activity 3: Context Window Simulation
  • Purpose: To practise prioritising information under a limited context budget.
  • Learning outcome: Select which information should remain available to complete a task.
  • Estimated time: 15–20 minutes.

Instructions: You have a context window of 300 tokens. You need to summarise a 500‑word article about renewable energy. You must choose which parts of the article to include in the context to get a good summary. List the sections or key points you would include, and explain why you prioritised them over others.

Article structure (simplified):

  • Introduction (100 words) — overview of renewable energy.
  • Solar energy (100 words) — benefits and challenges.
  • Wind energy (100 words) — benefits and challenges.
  • Hydropower (100 words) — benefits and challenges.
  • Geothermal (100 words) — benefits and challenges.
  • Conclusion (100 words) — future outlook.

You have 300 tokens available. Which sections would you include? Provide your reasoning.

Show model analysis and possible prioritisation

Possible prioritisation:

  • Include the introduction (overview) and conclusion (future outlook) as they frame the entire topic.
  • Include the most relevant or surprising sections (e.g., solar and wind, which are often the most discussed).
  • Omit sections that are less critical or can be summarised by the model from its own knowledge (e.g., hydropower, geothermal).

Reasoning: The model may already have general knowledge about renewable energy; providing the overview and the most important details will help it produce a balanced summary. Omitting some sections is acceptable because the model can fill in general facts, but the specific data from the article should be prioritised.

Key insight: When context is limited, you must choose what to keep based on relevance and novelty. The model can generate some information from its training data, but you want to ensure that the most important and specific details are included.

Activity 4: Conversation Drift
  • Purpose: To identify when a conversation has drifted from the original purpose.
  • Learning outcome: Diagnose conversation or context drift and identify when to restart or restate a task.
  • Estimated time: 10–15 minutes.

Instructions: Read the following multi‑turn interaction and identify the point at which the conversation drifted from the original task. Suggest what the user should do to refocus the conversation.

  • User: "Can you summarise the key findings of the IPCC report on climate change?"
  • AI: "The IPCC report highlights that global temperatures have risen by 1.1°C since pre‑industrial times, and that human activity is the primary driver."
  • User: "What about the impacts on agriculture?"
  • AI: "Agriculture is expected to face reduced yields in many regions, especially in tropical areas. Some crops may benefit from CO₂ fertilisation in the short term."
  • User: "Tell me about the political responses."
  • AI: "Political responses vary; some countries have signed the Paris Agreement and set net‑zero targets, while others have been slow to act."
  • User: "Can you give me a list of the countries that have the most ambitious targets?"
  • AI: "Countries like the UK, Germany, and Canada have set ambitious targets, but implementation is uneven."

At what point did the conversation drift from the original request (summarise the IPCC report)? What should the user do now?

Show model analysis

Drift point: The drift began at the second user question ("What about the impacts on agriculture?"). The original task was to summarise the IPCC report, but the user then asked a series of specific questions about impacts, political responses, and country targets. The AI responded to each question, but the original summary was never completed.

What the user should do: The user should restate the original request clearly, e.g., "I need a comprehensive summary of the IPCC report's key findings, covering all major aspects including physical science, impacts, and mitigation. Please provide a structured summary based on the report." The user may also need to reset the conversation or start a new session to clear the context.

Key insight: Once the conversation has drifted, the context window may be filled with unrelated details, and the model may lose track of the original request. Restarting or restating the task with a clear instruction is the best way to recover.

10. Exercises

These exercises help you apply the concepts. Each includes a self‑assessment option.

Exercise 1: Distinguish Prompts, Instructions, Context, and Output
  • Learning outcome: Distinguish these components in a given interaction.
  • Estimated time: 10 minutes.
  • Task: In the following interaction, identify the system instruction, user instruction, supplied context, and output. Also note any potential conflicts.
  • System instruction: "You are a friendly and concise assistant."
  • User: "I need a summary of the attached report on urban planning. Focus on sustainability and mention the budget."
  • Attached report: A 5‑page document with sections on sustainability, budget, community engagement, and infrastructure.
  • AI response: "The report emphasises sustainable development, with a budget of $5 million. Community engagement is also highlighted."
Show sample answer

System instruction: "You are a friendly and concise assistant."

User instruction: "I need a summary of the attached report on urban planning. Focus on sustainability and mention the budget."

Supplied context: The attached report.

Output: "The report emphasises sustainable development, with a budget of $5 million. Community engagement is also highlighted."

Potential conflicts: The user asked for a focus on sustainability and budget, but the output also mentions community engagement, which was not requested. The system instruction says "concise," but the output is somewhat brief and could be expanded.

Exercise 2: Identify Missing, Irrelevant, Inaccurate, Excessive, or Conflicting Context
  • Learning outcome: Identify context quality issues.
  • Estimated time: 10 minutes.
  • Task: For the following scenario, identify at least one example of each type of context issue: missing, irrelevant, inaccurate, excessive, and conflicting. Then suggest how you would fix the context to get a better output.

Scenario: A student asks an AI to "Explain the causes of the French Revolution." The student provides:

  • A lengthy Wikipedia article (excessive).
  • A note: "Focus on economic factors." (instruction).
  • A quote from a friend: "It was all about the Enlightenment." (inaccurate/oversimplified).
  • No mention of social inequality (missing).
  • Another instruction: "Also include the role of the monarchy." (conflicting with the focus on economics).
Show sample answer
  • Missing: Social inequality and the Estates‑General are not mentioned.
  • Irrelevant: The friend's quote is not authoritative and oversimplifies.
  • Inaccurate: The quote "It was all about the Enlightenment" is an oversimplification.
  • Excessive: The lengthy Wikipedia article may exceed the context window.
  • Conflicting: The instruction to focus on economics conflicts with the instruction to include the monarchy (which is a political factor).

Fix: Provide a concise summary of key causes (economic, social, political), omit the friend's quote, and give a clear instruction like "Explain the economic, social, and political causes of the French Revolution, with emphasis on economic factors."

Exercise 3: Prioritize Information Under a Context Limit
  • Learning outcome: Select which information to include when context is limited.
  • Estimated time: 10 minutes.
  • Task: You need to summarise a 2000‑word article about the history of the internet. Your context window is 500 tokens. Which sections would you prioritise? List the top three sections you would include and explain why.

Article sections:

  • Introduction (200 words) — overview of the internet's origins.
  • ARPANET development (300 words) — technical details.
  • Commercialisation (300 words) — growth of ISPs.
  • World Wide Web (400 words) — Tim Berners‑Lee, HTML, browsers.
  • Mobile internet (300 words) — smartphones and apps.
  • Social media (300 words) — rise of platforms.
  • Future trends (200 words) — AI, IoT, etc.
Show sample answer

Prioritised sections:

  1. Introduction: Provides context and the overall narrative.
  2. World Wide Web: A key milestone that popularised the internet.
  3. Future trends: Adds a forward‑looking perspective.

Reasoning: The introduction sets the stage, the Web section is essential because it transformed the internet, and the future trends section provides a conclusion. The other sections (ARPANET, commercialisation, mobile, social media) are important but the model may have general knowledge about them, so they can be omitted or briefly mentioned.

Alternative: If the task is to summarise the entire history, you might include ARPANET and commercialisation instead of future trends, depending on the target audience.

Exercise 4: Distinguish Supplied Context from Verified Evidence
  • Learning outcome: Explain why context is not the same as evidence.
  • Estimated time: 10 minutes.
  • Task: A user supplies a news article as context and asks the AI to "summarise the article and verify the claims." The AI produces a summary but does not verify anything. Explain why the AI cannot verify the claims, and what the user should do instead.
Show sample answer

The AI cannot verify claims because it does not have access to independent, authoritative sources during inference. It can only process the context you provide and its own training data. It cannot fact‑check or assess the truth of the article. The user should treat the article as a source and verify its claims against other trusted sources. The AI can summarise, but the user is responsible for verification.

Exercise 5: Diagnose Conversation or Context Drift
  • Learning outcome: Identify when a conversation has drifted and suggest corrective action.
  • Estimated time: 10 minutes.
  • Task: Read the following interaction and identify the drift point. Suggest a clear, concise restatement of the original task to refocus the AI.
  • User: "Help me plan a healthy weekly meal plan."
  • AI: "Sure! Here is a sample plan…" (provides a plan).
  • User: "That looks good, but I don't eat fish. Can you replace the fish dishes?"
  • AI: "Of course. I'll substitute tofu and legumes." (provides revised plan).
  • User: "What about the cost? Is this affordable?"
  • AI: "The cost varies; I can suggest cheaper alternatives if you like."
  • User: "Yes, please." (AI provides cheaper options).
  • User: "Actually, I want a plan that focuses on high‑protein, low‑carb meals."
Show sample answer

Drift point: The original task was to plan a healthy weekly meal plan. As the conversation progressed, the user added constraints (no fish, cost considerations, then high‑protein low‑carb). The AI adapted, but the original plan was lost. The final request for high‑protein low‑carb is a significant change.

Restatement: "Please create a new 7‑day meal plan that is high‑protein, low‑carb, does not include fish, and is cost‑effective. Provide a clear plan with breakfast, lunch, dinner, and snacks."

Key insight: When the task changes significantly, it's best to restate the full requirements to avoid confusion and ensure the output meets all criteria.

11. Homework Problems

Formative · Ungraded · Preparation for Lesson 2.4 and Assignment 1.

Problem 1: Prompt vs. Context
  • Learning outcome: Distinguish prompt, instructions, context, and output.
  • Estimated time: 8 minutes.
  • Task: Write a short paragraph explaining the difference between a user prompt and the context supplied in that prompt. Use an example.
Show sample answer

The user prompt is the main instruction or question you give to the AI. The context includes any additional information you provide to help the AI generate a better answer. For example, if you say, "Summarise this article," the prompt is the instruction, and the article itself is the context. The context is part of the input but is distinct from the instruction that tells the AI what to do with it.

Problem 2: Context Component Classification
  • Learning outcome: Classify different parts of an input as system instruction, user instruction, context, etc.
  • Estimated time: 10 minutes.
  • Task: You are using an AI system with the following input. Identify each component: system instruction, user instruction, supplied context, conversation history, and any irrelevant or conflicting elements.
  • System: "You are a polite and concise assistant."
  • User: "I need a list of the top 5 most populous countries, but only include those in Asia. Also, I'm in a hurry."
  • Previous turn: "What is the population of China?"
  • User also pastes a table of population data from a website (supplied context).
  • User adds: "By the way, I think India is the largest, but I'm not sure." (irrelevant/opinion).
Show sample answer
  • System instruction: "You are a polite and concise assistant."
  • User instruction: "I need a list of the top 5 most populous countries, but only include those in Asia."
  • Supplied context: The population data table.
  • Conversation history: The previous question about China's population.
  • Irrelevant: "I'm in a hurry." (not relevant to the task).
  • Potential conflict: The user's opinion about India may conflict with the data; the model should rely on the supplied data.
Problem 3: Context Quality Audit
  • Learning outcome: Identify missing, irrelevant, inaccurate, excessive, or conflicting context.
  • Estimated time: 10 minutes.
  • Task: A user asks an AI to "Write a persuasive email to convince my boss to approve a new project." The user provides: a 10‑page project proposal (excessive), a note: "The project will save money," a colleague's comment: "This will never work" (irrelevant/negative), and no budget information (missing). The user also says: "Be very persuasive." Identify the issues and suggest improvements.
Show sample answer
  • Excessive: 10‑page proposal may exceed the context window.
  • Irrelevant: The colleague's negative comment does not help.
  • Missing: No budget, timeline, or specific benefits beyond "save money."
  • Conflict: "Be persuasive" is vague; may lead to overly aggressive tone.

Improvements: Provide a concise summary of the project (1‑2 pages), include a bullet list of key benefits, budget, and timeline. Omit the colleague's comment. Give a clear instruction: "Draft a persuasive email to my boss that highlights the cost savings, efficiency gains, and aligns with our strategic goals."

Problem 4: Context Window Prioritization
  • Learning outcome: Prioritise information under a context limit.
  • Estimated time: 10 minutes.
  • Task: You need to generate a summary of a 1500‑word article about the history of the smartphone. Your context window is 400 tokens. The article has the following sections: Introduction (200 words), Early developments (300 words), iPhone launch (400 words), Android emergence (300 words), Modern smartphones (300 words). Which sections would you include? Justify your choice.
Show sample answer

Prioritise: Introduction (overview), iPhone launch (key milestone), Android emergence (competition), Modern smartphones (current state).

Reasoning: The introduction sets the stage, the iPhone launch is a pivotal event, Android emergence shows market dynamics, and modern smartphones bring the story to the present. Early developments may be less critical as the general history is widely known.

Alternative: If the focus is on technical evolution, include Early developments and iPhone, but drop Android. The choice depends on the task.

Problem 5: Excessive Context
  • Learning outcome: Explain why too much context can be problematic.
  • Estimated time: 8 minutes.
  • Task: A user pastes a 100‑page legal document into a chat and asks "What is the main liability clause?" The AI responds with a generic answer. Why might the AI's answer be poor, and what should the user do instead?
Show sample answer

The AI may have truncated the document (if it exceeds the context window) or may have been overwhelmed by the volume of text, missing the specific clause. Also, the model may not be reliable for legal analysis. The user should extract the relevant sections (e.g., the liability clause) and provide only that portion as context, or use a document‑specific search tool. The user should also verify the AI's output against the original document.

Problem 6: Missing Context
  • Learning outcome: Explain how missing context affects output.
  • Estimated time: 8 minutes.
  • Task: A user asks "What is the current exchange rate between USD and EUR?" The AI provides a rate from its training data (which may be months old). Why is this likely inaccurate, and what should the user do?
Show sample answer

The AI's training data has a cutoff date; it does not have real‑time information. The rate it provides may be outdated. The user should use a reliable source for current rates, or use a tool that can retrieve live data (if available). The user should not rely on the AI for current financial data unless it uses retrieval.

Problem 7: Conflicting Context
  • Learning outcome: Explain how conflicting context can affect output.
  • Estimated time: 10 minutes.
  • Task: A user provides two contradictory news articles about a political event and asks the AI to "summarise the situation." What challenges does this pose, and how should the user proceed?
Show sample answer

The AI may produce a summary that incorporates both perspectives, leading to confusion or contradiction. It cannot resolve the conflict without external verification. The user should instead identify the key points of disagreement, and then verify facts from authoritative sources. The user can ask the AI to highlight the discrepancies and then independently research the truth.

Problem 8: Evidence vs. Context
  • Learning outcome: Distinguish supplied context from verified evidence.
  • Estimated time: 8 minutes.
  • Task: A user says, "I'm providing a study as context. Please summarise it and tell me if its conclusions are correct." What is the problem with this request?
Show sample answer

The AI cannot determine if the study's conclusions are correct because it lacks the ability to evaluate the study's methodology or to verify its claims against other research. The AI can summarise, but the user must critically appraise the study themselves using their expertise and other sources. The AI is not a substitute for peer review or expert judgment.

Problem 9: Conversation Drift
  • Learning outcome: Diagnose conversation drift and suggest a reset.
  • Estimated time: 10 minutes.
  • Task: You are using an AI to plan a vacation. The conversation has gone off track: you asked about hotels, then restaurants, then activities, then transport. You now realise you haven't actually decided on a destination. Write a message to the AI to reset the conversation and start the planning process from scratch.
Show sample answer

Reset message: "Let's start over. I need to plan a 7‑day vacation for two adults. I have no destination in mind yet. Please help me choose a destination based on my preferences: we like beaches, moderate budget, and good food. Then we can plan accommodation, activities, and transport."

Reasoning: This restatement clearly defines the task, includes key preferences, and sets a logical order for planning.

Problem 10: Transfer — Unfamiliar Context
  • Learning outcome: Apply concepts to an unfamiliar context.
  • Estimated time: 12 minutes.
  • Task: A teacher uses an AI to generate lesson plans. The teacher provides a textbook chapter, a list of learning objectives, and a note about the class's previous knowledge. The AI produces a plan that includes activities that are too advanced. What context issue is likely at fault, and how could the teacher improve the prompt?
Show sample answer

The context may be missing specific information about the students' current level, or the learning objectives may be too vague. The AI may have relied on its training data about typical lesson plans, which may not match this class. The teacher should provide a more detailed description of the students' abilities, examples of activities they have completed, and clear constraints (e.g., "The activities should take 30 minutes and be suitable for intermediate learners"). Also, the teacher should review and adapt the generated plan.

12. Mini‑Project: Context‑to‑Output Analysis

Project Title: Context‑to‑Output Analysis
  • Purpose: To analyse how context shapes an AI output, including what is included, excluded, and how to verify.
  • Learning outcomes: Describe how prompts, context, and instructions influence generation; identify context quality issues and their effects; specify verification requirements.
  • Estimated effort: 60–90 minutes.

Instructions: Select a bounded, low‑risk task, or use the supplied task: "Generate a one‑page study guide for a university‑level introduction to psychology exam."

You may use an AI system to generate an output, or you can use supplied outputs (provided below). Then analyse the output in terms of the context you would need to provide, and what would be required for verification.

Required components:

  1. Human purpose: What is the goal of the output?
  2. Initial prompt: Write the prompt you would use (or used).
  3. Necessary context: List the information that should be included (e.g., textbook chapters, key concepts, exam format).
  4. Excluded context: What information should be left out (to avoid overloading)?
  5. Source labels: How would you indicate the sources for each piece of context?
  6. Missing information: What important information might be absent?
  7. Possible conflicting information: What could contradict the output?
  8. Context‑window priority: If the context window is limited, what would you prioritise?
  9. Plausible provisional output: Provide a sample output (generated or hypothetical).
  10. Output limitations: What are the likely weaknesses?
  11. Claims requiring verification: Which claims must be checked?
  12. Human‑review requirement: What should a human reviewer check?
  13. One context‑drift condition: How might the task drift in a longer conversation?
  14. One reason to restart or stop the interaction: When would you abandon the current context?

Supplied output (alternative):

If you are not using a live system, use this example output for analysis:

"Psychology is the scientific study of behaviour and mental processes. Key areas include cognitive, developmental, social, and clinical psychology. Memory is a complex process involving encoding, storage, and retrieval. Learning theories include classical and operant conditioning. Psychological disorders are treated using various therapies."

Deliverable: A structured analysis (text‑based) addressing all 14 components.

Privacy and academic‑integrity boundaries:
  • Use only synthetic, public, or generalised information.
  • Do not use personal, confidential, or proprietary content.
Show self‑review checklist
  • All 14 components are addressed.
  • Context quality issues are identified.
  • Verification steps are specific and practical.
  • The analysis is consistent with the lesson's concepts.
Show Hint 1: Defining purpose

Start by clarifying what the study guide should achieve: help students review key concepts, prepare for multiple‑choice questions, or for essays? This influences the content.

Show Hint 2: Prioritising context

If the context window is limited, include the textbook's table of contents, key definitions, and a list of major theories. Omit detailed case studies or examples that can be generated from general knowledge.

Show Hint 3: Verification

For a study guide, verify that all definitions are accurate, that key theorists are correctly named, and that examples match the textbook. Also check for missing topics that are on the exam syllabus.

Show solution architecture and partial example

Solution architecture:

  • Purpose: Create a review guide for the exam.
  • Prompt: "Create a one‑page study guide covering the major topics in introductory psychology: history, research methods, biological bases, learning, memory, cognition, development, social psychology, and disorders."
  • Necessary context: textbook chapter summaries, key terms, and a list of sample questions.
  • Excluded: full case studies, lengthy experiments.
  • Source labels: "According to the textbook, …"
  • Missing: specific exam format (multiple choice, essay).
  • Conflicting: different editions of textbooks may define concepts differently.
  • Priority: include definitions, key theorists, and major experiments.
  • Provisional output: (as supplied).
  • Limitations: may be too general, missing recent research.
  • Verification: check definitions against the textbook.
  • Human review: verify all key terms and ensure the guide matches the syllabus.
  • Drift: user might ask about specific disorders, leading away from the overview.
  • Stop condition: if the conversation becomes too detailed, restart with a fresh prompt.

Partial example (first few components):

Human purpose: Provide students with a concise, structured review for the exam.
Initial prompt: "Generate a one‑page study guide for introductory psychology, covering history, research methods, biological psychology, learning, memory, cognition, development, social psychology, and disorders. Use bullet points."
Necessary context: Textbook chapters on each topic, a list of key terms, and sample exam questions.
...

13. Self‑Check Quiz

These questions are ungraded and include explanatory feedback.

Question 1

What is a context window?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: The context window limits how much input (prompt, context, history) the model can process. It is measured in tokens.

Question 2

Why might a larger context window not guarantee better output?

Select one:
Show answer and explanation

Correct answer: C.

Explanation: Even with a large window, the model may not effectively use all the information (attention limitations), and larger windows can be computationally expensive. Quality and relevance of context matter more than size.

Question 3

If you provide conflicting information in the context, what might the model do?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: The model does not have a built‑in conflict resolution mechanism. It may combine conflicting pieces, or pick one at random, resulting in a muddled output.

Question 4

True or False: Information supplied in the prompt is automatically verified by the model.

Select one:
Show answer and explanation

Correct answer: False.

Explanation: The model treats the prompt and context as given; it does not fact‑check or verify. The user is responsible for ensuring the accuracy of context and for verifying the output.

Question 5

What is conversation drift?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: As the conversation adds new queries and responses, the original objective may be forgotten or diluted, leading to irrelevant or off‑topic outputs.

Question 6

If your context window is full, what might happen to older information?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Many models use a sliding window; when the limit is reached, the oldest tokens are dropped. This means that early parts of a conversation may no longer influence later outputs.

Question 7

Which of the following is an example of conflicting context?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Conflicting context occurs when the information provided is contradictory, which can confuse the model and lead to an unclear output.

Question 8

What should you do if you notice conversation drift?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Restating the task provides a fresh, unambiguous instruction and resets the context. Starting a new conversation can clear the accumulated history.

14. Assignment 1 Journal Connection

As you work through this lesson, consider these journal prompts for Assignment 1.

Journal prompts for Lesson 2.3
  1. Mental model shift: How has your understanding of prompts and context changed after learning about context windows and context quality? What did you previously assume about how an AI uses the information you provide?
  2. Context vs. evidence: Why is it important to distinguish between information you supply in a prompt and independently verified evidence? Give an example where this distinction is critical.
  3. Context drift: Describe a time (real or hypothetical) when a conversation with an AI drifted off topic. How did it happen, and how would you fix it using the techniques you learned?
  4. Question for Unit 3: Based on your understanding of prompts and context, what is one question you want Unit 3 to address about responsible AI use?

Note: Do not share personal, confidential, or proprietary information in your journal. Use general, synthetic, or public examples.

Show reflection‑planning checklist
  • I have reviewed the lesson content and my notes.
  • I have identified at least one change in my mental model about context.
  • I have articulated the distinction between context and evidence.
  • I have reflected on a conversation drift experience.
  • I have formulated at least one question for Unit 3.

15. Key Takeaways

16. Key Terms

Prompt
The user's input to the AI system, which may include instructions, questions, and context.
System instruction
A directive set by the developer or user to guide the model's behaviour (e.g., tone, role).
Task instruction
Specific instructions for what the model should do with the provided context (e.g., summarise, translate).
Supplied context
Additional information or documents provided by the user to help the model generate a better response.
Conversation history
Previous exchanges in a multi‑turn interaction that the model may use to maintain coherence.
Context window
The maximum number of tokens the model can process at one time, which includes prompt, context, and history.
Context quality
The relevance, accuracy, completeness, clarity, and consistency of the information provided as context.
Conversation drift
The gradual shift away from the original task or objective in a multi‑turn interaction.
Verified evidence
Information that has been independently confirmed by authoritative sources, as opposed to supplied context.
Truncation
The process of discarding older tokens when the context window is exceeded.
Attention
The model's mechanism for focusing on different parts of the input; it may not attend evenly to all tokens.
Restatement
Rephrasing the task or instruction to refocus the model, especially after drift.

17. Looking Ahead

In this lesson, you learned how prompts, instructions, context, and context windows shape the output of a generative AI system. You also learned to identify context quality issues and how to manage conversation drift.

In Lesson 2.4: Retrieval, Tools, Multimodal AI, and AI Systems, you will explore how AI systems can go beyond the supplied context by retrieving external information, using tools, and processing multiple modalities. You will also learn to distinguish a model from a complete AI system and ecosystem, and where human oversight fits in.

The concepts of context and context windows are foundational for understanding how retrieval‑augmented generation and tool use expand what an AI system can do — and the new risks they introduce.

Continue to Lesson 2.4: Retrieval, Tools, Multimodal AI, and AI Systems Return to Unit 2 Home

18. Sources & Further Reading

📖 Production note

Specific definitions, examples, and figures from Chapter 2 will be inserted once the textbook content is available. All unresolved items are recorded in the instructor and production notes.

👤