COMP 3270 / Unit 2 / Unit Home / Lesson 1

Lesson 2.1 · Week 2 · Reading: Chapter 2

From Training Data to Generative Models

Build a foundational mental model of how generative AI systems are created and how they operate — from training data and parameters to training, inference, and generation.

Suggested time: 90–110 minutes Opens Unit 2 Builds on Unit 1

Table of Contents

1. Overview

In Lesson 2.1, you will build a foundational mental model of how generative AI systems are created and how they operate. You will learn to distinguish artificial intelligence, machine learning, deep learning, generative models, foundation models, and large language models — and you will see why these terms are not interchangeable.

You will examine the conceptual role of training data, understand what model parameters represent, and distinguish training from inference and generation. Along the way, you will confront common misconceptions — such as the idea that a generative model is a giant database or that it “remembers” every document it was trained on.

This lesson prepares you for Lesson 2.2, where you will explore tokens, probability, and next‑token prediction. By the end, you will be able to explain why a model can produce fluent, coherent output without possessing human‑like understanding, memory, or authority.

Responsible‑use reminder: Do not enter personal, confidential, proprietary, privileged, or security‑sensitive information into any public or unapproved AI system. Use only synthetic, public, or generalised information for all activities.

2. Learning Outcomes

By the end of this lesson, you will be able to:

  1. Distinguish artificial intelligence, machine learning, deep learning, generative models, foundation models, and large language models at a conceptual level.
  2. Explain the role of training data in generative model development.
  3. Distinguish model training from inference and content generation.
  4. Explain the conceptual role of model parameters or weights.
  5. Identify limitations arising from training‑data gaps, errors, imbalances, or lack of currency.
  6. Explain why extensive training does not guarantee complete, current, or accurate knowledge.
  7. Connect technical system characteristics to selected AI opportunities and corresponding risks.
  8. Document changes in your mental model of how generative AI works.

3. Before You Begin

📖 Textbook integration note

This lesson is aligned with Chapter 2. Where specific definitions, examples, or figures from the textbook are required but not yet available, a production placeholder appears. All such items are recorded in the instructor and production notes.

4. Unit 1 Retrieval Practice

Before we move forward, activate what you learned in Unit 1. These questions are designed to refresh your memory and connect prior knowledge to today's content.

Retrieval Question 1

Identify one generative AI capability, one corresponding opportunity, and one corresponding risk.

Show suggested response and explanation

Capability: Generating human‑like text in response to a prompt.

Opportunity: Rapid drafting of emails, reports, or outlines, saving time and supporting brainstorming.

Risk: The generated text may contain factual errors, unsupported claims, or misleading phrasing, which can lead to poor decisions if not verified.

Explanation: The same capability that enables efficiency can also amplify mistakes. Fluency does not guarantee accuracy, and a confident‑sounding response can mask underlying errors.

Retrieval Question 2

In your own words, explain why fluency does not establish reliability. Provide a brief example.

Show suggested response and explanation

Explanation: A generative AI system can produce grammatically correct, coherent, and persuasive text even when the underlying claims are false, outdated, or unsupported. Fluency reflects the model's ability to produce likely language patterns, not its ability to verify facts.

Example: A model might generate a detailed summary of a historical event that includes plausible dates and names, but the summary could conflate two different events or misattribute a quote. The text reads smoothly, yet it is historically inaccurate.

Retrieval Question 3

Briefly state the AI Paradox as introduced in Unit 1.

Show suggested response and explanation

AI Paradox: Generative AI systems are remarkably capable of producing fluent, context‑sensitive, and seemingly intelligent outputs, yet they lack human understanding, intentionality, and accountability. The more capable they appear, the more important it becomes to recognise their limitations and to exercise responsible human judgment.

5. Opening Scenario

📚 The "Library" Assumption

Scenario: Alex is a graduate student in public policy who is using a generative AI system to help summarise research articles. Alex says:

"I treat the AI like a giant library. When I ask it a question, it searches through everything it has been trained on and retrieves the right answer. It's like a search engine, but it writes the answer in complete sentences."

Alex's supervisor cautions that this mental model may be misleading.

Questions for you:

  1. What assumptions does Alex appear to be making about how the AI system works?
  2. How might generative model training differ from a traditional database or library search?
  3. What questions remain unanswered about the system's operation or limitations?
Show guided analysis of the scenario

1. Assumptions Alex is making:

  • The AI stores complete documents in a searchable form.
  • Retrieval is the primary operation: the system finds and returns the best match.
  • The system's output is a faithful reproduction of a stored source.
  • If the training data is large enough, the system knows everything it was exposed to.

2. How generative model training differs from a library search:

  • A library search retrieves existing documents verbatim (or with minimal modification).
  • Generative training produces a set of numerical patterns (parameters) that encode statistical relationships, not complete documents.
  • During generation, the model produces new sequences of tokens based on learned patterns, not by copying from a stored document.
  • The model does not "know" which document a particular pattern came from; it has no internal index of sources.

3. Unanswered questions:

  • What specific data was the model trained on, and how current is it?
  • How does the model decide which patterns to apply in response to a prompt?
  • How can we verify the factual accuracy of a generated response?
  • How does the model handle information that was not present in its training data?
  • What role do prompts, context, and system instructions play in shaping the output?

Takeaway: Alex's "library" assumption is understandable but inaccurate. A generative model is not a repository of documents; it is a complex statistical system that produces language based on patterns learned from training data. This distinction is essential for responsible use.

6. Core Lesson Content

6.1 Artificial Intelligence and Related Concepts

Generative AI systems are part of a larger family of technologies. Understanding the relationships among these concepts helps you situate generative AI within the broader landscape of Computing and avoid treating all AI as the same.

[INSERT VERIFIED AI CONCEPT RELATIONSHIP DIAGRAM]
Figure 1. Simplified relationship among AI, machine learning, deep learning, generative models, foundation models, and large language models. (Placeholder — a verified diagram will be inserted in the production version.)
Table 1. Key AI concepts and their relationships
Concept Definition (conceptual level) Relationship to other concepts
Artificial Intelligence (AI) Broad field of Computing concerned with creating systems that can perform tasks that typically require human intelligence. Encompasses machine learning, but not all AI uses machine learning (e.g., rule‑based systems).
Machine Learning (ML) Subset of AI in which systems improve performance on a task through exposure to data, without being explicitly programmed for every step. ML is one approach within AI. Not all ML systems are generative.
Deep Learning (DL) Subset of ML using neural networks with many layers (deep neural networks) to learn hierarchical representations of data. DL is a method within ML. Many modern generative models use deep learning.
Generative Model A type of model that learns patterns from training data and can produce new content (text, images, audio, etc.) that resembles the training distribution. A generative model is often built using deep learning, but the term describes the purpose (generation) rather than the technique.
Foundation Model A large, general‑purpose model trained on broad data that can be adapted to many downstream tasks. Foundation models are a category of generative models (or broader) that serve as a base for further adaptation.
Large Language Model (LLM) A foundation model specifically designed to process and generate natural language, usually with billions of parameters. LLMs are a type of generative model and a type of foundation model, but not all generative models are LLMs (e.g., image generators).
Multimodal Foundation Model A foundation model that can process and generate multiple types of information, such as text and images. Extends the foundation‑model concept beyond language.
⚠️ Common misconceptions

"All AI uses machine learning."
Correction: Not all AI systems use machine learning. Early AI systems used symbolic reasoning, rule‑based logic, and expert systems without statistical learning. Machine learning is one major approach within AI, but it is not the whole field.

"All machine learning systems are generative."
Correction: Many ML systems are discriminative — they classify or predict based on input (e.g., spam detection, image recognition) rather than generating new content. Generative models are a subset of ML.

"All generative AI systems are large language models."
Correction: Generative AI includes image generators, music generators, video generators, code generators, and more. LLMs are specifically focused on language, but they are not the only type of generative system.

6.2 The Role of Training Data

Training data is the material used to teach a generative model. During training, the model processes vast amounts of examples — text, images, audio, or other forms — and learns statistical patterns. These patterns are encoded in the model's parameters (also called weights).

Key points about training data
  • Training data provides examples from which the model learns patterns, not a database to be queried directly.
  • Patterns are generalised: the model learns relationships, structures, and co‑occurrences, not verbatim copies of documents.
  • Training data differs from user‑provided context: during inference, the user supplies a prompt and possibly additional context; these are not part of the training data and do not update the model's parameters.
  • Training data influences capabilities and limitations: if the data contains gaps, errors, imbalances, or historical biases, the model's outputs will reflect those issues.
  • Training data is not normally fully disclosed: for many commercial systems, the exact composition of the training data is proprietary and not publicly available.
  • Large data does not mean complete or current knowledge: training data has a cutoff date and may not include recent events or specialised information.
📖 Textbook reference

[INSERT DEFINITION OF TRAINING DATA FROM CHAPTER 2]

[INSERT TEXTBOOK EXAMPLE ILLUSTRATING HOW TRAINING DATA SHAPES MODEL BEHAVIOUR]

6.3 Model Parameters (Weights)

Parameters (or weights) are the numerical values inside a model that are adjusted during training. They encode the patterns the model has learned from the training data. Think of them as the internal settings that determine how the model responds to inputs.

⚠️ Common misconception

"Parameters are stored sentences or conscious memories."
Correction: Parameters are numbers — usually floating‑point numbers — that represent statistical relationships. They are not human‑readable knowledge entries, nor are they explicit rules for every possible answer. A model with billions of parameters has a vast space of possible relationships, but those relationships are not stored as sentences or concepts in the human sense.

🔍 Analogy: a high‑dimensional map

Imagine a map of a city where every street, intersection, and landmark is represented by coordinates and connections. The map does not contain the actual buildings or people — it contains a representation of relationships among locations. When you ask the model a question, it "navigates" this internal map to produce a route (the response). The map is not the city itself, and the route is not a stored document; it is a path generated using the relationships encoded in the map.

Limitations of the analogy: A city map is static and human‑readable; model parameters are dynamic, high‑dimensional, and not interpretable by direct inspection. The analogy helps illustrate relationships, but it does not capture the complexity of neural networks.

6.4 Training, Inference, Generation, and Later Adaptation

It is essential to distinguish the different phases in a generative AI system's life cycle.

Table 2. Key phases in a generative model's life cycle
Phase Definition Key characteristics
Training The process through which model parameters are adjusted using extensive data and computational procedures. Requires large datasets, significant compute, and time. Parameters are updated based on the data. The model learns patterns.
Inference The process through which a trained model processes an input and produces an output. The model is frozen (parameters do not change). The model applies learned patterns to new inputs.
Generation The sequence through which the system produces content during inference. Generation is the specific act of producing output (e.g., text, image) during inference.
Later adaptation Additional steps after initial training to tailor the model for specific tasks or behaviours. Includes fine‑tuning, instruction tuning, preference optimisation, retrieval integration, system‑level instructions, and tool integration.
📌 Important distinction

Training is not the same as inference, and inference is not the same as retrieval. During training, the model learns patterns. During inference, it applies those patterns to a new input. The model's parameters are not updated during inference.

Later adaptation methods (like fine‑tuning) do change the parameters, but they are separate from the initial training process and from inference.

6.5 Why Extensive Training Does Not Guarantee Complete or Accurate Knowledge

A model may be trained on a vast corpus of data, yet it can still produce outputs that are incomplete, inaccurate, or outdated. Understanding why is essential for responsible use.

🧠 Key insight

A model's fluent output should never be taken as evidence that the information is complete, current, or accurate. Fluency reflects linguistic plausibility, not factual verification. Always treat generated content as provisional and subject to human review and verification.

📋 Core concepts recap
  • AI is the broad field; ML is a subset; deep learning is a subset of ML.
  • Generative models are designed to produce new content; foundation models are large, general‑purpose generative models; LLMs are foundation models for language.
  • Training data provides examples from which the model learns patterns.
  • Parameters encode those patterns as numerical values.
  • Training adjusts parameters; inference uses the trained model to process inputs; generation is the production of output during inference.
  • Training data limitations — gaps, errors, imbalances, and currency — affect output quality.
  • Extensive training does not guarantee complete, current, or accurate knowledge.

7. Process Walkthrough: From Training Data to Generated Output

This walkthrough traces the journey from raw training data to a generated response, highlighting the key stages and where human judgment is required.

[INSERT VERIFIED TRAINING‑TO‑INFERENCE DIAGRAM]
Figure 2. Simplified conceptual flow from training data to generated output. (Placeholder — a verified diagram will be inserted in the production version.)
Step‑by‑step conceptual flow
  1. Training data is assembled. A large collection of texts (or images, audio, etc.) is gathered. The data may come from books, websites, articles, and other sources.
  2. Training occurs. The model processes the training data. Its parameters are repeatedly adjusted to minimise the difference between its predictions and the actual data. This is a computationally intensive process that may take weeks or months.
  3. A trained model emerges. After training, the model's parameters encode statistical patterns from the training data. The model is now ready for inference.
  4. Inference begins with a user prompt. A user submits a prompt (e.g., "Explain the difference between training and inference").
  5. The model processes the prompt. Using its learned parameters, the model generates a response by predicting tokens one at a time, guided by the prompt and any additional context.
  6. A generated output is produced. The model returns a sequence of tokens — a response that is shaped by the training data, the parameters, the prompt, and the system's settings.
  7. A human reviews the output. The output should be evaluated for accuracy, relevance, completeness, and appropriateness. The human reviewer — not the model — bears responsibility for any decisions made based on the output.

Simplification note: This walkthrough simplifies many technical details (e.g., the architecture of neural networks, the specifics of the training algorithm). It is intended to build a conceptual foundation, not to serve as a complete technical specification.

Failure points: The process can fail at several stages: poor training data, inadequate training, a misleading prompt, missing context, retrieval errors, or human over‑reliance on unverified output. Each of these points is an opportunity for responsible human oversight.

8. Worked Example: A Simplified Training‑to‑Output Scenario

Scenario: Health information summary

Human purpose: A public health officer needs a preliminary summary of current guidance on hand hygiene in community settings.

AI application: A generative AI system with a large language model (LLM) is used to generate a draft summary.

Relevant model process: The LLM has been trained on a broad corpus of texts, including some health‑related documents, but its training data may not include the most recent guidance from a specific authority.

Prompt: "Summarise the key recommendations for hand hygiene in community settings, including frequency and technique."

Context: No additional context is supplied in this example (in a real setting, the officer might include a link or a recent policy document).

Generation process: The model tokenises the prompt, then predicts the next token repeatedly, drawing on patterns learned during training. The output is a sequence of tokens that forms a coherent paragraph.

Provisional output (simplified):

"Hand hygiene is an important measure to prevent the spread of infection. Recommendations typically include washing hands with soap and water for at least 20 seconds, especially after coughing, sneezing, or using the restroom. When soap and water are not available, an alcohol‑based hand sanitizer with at least 60% alcohol may be used. Hands should be dried thoroughly with a clean towel or air dryer."

Capability demonstrated: The model produces fluent, well‑structured text that is plausible and consistent with general public health messaging.

Limitation: The output does not cite a specific, authoritative source. It may be based on training data from various sources, some of which may be outdated or not applicable to the officer's jurisdiction. The model cannot indicate the date of the guidance it is summarising.

Verification need: The health officer must check the output against current, authoritative guidance from their jurisdiction (e.g., a national or regional public health agency). The output is a draft, not a final product.

Responsible human action: The officer reviews the output, verifies each claim against a trusted source, makes corrections, and adds a citation. The final version is human‑approved, not model‑generated.

Preliminary conclusion: The model can provide a useful starting point, but the output must be independently verified and contextualised by a knowledgeable human. The model is not a substitute for professional judgment.

9. Guided Learning Activities

Activity 1: AI Concept Relationship Map
  • Purpose: To visualise and internalise the relationships among AI, ML, deep learning, generative models, foundation models, and LLMs.
  • Learning outcome: Distinguish these concepts and explain their overlaps and differences.
  • Estimated time: 15–20 minutes.

Instructions: Create a text‑based or visual map that arranges and connects the following concepts: Artificial Intelligence, Machine Learning, Deep Learning, Generative Model, Foundation Model, Large Language Model, Application, System.

For each connection, add a short phrase explaining the relationship (e.g., "is a subset of," "can be built using," "is a type of").

Expected product: A concept map (text‑based or drawn) with labelled relationships.

Self‑assessment criteria:

  • All eight concepts are included.
  • Each concept is placed correctly relative to the others.
  • Relationships are labelled with accurate phrases.
  • The map shows that categories may overlap but are not interchangeable.
Show model analysis and suggested structure

Suggested relationship structure:

  • Artificial Intelligence — the broadest category.
  • Machine Learning — a subset of AI.
  • Deep Learning — a subset of ML.
  • Generative Model — a type of model that can be built using deep learning (or other approaches).
  • Foundation Model — a large, general‑purpose generative model (or broader).
  • Large Language Model — a type of foundation model focused on language.
  • Application — a user‑facing product that uses one or more models.
  • System — a complete operational arrangement that includes models, applications, interfaces, data, tools, and human roles.

Key insight: These categories are not a strict ladder. For example, not all generative models are foundation models, and not all foundation models are LLMs. The map should show overlap without implying that every item is a subset of the next.

Activity 2: Training, Inference, or Retrieval?
  • Purpose: To distinguish among training, inference, generation, retrieval, and later adaptation.
  • Learning outcome: Classify activities as training, inference, generation, retrieval, later adaptation, or insufficient information.
  • Estimated time: 10–15 minutes.

Instructions: For each scenario, indicate which phase(s) are occurring. Choose from: model training, inference, generation, retrieval, later adaptation, or insufficient information.

  1. A model processes a user's question and produces a response.
  2. A developer adjusts the model's parameters using a new dataset of customer service transcripts.
  3. A system searches a company policy database and returns relevant passages to include in the model's context.
  4. A model produces a draft email after being given a prompt and a list of key points.
  5. A team uses human feedback to fine‑tune a model so that it provides more concise answers.
  6. A user pastes a long article into the chat and asks the model to summarise it.
Show answers and explanations
  1. Inference and generation. The model is processing a new input (inference) and producing content (generation). Parameters are not updated.
  2. Later adaptation (fine‑tuning). The parameters are being adjusted using new data, which is a form of continued training or adaptation.
  3. Retrieval. The system is searching an external source and returning passages. The model itself is not generating new content at this stage.
  4. Inference and generation. The model is processing the prompt and key points (inference) and producing new text (generation).
  5. Later adaptation (fine‑tuning / preference optimisation). Human feedback is being used to adjust the model's behaviour.
  6. Inference and generation. The user supplies context (the article) as part of the prompt; the model processes it (inference) and produces a summary (generation). This is not retrieval because the context is supplied directly, not retrieved from an external source.

Note: Some scenarios may involve multiple phases. For example, a system might retrieve information, then use the model to generate a response. The key is to identify the primary activity described.

Activity 3: Training‑Data Assumption Audit
  • Purpose: To identify and correct common misconceptions about training data.
  • Learning outcome: Classify statements as broadly supportable, misleading, false, or context dependent.
  • Estimated time: 10–15 minutes.

Instructions: For each statement, classify it and provide a brief explanation.

  1. "The model remembers every document in its training data."
  2. "A larger data set guarantees accurate answers."
  3. "Training data and prompt context are the same thing."
  4. "If the model was trained recently, every answer is current."
  5. "Training patterns can influence both capability and bias."
Show answers and explanations
  1. False. The model does not store documents; it stores patterns in numerical parameters. It cannot reproduce a document verbatim in most cases, and it does not have a "memory" of individual documents.
  2. Misleading / context dependent. A larger dataset can improve coverage and reduce overfitting, but it does not guarantee accuracy. Data quality, relevance, and currency matter more than sheer size.
  3. False. Training data is used to create the model; prompt context is supplied by the user during inference. They are fundamentally different in purpose and timing.
  4. False. A recent training cutoff does not mean the model has current information. The model is not automatically updated; it reflects the state of the data at the time of training.
  5. Broadly supportable. The patterns learned from training data influence both the model's capabilities (e.g., fluency, reasoning) and its biases (e.g., demographic skew, cultural assumptions). This is a well‑documented characteristic of generative models.
Activity 4: Explaining Parameters Without Stored Documents
  • Purpose: To articulate what model parameters represent in a way that avoids the "stored document" misconception.
  • Learning outcome: Explain the conceptual role of model parameters using an accessible analogy while acknowledging its limitations.
  • Estimated time: 10–15 minutes.

Instructions: Write a short explanation (1–2 paragraphs) that:

  • Describes what model parameters (weights) are;
  • Uses an analogy to make the concept accessible;
  • Explains the limitation(s) of the analogy;
  • Avoids describing parameters as stored sentences, conscious memories, or explicit rules.
Show a model explanation

Model explanation:

Model parameters are numerical values — think of them as the internal settings of a complex system. Imagine a very large, multi‑dimensional map of relationships among words, phrases, and ideas. Each parameter is like a coordinate or a connection strength on that map. When you ask a question, the model uses these coordinates to navigate the map and find a path that corresponds to a plausible response.

Analogy: The map helps us see that the model is not storing complete documents; it is storing relationships. The map is not a library — it is a representation of how concepts are connected.

Limitation: The map analogy makes the model seem static and interpretable, but real model parameters are high‑dimensional, dynamic, and not directly readable by humans. The map is a simplified illustration, not an exact description of the model's internal workings.

10. Exercises

These exercises help you apply the concepts from the lesson. Each includes a self‑assessment option.

Exercise 1: Concept Hierarchy
  • Learning outcome: Distinguish AI, ML, deep learning, generative models, and LLMs.
  • Estimated time: 10 minutes.
  • Task: Construct a concept hierarchy that connects the following terms. Use phrases such as "is a subset of," "is a type of," or "can be built with." Include at least one example for each term: Artificial Intelligence, Machine Learning, Deep Learning, Generative Model, Large Language Model.
Show sample solution

Sample hierarchy:

  • Artificial Intelligence (broad field)
    • Machine Learning (subset of AI; systems that learn from data)
      • Deep Learning (subset of ML; uses neural networks with many layers)
        • Generative Model (type of model; produces new content)
          • Large Language Model (type of generative model; focused on language)

Examples:

  • AI: a chess‑playing program (may use search and rules, not necessarily ML).
  • ML: a spam filter that learns from labelled email data.
  • Deep Learning: a neural network with many layers that classifies images.
  • Generative Model: a system that generates poetry from a prompt.
  • LLM: a model like the one powering a conversational AI assistant.

Note: The hierarchy shows a common relationship, but it is not a strict ladder. For example, not all generative models use deep learning, and not all ML systems are generative.

Exercise 2: Training vs. Inference
  • Learning outcome: Distinguish training, inference, generation, retrieval, and later adaptation.
  • Estimated time: 10 minutes.
  • Task: For each scenario, classify the primary activity as training, inference, generation, retrieval, or later adaptation. Provide a one‑sentence justification.
  1. A developer runs a script that updates the model's parameters using a new corpus of financial reports.
  2. A user enters a prompt and the model produces a draft business letter.
  3. A system queries a customer database and returns a list of recent transactions to include in the model's context.
  4. A team uses a reinforcement learning method to adjust the model's output preferences based on human ratings.
  5. A model processes a multi‑turn conversation and generates a closing remark.
Show answers and explanations
  1. Later adaptation (fine‑tuning). The parameters are being updated using new data after initial training.
  2. Inference and generation. The model processes the prompt (inference) and produces output (generation).
  3. Retrieval. The system is searching and returning external information, not generating new content.
  4. Later adaptation (preference optimisation). The model's behaviour is being adjusted based on human feedback.
  5. Inference and generation. The model processes the conversation history (inference) and produces a response (generation).
Exercise 3: Explaining Parameters
  • Learning outcome: Explain the conceptual role of model parameters without describing them as stored documents.
  • Estimated time: 10–15 minutes.
  • Task: Write a short paragraph (5–7 sentences) that explains what model parameters are and what they do. Avoid the following phrases: "stored sentences," "conscious memories," "explicit rules," "complete knowledge." Use an analogy if helpful, but explain its limitations.
Show a model response

Model parameters are numerical values inside a neural network that encode patterns learned from training data. They are like the settings on a complex instrument — they determine how the instrument responds to different inputs. When a model receives a prompt, the parameters influence the sequence of operations that produce the output.

A useful analogy is a high‑dimensional map: the parameters are the coordinates and connection strengths that define the map. The map itself is not a collection of stored documents; it is a representation of relationships among concepts. The model "navigates" this map to generate a response.

The analogy has limitations: a map is static and interpretable, but model parameters are dynamic, high‑dimensional, and not directly readable. The map helps us understand relationships, but it does not capture the full complexity of the model.

Exercise 4: Training‑Data Limitations
  • Learning outcome: Identify limitations arising from training‑data gaps, errors, imbalances, or lack of currency.
  • Estimated time: 10–15 minutes.
  • Task: Read the following scenarios and identify the training‑data limitation that is most relevant. Suggest one way a responsible human user could mitigate the risk.
  1. A model is asked to summarise the latest research on a rapidly evolving disease, but its training data is two years old.
  2. A model generates job descriptions that consistently describe leadership roles using male‑associated language, reflecting historical patterns in the training data.
  3. A model is asked to explain a specialised legal concept, but its training data contains only general‑purpose texts and few legal documents.
  4. A model produces a confident‑sounding answer that appears to combine facts from two different sources in a way that creates a false claim.
Show suggested answers and mitigation strategies
  1. Lack of currency. The model's knowledge is outdated. Mitigation: Verify the output against current, authoritative sources; use retrieval‑augmented generation to bring in recent data; treat the output as a starting point, not a final answer.
  2. Data imbalance / historical bias. The training data reflects historical gender biases. Mitigation: Review and revise generated language; use inclusive language guidelines; involve human reviewers with diverse perspectives.
  3. Data gap. The model lacks sufficient domain‑specific training data. Mitigation: Consult a human expert; use retrieval from specialised sources; avoid relying on the model for high‑stakes legal decisions.
  4. Data error / pattern mis‑generalisation. The model has combined patterns in a way that produces a false claim. Mitigation: Fact‑check all claims; trace claims to sources where possible; treat the output as provisional.

11. Homework Problems

Formative · Ungraded · Recommended preparation for Lesson 2.2, the Unit 2 cumulative activities, and Assignment 1.

Problem 1: Concept Hierarchy
  • Learning outcome: Distinguish AI, ML, deep learning, generative models, foundation models, and LLMs.
  • Estimated time: 10 minutes.
  • Task: Draw or write a concept hierarchy that includes all six terms. For each term, provide a one‑sentence definition and a brief example. Indicate at least two non‑interchangeability relationships (e.g., "not all X are Y").
Show solution design and hints

Solution design:

  • Start with AI at the top, then branch to ML and other approaches.
  • Under ML, place deep learning.
  • Under deep learning (or alongside it), place generative models.
  • Under generative models, place foundation models.
  • Under foundation models, place LLMs.
  • Add at least two "not all" statements (e.g., "not all generative models are foundation models").

Hint 1: Consider the breadth of each category. AI is the broadest; LLMs are quite specific.

Hint 2: A generative model can exist without being a foundation model (e.g., a smaller image generator).

Problem 2: Training vs. Inference
  • Learning outcome: Distinguish model training from inference and generation.
  • Estimated time: 10 minutes.
  • Task: A newspaper article describes an AI system that "learned from millions of articles and now answers questions." In a short paragraph, explain why this statement conflates training and inference. What would be a more accurate way to describe the system?
Show sample answer

The statement conflates training and inference by implying that the system is continuously learning from its interactions. In reality, the system was trained on a large dataset of articles during its development phase, which produced a trained model. When the system answers questions, it is performing inference — applying the patterns it learned during training to new inputs. The model's parameters are not updated during these interactions.

A more accurate description would be: "The system was trained on millions of articles and can now answer questions by applying the patterns it learned during that training."

Problem 3: Parameters Explanation
  • Learning outcome: Explain the conceptual role of model parameters.
  • Estimated time: 10–15 minutes.
  • Task: Write a dialogue between two students: one who believes that model parameters are "stored facts" and another who corrects this misconception. The dialogue should be 6–8 exchanges and should use an analogy or concrete explanation.
Show solution design and hints

Solution design:

  • Student A states a misconception (e.g., "The model has a database of facts").
  • Student B corrects with a concise explanation of parameters as numerical patterns.
  • Student B uses an analogy (e.g., a map, a recipe, a language model).
  • Student A asks a clarifying question.
  • Student B explains the limitation of the analogy.

Hint 1: The dialogue should show the difference between a database (stored documents) and a neural network (encoded patterns).

Hint 2: Mention that parameters are not human‑readable and do not represent individual facts.

Problem 4: Training‑Data Misconceptions
  • Learning outcome: Identify and correct common misconceptions about training data.
  • Estimated time: 10 minutes.
  • Task: Evaluate the following claim: "If the model was trained on the entire internet, it must be able to answer any question correctly." Explain why this claim is misleading. Include at least three distinct reasons.
Show sample answer

The claim is misleading for several reasons:

  1. Data quality: The internet contains contradictory, outdated, and incorrect information. The model learns patterns from all of it, including errors and biases.
  2. Data representation: The model does not store the entire internet; it stores statistical patterns in parameters. It cannot "search" the internet during inference.
  3. Data currency: Training data has a cutoff date; the model does not know about events that occurred after that date.
  4. Generalisation limits: Even with broad training, the model may not generalise well to specialised or novel topics that were poorly represented in the data.
  5. No source memory: The model cannot tell you which document a particular pattern came from, making verification difficult.
Problem 5: Data Quality
  • Learning outcome: Identify limitations arising from training‑data quality issues.
  • Estimated time: 10 minutes.
  • Task: A company uses a generative AI model to screen job applications. The model was trained on historical hiring data, which reflects past hiring decisions that favoured certain demographic groups. Identify at least two ways this training‑data imbalance could affect the model's outputs and propose one mitigation strategy.
Show sample answer

Effects:

  • The model may generate language in job descriptions that subtly encourages or discourages certain groups from applying.
  • The model may rank or summarise applications in ways that reflect historical biases, disadvantaging candidates from underrepresented groups.

Mitigation strategy:

  • Audit the training data for demographic imbalances and, where possible, balance the dataset or use bias‑mitigation techniques.
  • Review all model‑generated outputs for fairness before they are used in decision‑making.
  • Involve human reviewers with diverse perspectives in the evaluation process.
  • Use the model only as a draft generator, not as a final decision‑maker.
Problem 6: Currentness
  • Learning outcome: Explain why a model's knowledge may be outdated.
  • Estimated time: 8 minutes.
  • Task: A student asks a generative AI model about the current prime minister of a country. The model provides an answer that was correct two years ago but is now outdated. The student is frustrated. Write a short paragraph explaining to the student what happened and what the student should do next.
Show sample answer

The model's training data has a cutoff date, which means the model only knows about events up to that point. If the prime minister changed after the training data was collected, the model cannot know about the change because it has not been updated. The model does not automatically access current information.

To get an accurate and current answer, the student should verify the information using a reliable, up‑to‑date source — such as an official government website or a reputable news outlet. The model's output is a draft that requires human verification.

Problem 7: Contextual Application
  • Learning outcome: Apply technical understanding to a practical context.
  • Estimated time: 12 minutes.
  • Task: A healthcare administrator wants to use a generative AI system to draft responses to patient inquiries. The administrator assumes that because the system was trained on a large corpus of medical literature, it can provide reliable health information. Identify three technical limitations that the administrator should consider, and suggest one human oversight practice to address each.
Show sample answer

Limitations:

  1. Data currency: Medical knowledge evolves rapidly; the model may be using outdated information.
  2. Data completeness: The model may not have been trained on the specific policies, guidelines, or protocols of the administrator's institution.
  3. No source memory: The model cannot cite the specific sources it used, making it difficult to verify the reliability of its outputs.

Oversight practices:

  • For currency: Implement a human review step where a qualified professional checks the output against current, authoritative guidelines.
  • For completeness: Use retrieval‑augmented generation to bring in institutional policies, and have a human reviewer confirm alignment.
  • For source memory: Train staff to treat all generated content as provisional and to verify claims against trusted sources before using them.
Problem 8: Opportunity‑Risk Connection
  • Learning outcome: Connect technical system characteristics to opportunities and risks.
  • Estimated time: 10 minutes.
  • Task: Select one technical characteristic of generative models (e.g., pattern learning from data, probabilistic generation, parameter‑based representation). Identify one opportunity and one corresponding risk that arises from that characteristic. Explain why the risk is not eliminated simply by having a large training dataset.
Show sample answer

Characteristic: Pattern learning from training data.

Opportunity: The model can generalise across a wide range of topics, providing helpful summaries, drafts, and ideas across many domains.

Corresponding risk: The model may reproduce and amplify biases, stereotypes, or factual errors present in the training data.

Why a large dataset does not eliminate the risk: A larger dataset does not guarantee that the data is balanced, accurate, or representative. In fact, a larger dataset may include more examples of the same biases, reinforcing them. The model learns patterns from whatever is in the data, regardless of whether those patterns are desirable or accurate. Human oversight and verification remain essential.

Problem 9: Transfer
  • Learning outcome: Apply concepts from the lesson to an unfamiliar context.
  • Estimated time: 12 minutes.
  • Task: A city council is considering using a generative AI system to draft public‑facing communications about new policies. The system was trained on a general corpus of English‑language texts. A councillor asks: "Since the system was trained on so much data, it should be able to write perfectly for our city's audience, right?" Respond to the councillor in a way that addresses training‑data limitations, the training‑inference distinction, and the need for human oversight. Write your response as a short memo.
Show sample answer and solution design

Sample memo:

To: City Council
From: [Your name]
Subject: Considerations for using generative AI for public communications

Thank you for the question. While the system was trained on a large corpus of English‑language texts, there are several important limitations to consider:

  • Training data does not equal audience knowledge: The system was not trained on our city's specific policies, tone, or community values. It may produce language that is generic or misaligned with our communication standards.
  • Training is not inference: The system was trained in the past; it does not automatically update its knowledge or adapt to our city's current needs.
  • Human oversight is essential: All generated communications should be reviewed and approved by a qualified human before they are published. The system can provide a draft, but it cannot take responsibility for accuracy, tone, or appropriateness.

I recommend that we use the system as a drafting aid only, with clear human‑review processes in place. We should also consider using retrieval to bring in our own policy documents and train our staff to verify all outputs.

12. Mini‑Project: Generative Model Concept Map

Project Title: Generative Model Concept Map
  • Purpose: To synthesise the key concepts from Lesson 2.1 into a visual or text‑based concept map that shows relationships, misconceptions, and implications.
  • Learning outcomes: Define and relate the key terms from this lesson; identify common misconceptions and correct them; connect technical characteristics to opportunities and risks.
  • Estimated effort: 60–90 minutes.

Context choices: Select one context for your concept map:

  • A learning context (e.g., a student using AI for study support).
  • A professional context (e.g., a communicator drafting content).
  • A creative context (e.g., an artist exploring AI‑generated images).
  • An everyday context (e.g., a person using a conversational AI).

Required components:

  1. Artificial Intelligence — one‑sentence definition
  2. Machine Learning — one‑sentence definition
  3. Deep Learning — one‑sentence definition
  4. Neural‑network model — one‑sentence definition
  5. Generative Model — one‑sentence definition
  6. Foundation Model — one‑sentence definition
  7. Large Language Model — one‑sentence definition
  8. Training Data — one‑sentence definition
  9. Parameters — one‑sentence definition
  10. Inference — one‑sentence definition
  11. Prompt — one‑sentence definition
  12. Generated Output — one‑sentence definition
  13. Human Reviewer — one‑sentence definition

Plus: Add labelled relationships between these concepts, two common misconceptions with corrections, one opportunity and one corresponding risk connected to a technical characteristic, and one question for later study.

Deliverable: A text‑based or visual concept map. If text‑based, use indentation or bulleted lists to show relationships. Include all 13 terms and the additional elements.

Privacy and academic‑integrity boundaries:
  • Use only synthetic, public, or generalised information.
  • Do not use personal, confidential, or proprietary content.
  • If you use an AI tool to assist, document your use according to course policy.
Show self‑review checklist
  • All 13 terms are included.
  • Each term has a clear, concise definition.
  • Relationships are explicitly labelled.
  • Two misconceptions are identified and corrected.
  • One opportunity and one risk are connected to a technical characteristic.
  • One question for later study is included.
  • The map reflects that categories may overlap but are not interchangeable.
Show Hint 1: Organising the map

Start with a broad category (AI) and branch out to more specific concepts. Group related terms (e.g., training data, parameters, inference) together. Use arrows or indentation to show relationships such as "is a subset of," "informs," or "produces."

Show Hint 2: Misconceptions to address

Common misconceptions include: "All AI uses machine learning," "Parameters are stored documents," and "Training data is the same as context." Choose two and explain why they are incorrect.

Show Hint 3: Opportunity‑risk connection

Choose a technical characteristic such as "pattern learning from data" or "probabilistic generation." Then identify an opportunity (e.g., generalisation across domains) and a risk (e.g., reproducing biases). Explain the connection.

Show solution architecture and partial example

Solution architecture:

  • Use a hierarchical structure with AI at the top, then branch to ML, then DL, then generative models, then foundation models, then LLMs.
  • Add a separate cluster for the inference process: prompt → model (with parameters) → generated output → human reviewer.
  • Link training data and parameters with a relationship label: "Training data informs parameter adjustment."
  • Add two misconception boxes with corrections.
  • Add a callout for an opportunity‑risk pair.
  • End with a "Question for later study" box.

Partial example (text‑based outline):

Artificial Intelligence
└── Machine Learning (subset of AI)
    └── Deep Learning (subset of ML, uses neural networks)
        └── Generative Model (type of model; produces new content)
            └── Foundation Model (large, general‑purpose generative model)
                └── Large Language Model (foundation model for language)
[Other cluster: Inference process]
Prompt → Model (Parameters) → Generated Output → Human Reviewer
[Training Data] → [informs] → [Parameters]
[Misconception 1: "All AI uses ML." → Correction: …]
[Misconception 2: "Parameters are stored facts." → Correction: …]
[Opportunity: …] [Risk: …]
[Question for later study: …]

Assessment criteria:

  • Completeness: all required components are present.
  • Accuracy: definitions and relationships are conceptually correct.
  • Clarity: the map is easy to read and understand.
  • Reflection: misconceptions are thoughtfully addressed; the opportunity‑risk pair is well‑chosen.

13. Self‑Check Quiz

These questions are ungraded and include explanatory feedback.

Question 1

Which of the following is not a subset of machine learning?

Select one:
Show answer and explanation

Correct answer: C. Rule‑based systems.

Explanation: Rule‑based systems are a traditional approach to AI that does not use machine learning. They rely on explicit rules encoded by humans, rather than learning patterns from data. Deep learning, generative models, and neural networks are all associated with machine learning.

Question 2

What is the primary purpose of training a generative model?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Training is the process of adjusting the model's parameters based on the training data so that the model learns statistical patterns. The model does not store documents or create a searchable index; it encodes patterns numerically.

Question 3

Which statement about model parameters is most accurate?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Parameters (weights) are numerical values inside a neural network that encode statistical patterns learned from training data. They are not human‑readable, not explicit rules, and not the training data itself.

Question 4

During inference, the model's parameters are:

Select one:
Show answer and explanation

Correct answer: B.

Explanation: During inference, the model's parameters are not updated. The model applies the learned patterns (encoded in the parameters) to the new input. Training is a separate phase where parameters are adjusted.

Question 5

A large language model is best described as:

Select one:
Show answer and explanation

Correct answer: A.

Explanation: A large language model is a type of foundation model that is specifically designed to process and generate natural language. It is not a search engine, a rule‑based system, or a database of sentences.

Question 6

Which of the following is not a limitation that can arise from training data?

Select one:
Show answer and explanation

Correct answer: C.

Explanation: Training data does not enable perfect accuracy. Gaps, errors, imbalances, and currency issues all limit the model's performance. Perfect accuracy is not a characteristic of training data or any generative model.

Question 7

True or False: A generative model can learn patterns from training data that reflect historical biases.

Select one:
Show answer and explanation

Correct answer: True.

Explanation: Generative models learn patterns from their training data, including any biases, stereotypes, or historical patterns present in that data. This is a well‑documented limitation that requires human oversight and mitigation.

Question 8

What is the most appropriate response to a model's fluent but unverified output?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Fluency does not establish reliability. All generated content should be verified against authoritative sources before it is used, shared, or acted upon. The human user bears responsibility for accuracy.

14. Assignment 1 Journal Connection

Unit 2 contributes to Assignment 1: AI Learning and Reflection Journal. As you work through this lesson, consider the following journal prompts. You may wish to record your responses for later use in your journal.

Journal prompts for Lesson 2.1
  1. Mental model shift: Before this lesson, how did you imagine a generative AI system worked? What was your mental model? How has it changed after learning about training data, parameters, and the training‑inference distinction?
  2. Training‑data assumptions: What did you previously assume about the training data used for generative AI systems? Which of those assumptions have been corrected or refined?
  3. Technical characteristic and risk: Identify one technical characteristic of generative models (e.g., pattern learning from data) and explain one opportunity and one corresponding risk. How does this connection affect your trust in AI‑generated content?
  4. Question for Unit 3: Based on what you have learned about how models are built and operate, what is one question you want Unit 3 to address about responsible and effective AI use?

Note: Do not share personal, confidential, or proprietary information in your journal. Use general, synthetic, or public examples.

Show reflection‑planning checklist
  • I have reviewed the lesson content and my notes.
  • I have identified at least one change in my mental model.
  • I have documented a specific assumption I held and how it has been revised.
  • I have articulated an opportunity‑risk pair connected to a technical characteristic.
  • I have formulated at least one question for Unit 3.
  • My reflection is based on general, synthetic, or public information.

15. Key Takeaways

16. Key Terms

Artificial Intelligence (AI)
The broad field of Computing concerned with creating systems that can perform tasks that typically require human intelligence.
Machine Learning (ML)
A subset of AI in which systems improve performance on a task through exposure to data, without being explicitly programmed for every step.
Deep Learning (DL)
A subset of ML using neural networks with many layers to learn hierarchical representations of data.
Generative Model
A model that learns patterns from training data and can produce new content that resembles the training distribution.
Foundation Model
A large, general‑purpose model trained on broad data that can be adapted to many downstream tasks.
Large Language Model (LLM)
A foundation model specifically designed to process and generate natural language, usually with billions of parameters.
Training Data
The examples used to adjust a model's parameters during the training process.
Parameters (Weights)
Numerical values inside a model that encode patterns learned from training data.
Training
The process through which model parameters are adjusted using extensive data and computational procedures.
Inference
The process through which a trained model processes an input and produces an output.
Generation
The sequence through which the system produces content during inference.
Later Adaptation
Additional steps after initial training to tailor a model for specific tasks or behaviours (e.g., fine‑tuning, instruction tuning).
Prompt
The user's input or instruction provided to the model during inference.
Retrieval
The process of searching an external source and returning relevant material for inclusion in the model's context.

17. Looking Ahead

In this lesson, you built a conceptual foundation for understanding how generative AI systems are created and how they operate. You learned to distinguish training from inference, to explain the role of parameters, and to identify limitations that arise from training data.

In Lesson 2.2: Tokens, Probability, and Content Generation, you will explore how models process text as tokens, how next‑token prediction works, and why probability is not the same as truth. You will also investigate why the same prompt can produce different outputs and why fluent language does not guarantee factual accuracy.

As you move forward, keep in mind that the model's training and its inference‑time behaviour are distinct but connected. The patterns learned during training shape every interaction, but the model does not "learn" from those interactions unless it is explicitly adapted.

Continue to Lesson 2.2: Tokens, Probability, and Content Generation Return to Unit 2 Home

18. Sources & Further Reading

📖 Production note

Specific definitions, examples, and figures from Chapter 2 will be inserted once the textbook content is available. All unresolved items are recorded in the instructor and production notes.

👤