COMP 3270 / Module 1 / Unit 1 / Lesson 2

Lesson 1.2 · Week 1 · Reading: Chapter 1

Capabilities, Opportunities, and Corresponding Risks

Examine the specific capabilities of generative AI systems, the opportunities they create, and the corresponding risks and limitations that accompany those capabilities.

Suggested time: 3–4 hours Continues Unit 1 Builds on Lesson 1.1

Table of Contents

1. Lesson Overview

In Lesson 1.1, you explored what generative AI is, how it differs from non‑generative AI, and where it appears in contemporary life. In this lesson, you will go deeper: you will examine the specific capabilities of generative AI systems, the opportunities they create, and the corresponding risks and limitations that accompany those capabilities.

You will learn why a system that can produce fluent, detailed, and professional‑looking content may still produce unreliable, biased, or harmful outputs. You will explore the distinction between technical capability (what a system can do) and contextual suitability (whether it should be used for a particular purpose). You will also consider who benefits from AI use and who bears the burdens of verification, risk, and consequences.

This lesson introduces the AI Paradox at a practical level: every significant AI opportunity may be accompanied by a corresponding risk. You will practice identifying these pairs and begin to think about safeguards and human oversight — topics that will be developed fully in Lesson 1.3 and Unit 3.

In this lesson you will:
  • Identify major capabilities of generative AI systems.
  • Describe opportunities associated with those capabilities.
  • Identify corresponding risks and limitations.
  • Distinguish technical capability from contextual suitability.
  • Explain why fluency, confidence, and formatting do not establish reliability.
  • Identify beneficiaries and burden bearers in AI‑supported activities.
  • Practice opportunity‑risk mapping in different contexts.

2. Lesson Learning Outcomes

After completing this lesson, you will be able to:

  1. Describe major capabilities of generative AI systems.
  2. Identify opportunities associated with specific capabilities.
  3. Identify corresponding risks and limitations.
  4. Explain why technical capability does not establish contextual suitability.
  5. Explain why fluency, confidence, detail, and professional formatting do not establish factual reliability.
  6. Identify beneficiaries and possible burden bearers in an AI‑supported activity.
  7. Create opportunity‑risk pairs for different capabilities and contexts.
  8. Distinguish between demonstrated capability and unsupported capability claims.

3. Before You Begin

Responsible‑use reminder: Do not enter personal, confidential, proprietary, privileged, or security‑sensitive information into any public or unapproved generative AI system. Use only synthetic, public, or generalised information for all activities.

4. Opening Scenario

Scenario: A polished document with hidden problems

Dr. Chen is a public health researcher preparing a report on community vaccination rates. She is under a tight deadline and decides to use a generative AI tool to help draft the report. She inputs her data and a prompt requesting a summary of findings, recommendations, and a call to action.

The AI generates a polished, 10‑page report with:

  • Clear headings and professional formatting.
  • Confident language and technical vocabulary.
  • Detailed tables and charts (synthesised from her data).
  • References to "peer‑reviewed studies" that appear credible.
  • Specific recommendations with a sense of urgency.

Dr. Chen is impressed. The report looks complete and authoritative. However, as she reviews it more carefully, she notices:

  • Some of the "cited" studies do not actually exist.
  • One of the recommendations is based on an assumption that is not supported by her data.
  • The tone is alarmist in a way that may undermine trust in the public health message.
  • Some of the numerical precision in the tables gives an illusion of accuracy, but the underlying calculations are questionable.

Questions to consider:

  1. What capabilities of generative AI are evident in this scenario?
  2. What opportunities did Dr. Chen gain by using the AI?
  3. What risks or limitations are present?
  4. Why might Dr. Chen have been inclined to trust the report initially?
  5. What human review is required before the report can be used?
Show guided analysis of the scenario

Capabilities demonstrated:

  • Rapid generation of a long, structured document.
  • Reformatting and presentation of data in tables.
  • Generation of recommendations and a call to action.
  • Adaptation of tone and format to a professional style.
  • Simulation of "authoritative" content (including fabricated citations).

Opportunities:

  • Time savings on drafting and formatting.
  • A starting point that Dr. Chen can refine.
  • Generation of ideas and structure.

Risks and limitations:

  • Fabrication: The AI generated false references.
  • Unsupported assumptions: The recommendation was not grounded in evidence.
  • Misleading precision: Numerical tables appeared accurate but were not verified.
  • Tone mismatch: The alarmist tone could undermine the message.
  • Hidden verification work: Dr. Chen must check everything.

Why she might have trusted it initially:

  • The report was fluent and well‑formatted.
  • It used technical vocabulary and confident language.
  • It included details (tables, citations) that increased perceived credibility.
  • She was under time pressure and may have wanted to believe the output was ready.

Human review required:

  • Verify all cited sources.
  • Check recommendations against the original data.
  • Assess tone and appropriateness for the audience.
  • Validate all numerical calculations.
  • Ensure the report is accurate, balanced, and complete.

Key lesson: Fluency, structure, and confidence do not establish reliability. Dr. Chen must apply rigorous human judgment and verification.

5. Major Capabilities of Generative AI

Generative AI systems exhibit a wide range of capabilities. These capabilities vary according to the system, version, task, context, information, language, user, and evaluation method. A capability that is demonstrated in one setting may not be reliable in another.

Major capabilities
  • Rapid generation: Producing large volumes of text, images, or other content in seconds.
  • Reformatting: Changing the format of content (e.g., bullet points to paragraphs, prose to a table).
  • Summarisation: Condensing longer texts while preserving key information.
  • Translation: Converting text from one language to another.
  • Classification: Assigning categories or labels to content (e.g., sentiment, topic).
  • Comparison: Identifying similarities and differences between items.
  • Ideation: Generating new ideas, hypotheses, or creative concepts.
  • Question generation: Producing questions (e.g., study questions, interview questions).
  • Pattern‑based transformation: Applying learned patterns to transform input (e.g., rewriting text in a different style).
  • Simulation: Creating scenarios, dialogues, or models.
  • Multimodal generation: Producing content across multiple modalities (text + images, text + audio, etc.).
  • Adaptation of tone and format: Adjusting the style, tone, or structure of output to suit a particular audience or purpose.

These capabilities are not guaranteed to be present in every system, nor are they consistently reliable. For example, a system that can generate a summary in one context may produce a misleading summary in another. A system that can translate between English and French may perform poorly between English and a less‑resourced language.

Example: Variation in capability

A generative AI system may be able to:

  • Generate a convincing essay on a well‑known topic (capability).
  • Generate a convincing essay on a niche topic with less training data (limited capability).
  • Generate an essay in English (capability).
  • Generate an essay in a language with limited training data (limited capability).

The same system can demonstrate different levels of capability depending on the task, language, and context.

Self‑check: Capabilities

Question: Which of the following is not a typical capability of generative AI?

  • A. Generating a summary of a long article
  • B. Translating text from English to French
  • C. Performing arithmetic calculations with 100% accuracy
  • D. Generating a list of study questions

Answer: C. While some AI systems can perform arithmetic, it is not a defining capability of generative AI, and accuracy is not guaranteed. Generative AI is primarily focused on content creation and transformation.

6. Opportunities

Each capability creates opportunities — legitimate benefits that AI‑supported activities may provide. These opportunities are not guaranteed; they depend on how the capability is applied, in what context, and with what safeguards.

Table 1. Capabilities and associated opportunities
Capability Opportunity Example
Rapid generation Faster creation of drafts, outlines, and initial content A student generates a first draft of an essay to edit and refine.
Summarisation Faster orientation to lengthy or complex material A researcher summarises 50 articles to identify relevant ones.
Translation Expanded access to information across languages A community worker translates a health notice into multiple languages.
Ideation Broader exploration of possibilities and perspectives A writer generates 20 story ideas to overcome creative block.
Question generation Enhanced self‑testing and learning support A student generates practice questions from lecture notes.
Adaptation of tone Tailored communication for different audiences A manager rephrases a technical memo for a non‑technical audience.
Key insight: Opportunities are potential benefits. They are not realised automatically — they require appropriate use, human judgment, and verification. The same capability can create opportunities in one context and risks in another.
Self‑check: Opportunities

Question: Which of the following is an opportunity associated with the capability of rapid generation?

  • A. Faster production of misinformation
  • B. Faster creation of a first draft for editing
  • C. Reduced need for human oversight
  • D. Guaranteed accuracy of outputs

Answer: B. Rapid generation can enable faster creation of drafts, which can then be refined by a human. The other options describe risks (A, C) or an unrealistic claim (D).

7. Limitations and Risks

Every capability also carries corresponding risks and limitations. These are not necessarily inevitable, but they are possible and must be anticipated and managed.

Common limitations and risks:
  • Hallucination / fabrication: The system generates content that is not supported by evidence or is entirely invented.
  • Unsupported claims: The output makes assertions without evidence or with insufficient evidence.
  • Outdated information: The system's knowledge may not include recent developments.
  • Missing context: The output omits important conditions, caveats, or background.
  • Misleading omission: Important information is left out, creating a misleading impression.
  • Overgeneralisation: The system applies patterns inappropriately to new contexts.
  • Bias: The output reflects biases present in the training data, leading to unfair or discriminatory outcomes.
  • Unequal performance: The system performs well for some languages, topics, or cultural contexts but poorly for others.
  • Privacy exposure: The system may inadvertently reveal or leak personal or sensitive information.
  • Obscured authorship: The source of the content is unclear, making accountability difficult.
  • Overreliance: Users may trust the system too much, reducing critical thinking and verification.
  • Automation bias: Users may accept AI‑generated content without questioning it because it comes from a "system."
  • Hidden verification work: The time saved in generation may be offset by the time required for verification.
  • Capability loss: Over‑dependence on AI may lead to erosion of human skills.
  • Weak accountability: It may be unclear who is responsible when something goes wrong.
  • Inappropriate professional use: Using AI in contexts where professional judgment is required (e.g., health, law) without adequate oversight.
Example: The same capability, different risks

Capability: Summarisation

  • Opportunity: Faster orientation to a long document.
  • Risk: The summary may omit important caveats, conditions, or dissenting views, leading the reader to an incomplete or misleading understanding.

Capability: Translation

  • Opportunity: Expanded language access.
  • Risk: The translation may alter meaning, lose nuance, or misinterpret cultural context.
Self‑check: Limitations and risks

Question: Which of the following is a risk associated with rapid generation?

  • A. Faster generation of accurate information
  • B. Faster generation of misinformation
  • C. Reduced need for verification
  • D. Guaranteed reliability of outputs

Answer: B. Rapid generation can also produce misinformation quickly. This is a risk that must be managed. The other options describe opportunities (A) or unrealistic claims (C, D).

8. Fluency vs. Reliability

A generative AI output may appear credible because it contains features that we associate with expertise and authority. However, these features do not independently establish truth, authenticity, or reliability.

Features that increase perceived credibility — but do not prove accuracy:
  • Confident language: "Clearly," "undoubtedly," "the evidence shows."
  • Polished grammar: Correct spelling, punctuation, and syntax.
  • Technical vocabulary: Domain‑specific terms that sound expert.
  • Detailed explanations: Elaborate descriptions that appear thorough.
  • Headings and structure: Clear organisation that suggests completeness.
  • Tables and figures: Visual representations that imply precision.
  • Citations: References to sources that may be fabricated.
  • Numerical precision: Specific numbers that seem authoritative.
  • Familiar terminology: Language that sounds like what we expect.
  • Professional tone: Formal, measured language appropriate to a professional context.

These features do not independently establish:

Example: Fluency without reliability

AI‑generated text:

"According to a 2023 study published in the Journal of Public Health, vaccination rates in rural areas are 34% lower than in urban areas. This disparity is primarily due to logistical barriers, which can be addressed through mobile vaccination units. The evidence strongly supports immediate implementation of such units to achieve herd immunity."

Analysis:

  • Fluency: The text is grammatically correct, uses technical terms, includes a citation, and makes a specific claim (34%).
  • Reliability issues: The "study" may be fabricated; the 34% figure may be unsupported; the conclusion may be too strong for the evidence; mobile units may not be the only or best solution.

Conclusion: The text appears credible but requires verification before it can be trusted or used.

Self‑check: Fluency vs. reliability

Question: Which of the following features increases the perceived credibility of an AI‑generated output but does not prove its reliability?

  • A. The output is grammatically correct.
  • B. The output includes citations to sources.
  • C. The output uses professional terminology.
  • D. All of the above.

Answer: D. All of these features can increase perceived credibility, but none independently establish truth, authenticity, or reliability. All outputs must be verified.

9. Technical Capability vs. Contextual Suitability

There is a critical distinction between:

Key distinction

A system may be technically capable of generating a health explanation, a legal summary, an assessment comment, an employment recommendation, a financial comparison, or a public notice — without being an appropriate final authority for any of these purposes.

Table 2. Capability vs. suitability: examples
Task Technically capable? Contextually suitable? Why?
Generate a summary of a news article Yes Maybe (with verification) Low stakes; summary can be checked against the original.
Generate a legal contract Yes (can produce text) No (without lawyer review) High stakes; legal consequences of errors.
Generate a health diagnosis Yes (can produce text) No High stakes; risk of serious harm; requires medical qualification.
Generate a study question Yes Yes (with review) Low stakes; can be checked by a human.
Generate a financial investment recommendation Yes No (without qualified advisor review) High stakes; risk of financial loss.
Key insight: Just because AI can do something does not mean it should be used for that purpose. Suitability depends on context, stakes, safeguards, and human accountability.
Self‑check: Capability vs. suitability

Question: A generative AI system can produce a detailed job performance evaluation. Which of the following statements is most accurate?

  • A. Because the system can produce it, the evaluation is suitable for use.
  • B. The system is technically capable, but suitability depends on context, stakes, and review.
  • C. The system cannot produce a job evaluation because it requires human judgment.
  • D. The system's output should be used without review because it is objective.

Answer: B. Technical capability does not establish suitability. A job evaluation has significant consequences for employees and requires careful human judgment, verification, and accountability.

10. Beneficiaries and Burden Bearers

Every AI‑supported activity affects people. It is important to ask:

Example: AI in hiring
  • Capability: Screening job applications.
  • Opportunity: Faster processing of large numbers of applications.
  • Risk: Bias against certain demographic groups.
  • Beneficiaries: Employers (time saved), some applicants (if they match the system's criteria).
  • Burden bearers: Applicants who are unfairly screened out; human recruiters who must review and verify decisions; the company if the system's decisions are challenged.

Key question: Who is responsible for ensuring the screening is fair and accurate?

Common burden bearers include:

Self‑check: Beneficiaries and burden bearers

Question: In the scenario of using AI to draft a public health report, who is a potential burden bearer?

  • A. The AI system itself
  • B. The public health researcher (Dr. Chen) who must verify the report
  • C. The training data
  • D. The AI company's shareholders

Answer: B. Dr. Chen must perform the verification work and bear the responsibility for the accuracy of the report. The AI system (A) is not a burden bearer in the human sense; training data (C) and shareholders (D) are not directly affected in the same way.

11. Worked Example: AI‑Generated Customer Service Responses

This worked example demonstrates how to analyse a generative AI use by identifying capabilities, opportunities, risks, beneficiaries, burden bearers, and the distinction between capability and suitability.

Scenario: A retail company uses a generative AI system to draft responses to customer service emails. The AI generates replies that are grammatically correct, empathetic in tone, and include relevant information from the company's knowledge base.

Table 3. Analysis of the AI‑generated customer service use case
Element Analysis
Capability Rapid generation of draft responses in a professional, empathetic tone; retrieval and integration of information from a knowledge base.
Opportunity Faster response times to customers; reduced workload for human agents; consistent tone and style.
Risk / limitation AI may provide inaccurate information; may fail to handle complex cases; may escalate issues incorrectly; may not understand nuance or emotion; may generate responses that sound empathetic but are not actually helpful.
Beneficiary Customers (faster responses); company (cost savings, efficiency); human agents (reduced workload for routine cases).
Burden bearer Customers who receive inaccurate information; human agents who must review, correct, or escalate AI‑generated responses; the company if errors lead to complaints or legal issues.
Capability vs. suitability The system is capable of generating a response, but it is not suitable for all types of customer service inquiries. It should be used for routine, low‑risk inquiries with human review.
Human‑review need All AI‑generated responses should be reviewed by a human agent before sending. Complex, sensitive, or high‑stakes cases should be handled by humans.
Fluency concern The AI's empathetic tone may make the response seem credible, but the content may still be incorrect or incomplete.
Preliminary conclusion AI‑generated responses can be a useful starting point for routine inquiries, but human oversight and verification are essential. The system is a tool to augment, not replace, human customer service agents.
Key takeaway: The AI system can generate responses quickly and with appropriate tone, but the company must ensure that responses are accurate, appropriate, and complete before they reach customers.
Show reflection questions for this worked example
  • What would happen if the AI‑generated response contained incorrect information about a product?
  • How might the company ensure that human agents are not overburdened by reviewing AI‑generated responses?
  • What are the ethical implications of using AI to generate "empathetic" responses?

12. Guided Learning Activities

Activity 12.1: Capability and Limitation Inventory
  • Purpose: To practise identifying capabilities, limitations, and human‑review requirements in an AI system.
  • Learning outcome: Describe major capabilities and limitations of generative AI.
  • Estimated time: 20–25 minutes.

Instructions:

Option A (live system): Use an institutionally approved generative AI system. Enter a prompt such as: "Explain the concept of machine learning to a beginner." Examine the output.

Option B (supplied output): Use the following sample output:

"Machine learning is a subfield of artificial intelligence that enables systems to learn from data without being explicitly programmed. It has been used in everything from self‑driving cars to medical diagnosis. By 2025, machine learning will be able to solve all human problems and will make human decision‑making obsolete."

Record the following:

  1. Demonstrated capability: What can the system do?
  2. Unsupported capability claim: Does the system claim something it cannot do?
  3. Apparent limitation: What is missing or problematic?
  4. Claim requiring verification: What needs to be checked?
  5. Sensitive information to avoid: What should not be entered?
  6. Consequence if the output is wrong: What could happen if this output is used?
  7. Human‑review requirement: What review is needed?
Show sample analysis (for supplied output)
  1. Demonstrated capability: The system can generate a clear, structured explanation of machine learning.
  2. Unsupported capability claim: "By 2025, machine learning will be able to solve all human problems" — this is an unsupported, overly optimistic prediction.
  3. Apparent limitation: The claim about solving all human problems is not supported; the statement is overgeneralised.
  4. Claim requiring verification: The claim about 2025 and "all human problems" requires verification (and is likely false).
  5. Sensitive information: Not applicable here, but in general, do not enter personal or proprietary data.
  6. Consequence if wrong: If someone believed the claim, they might overestimate AI's capabilities and make poor decisions.
  7. Human‑review requirement: A human must check the claim against reliable sources and provide a balanced perspective.
Activity 12.2: Fluency vs. Reliability Exercise
  • Purpose: To practise identifying features that increase perceived credibility without proving reliability.
  • Learning outcome: Explain why fluency, confidence, detail, and professional formatting do not establish factual reliability.
  • Estimated time: 20–25 minutes.

Instructions: Read the following AI‑style response and identify the issues before revealing the guided analysis.

AI‑generated response:

"According to a comprehensive 2024 meta‑analysis published in the Journal of Educational Technology, the use of generative AI in higher education increases student learning outcomes by 42% on average. This finding is consistent across all disciplines, student populations, and institutional types. The evidence is clear: universities that do not adopt AI will fall behind. Furthermore, AI‑powered tutors have been shown to reduce achievement gaps by 30%, making education more equitable. These results are definitive and require immediate implementation."

Identify the following issues in the text:

  1. Accurate general information: Is there anything that is plausibly accurate?
  2. Unsupported assumption: What is assumed without evidence?
  3. Overgeneralisation: Where is a claim applied too broadly?
  4. Claim requiring a current authoritative source: What needs a verifiable source?
  5. Conclusion stronger than the evidence: Where does the text go beyond what is justified?
  6. Polished feature that increases perceived credibility: What makes it look credible?
Show guided analysis
  1. Accurate general information: The general topic (AI in education) is plausible; generative AI is being studied in educational contexts.
  2. Unsupported assumption: The "42% increase" and "30% reduction" are specific numbers that are not supported by any source shown.
  3. Overgeneralisation: "Consistent across all disciplines, student populations, and institutional types" — this is highly unlikely and unsupported.
  4. Claim requiring a current authoritative source: The meta‑analysis citation is specific but likely fabricated; the 42% figure needs a verifiable source.
  5. Conclusion stronger than the evidence: "Definitive and require immediate implementation" — the evidence is presented as conclusive when it is not.
  6. Polished feature: The text uses a citation, a specific percentage, confident language ("clear," "definitive"), and professional terminology ("meta‑analysis," "achievement gaps").

Key lesson: The text is fluent and professional, but it contains unsupported claims, overgeneralisations, and conclusions that go beyond the evidence. It requires verification.

Activity 12.3: Opportunity‑Risk Pairing
  • Purpose: To practise connecting capabilities with opportunities, risks, stakeholders, contextual factors, and safeguards.
  • Learning outcome: Identify an opportunity associated with a generative AI capability and a corresponding risk.
  • Estimated time: 20–25 minutes.

Instructions: For each of the following capabilities, complete the table by identifying:

  • An intended opportunity
  • At least one corresponding risk
  • A stakeholder who might be affected
  • A contextual factor that may increase or reduce the risk
  • A possible safeguard
Opportunity‑risk pairing activity
Capability Opportunity Risk Stakeholder Contextual factor Safeguard
Rapid generation
Summarisation
Translation
Show sample answers
Sample opportunity‑risk pairing
Capability Opportunity Risk Stakeholder Contextual factor Safeguard
Rapid generation Faster creation of draft documents Faster generation of misinformation Readers / public Whether the content is reviewed before publication Human review and verification before distribution
Summarisation Faster orientation to lengthy texts Omission of key caveats or conditions Decision‑makers relying on summary Complexity and nuance of the original material Comparing summary to original; using multiple summaries
Translation Expanded access to information Loss of meaning or cultural nuance Non‑native language speakers Domain specificity (e.g., legal vs. general) Human translator review; back‑translation

Note: Your answers may vary; the key is to identify plausible opportunities, risks, stakeholders, factors, and safeguards.

13. Exercises

These exercises are designed to help you apply the concepts from this lesson. Attempt each exercise before revealing the solutions or hints.

Exercise 1: Distinguish Demonstrated Capability from Marketing Claim
  • Learning outcome: Describe major capabilities and limitations of generative AI.
  • Estimated time: 10–12 minutes.
  • Task: Read the following statements about a generative AI system. For each statement, indicate whether it describes a demonstrated capability (what the system can actually do), an unsupported capability claim (what it claims but cannot reliably do), or insufficient information (you cannot tell).
  1. "This system can generate a 500‑word essay on any topic in under 10 seconds."
  2. "This system understands the emotional state of the user."
  3. "This system can translate text between English and Spanish with high accuracy for general topics."
  4. "This system will replace all human writers within five years."
  5. "This system can generate code in Python that compiles and runs without errors."
Show hint

Consider whether the claim is verifiable, whether it is realistic, and whether it is supported by evidence. Unsupported claims often use absolute language ("all," "any," "will replace") or refer to capabilities that require human understanding.

Show sample solution
  1. Demonstrated capability — many systems can generate text quickly. (Note: "on any topic" may be an overstatement, but the core capability is demonstrated.)
  2. Unsupported capability claim — AI does not "understand" emotions in the human sense; it may detect sentiment, but not emotional understanding.
  3. Demonstrated capability — many systems can translate between major languages with reasonable accuracy for general content.
  4. Unsupported capability claim — this is a prediction, not a demonstrated capability, and it is speculative.
  5. Mixed / uncertain — some systems can generate working code, but not always; success depends on the complexity of the task and the language. This statement is too broad.
Exercise 2: Distinguish Technical Capability from Contextual Suitability
  • Learning outcome: Explain why technical capability does not establish contextual suitability.
  • Estimated time: 12–15 minutes.
  • Task: For each of the following uses, explain whether the system is technically capable (can it produce the output?) and whether the use is contextually suitable (should it be used for this purpose?). Justify your reasoning.
  1. Using a generative AI system to draft a social media post for a personal blog.
  2. Using a generative AI system to generate a medical treatment plan for a patient with a chronic condition.
  3. Using a generative AI system to generate a list of potential interview questions for a job candidate.
  4. Using a generative AI system to generate a final legal opinion that will be submitted to a court.
Show hint

Consider the stakes, the consequences of errors, the need for professional qualifications, and the availability of human oversight.

Show sample solution
  1. Technically capable: Yes. Contextually suitable: Yes, with review. Low stakes; the user can review and edit before posting.
  2. Technically capable: Yes (can generate text). Contextually suitable: No. High stakes; requires medical expertise and cannot be delegated to AI without extensive qualification and oversight.
  3. Technically capable: Yes. Contextually suitable: Possibly, with review. Can be a useful starting point, but the interviewer must ensure questions are appropriate, legal, and relevant.
  4. Technically capable: Yes (can generate text). Contextually suitable: No. Legal opinions have significant consequences and require a qualified legal professional.
Exercise 3: Identify Features That Increase Perceived Credibility
  • Learning outcome: Explain why fluency, confidence, detail, and professional formatting do not establish factual reliability.
  • Estimated time: 12–15 minutes.
  • Task: Read the following AI‑generated passage. Identify at least four features that make it appear credible, and explain why each feature does not independently establish reliability.

"The 2023 Global AI Readiness Report, published by the International Institute for Digital Futures, found that 87% of organisations have adopted generative AI in at least one business function. This represents a 40% increase from 2022. The report concludes that organisations not adopting AI will face a 'competitiveness cliff' by 2026. These findings are statistically significant at p < 0.01 and have been peer‑reviewed by leading experts."

Show hint

Look for features like: citations, specific percentages, authoritative‑sounding language, technical terms, and strong conclusions.

Show sample solution
  • "87% of organisations" — Specific numbers increase perceived precision, but the number may be fabricated or based on unrepresentative data.
  • "Global AI Readiness Report" — The citation sounds authoritative, but the source may be fabricated or misrepresented.
  • "statistically significant at p < 0.01" — Technical language suggests scientific rigour, but the claim may be unsupported or misinterpreted.
  • "peer‑reviewed by leading experts" — This implies validation, but it may be fabricated or overstated.
  • "competitiveness cliff" — Dramatic language creates urgency and implies authority, but it is a rhetorical device, not evidence.

Conclusion: The passage is designed to appear credible, but none of these features independently establishes the truth of the claims.

Exercise 4: Create Opportunity‑Risk Pairs for Two Different Contexts
  • Learning outcome: Identify an opportunity associated with a generative AI capability and a corresponding risk.
  • Estimated time: 15–18 minutes.
  • Task: Choose two different contexts from the list below. For each context, identify a generative AI capability, an opportunity, and a corresponding risk. Then identify one stakeholder who might be affected and one safeguard.
  • Educational assessment (e.g., grading or feedback)
  • Creative writing (e.g., generating story ideas)
  • Customer service (e.g., responding to inquiries)
  • Public health communication (e.g., drafting health messages)
Show sample solution

Context 1: Educational assessment

  • Capability: Generating feedback on student work.
  • Opportunity: Providing immediate, consistent feedback to students.
  • Risk: Feedback may be generic, inaccurate, or miss nuanced issues; may not be pedagogically sound.
  • Stakeholder: Students (who receive feedback) and instructors (who are accountable).
  • Safeguard: Instructors should review and personalise AI‑generated feedback before it is given to students.

Context 2: Public health communication

  • Capability: Generating health messages in multiple languages.
  • Opportunity: Reaching diverse populations with clear, translated health information.
  • Risk: Translation may be culturally inappropriate or medically inaccurate.
  • Stakeholder: The public (who rely on the messages) and health authorities (who are responsible for accuracy).
  • Safeguard: All messages should be reviewed by a qualified health professional and a cultural liaison.

14. Homework Problems

Formative · Ungraded · Recommended preparation for Assignment 1

Problem 1: Capability Identification
  • Estimated time: 8–10 minutes.
  • Task: A generative AI system is described as being able to "take a complex topic and explain it in simple terms for a non‑expert audience." Identify the capabilities involved and explain why each is a capability of generative AI.
Show hint

Consider capabilities like summarisation, adaptation of tone and format, and simplification.

Show sample solution
  • Summarisation: The system must identify the key points of the complex topic.
  • Adaptation of tone and format: The system must rephrase the content for a non‑expert audience.
  • Pattern‑based transformation: The system applies patterns of simplification and explanation learned from training data.
Problem 2: Limitation Identification
  • Estimated time: 8–10 minutes.
  • Task: A generative AI system is used to generate a summary of a scientific study. The summary is fluent and well‑structured, but it omits important limitations and caveats that were present in the original study. What limitation of generative AI does this illustrate? Explain.
Show hint

Consider the risks of summarisation: omission of important details, caveats, or conditions.

Show sample solution

This illustrates the risk of missing context or misleading omission. The AI may generate a fluent summary that appears complete but omits important information, leading the reader to an incomplete or misleading understanding. This is a common limitation of AI summarisation, especially for complex or nuanced content.

Problem 3: Fluency vs. Reliability
  • Estimated time: 10–12 minutes.
  • Task: Write a short paragraph that looks credible but contains a factual error. Then identify the features that make it look credible and explain why the error might be missed by a casual reader.
Show hint

Use features like confident language, a specific detail, a citation, or professional terminology to make the text appear credible. Then introduce a subtle factual error.

Show sample solution

Sample paragraph:

"According to the 2023 Canadian Digital Health Survey, 78% of Canadians use digital health tools, with adoption highest among seniors aged 65 and older at 82%. These findings, published in the Canadian Journal of Health Informatics, demonstrate that digital health is universally embraced across all age groups, with no significant demographic barriers. The evidence clearly supports a national digital health strategy."

Features that make it look credible:

  • Specific percentage (78%, 82%).
  • Citation to a named survey and journal.
  • Confident language ("clearly supports," "universally embraced").
  • Professional terminology ("digital health," "demographic barriers").

Factual error: The claim that adoption is "highest among seniors aged 65 and older at 82%" may be false; in reality, adoption of digital health tools is often lower among seniors. The error might be missed because the text appears authoritative and uses specific numbers.

Problem 4: Capability vs. Suitability
  • Estimated time: 10–12 minutes.
  • Task: A generative AI system can generate a detailed financial report with charts and analysis. Is this system suitable for use in a high‑stakes investment decision? Justify your answer.
Show hint

Consider the stakes, the need for accuracy, the role of professional qualifications, and the importance of human judgment in high‑stakes financial decisions.

Show sample solution

No, not suitable without extensive review and oversight. The system may be technically capable of generating a report with plausible charts and analysis, but high‑stakes investment decisions require:

  • Accurate and verified data.
  • Professional financial expertise.
  • Consideration of current market conditions.
  • Accountability for decisions.
  • Human judgment to interpret and apply information.

An AI‑generated report could be a useful input, but it should not be the sole basis for a high‑stakes decision. A qualified financial professional should review, verify, and take responsibility for any decisions.

Problem 5: Opportunity‑Risk Mapping
  • Estimated time: 12–15 minutes.
  • Task: A journalist uses a generative AI system to generate draft articles about breaking news. The AI can produce a draft in under 30 seconds, allowing the journalist to publish quickly. Identify the opportunity and at least two risks associated with this use. Then identify one safeguard the journalist should use.
Show hint

Consider the risks of rapid generation in a news context: accuracy, verification, bias, and the importance of journalistic integrity.

Show sample solution
  • Opportunity: Faster publication of news, enabling the journalist to cover more stories and provide timely information.
  • Risks:
    • The AI may generate inaccurate or misleading information, damaging the journalist's credibility.
    • The AI may not verify facts or may include fabricated information.
    • The AI may not capture nuance, context, or the human impact of the story.
  • Safeguard: The journalist must verify all facts, check all sources, and review the AI‑generated draft for accuracy, tone, and completeness before publication.
Problem 6: Beneficiary and Burden Bearer
  • Estimated time: 10–12 minutes.
  • Task: A healthcare organisation uses a generative AI system to draft patient discharge summaries. The AI generates a summary based on the patient's medical records. Identify two beneficiaries and two burden bearers of this use.
Show hint

Beneficiaries might include patients (faster discharge), doctors (time saved), or the organisation (efficiency). Burden bearers might include doctors who must review and verify, or patients if the summary contains errors.

Show sample solution
  • Beneficiaries:
    • Patients: May receive discharge summaries more quickly, allowing them to leave the hospital sooner.
    • Healthcare providers: May save time on documentation, allowing them to spend more time with patients.
  • Burden bearers:
    • Physicians: Must review each AI‑generated summary for accuracy and completeness, bearing the responsibility for any errors.
    • Patients: May suffer if the summary contains errors (e.g., incorrect medication instructions).
Problem 7: Safeguard Design
  • Estimated time: 10–12 minutes.
  • Task: A university is considering using a generative AI system to provide feedback on student essays. The system can generate detailed, personalised feedback in seconds. Propose two safeguards that the university should implement to ensure the feedback is appropriate and effective.
Show hint

Consider safeguards related to quality control, human oversight, privacy, and the pedagogical value of the feedback.

Show sample solution
  • Safeguard 1: All AI‑generated feedback should be reviewed by a qualified instructor before being given to students. The instructor should check for accuracy, relevance, and pedagogical appropriateness.
  • Safeguard 2: The AI system should not store or retain student essays or personal information. All data should be processed securely and deleted after use, to protect student privacy.
  • Safeguard 3: The university should periodically audit a sample of AI‑generated feedback to ensure quality and consistency.
Problem 8: Transfer to an Unfamiliar Context
  • Estimated time: 12–15 minutes.
  • Task: You are a municipal planner using a generative AI system to draft a public consultation report based on community feedback. The AI summarises thousands of comments and generates recommendations. What opportunities and risks do you see? What human oversight is required?
Show hint

Consider the need to accurately represent community views, the risk of missing minority perspectives, and the importance of transparency and accountability in public decision‑making.

Show sample solution
  • Opportunities: The AI can quickly process large volumes of feedback, identify common themes, and generate a draft report. This saves time and may help planners see patterns they would otherwise miss.
  • Risks:
    • The AI may misrepresent community views, especially minority perspectives.
    • The AI may oversimplify complex or nuanced feedback.
    • The AI may generate recommendations that are not feasible or appropriate for the community.
  • Human oversight:
    • A planner must review the AI‑generated summary against the original feedback to ensure accuracy.
    • A planner must assess the recommendations for feasibility and appropriateness.
    • The final report should include a clear disclosure that AI was used, and the planner should take responsibility for the content.

15. Mini‑Project: AI Opportunity‑Risk Map

Project: AI Opportunity‑Risk Map
  • Purpose: To integrate concepts from this lesson by analysing a specific AI capability in a specific context.
  • Learning outcomes: Describe major capabilities and limitations; identify opportunities and risks; identify beneficiaries and burden bearers; distinguish capability from suitability.
  • Estimated effort: 60–75 minutes.

Instructions: Select one generative AI capability and analyse it in one specific context. Complete the following sections.

Suggested capabilities:

  • Rapid generation
  • Summarisation
  • Translation
  • Ideation
  • Question generation
  • Adaptation of tone and format

Suggested contexts:

  • Educational assessment
  • Creative writing
  • Customer service
  • Public health communication
  • Workplace documentation
  • Personal planning

Required sections:

  1. Capability: What is the capability you are analysing?
  2. Intended use: How would this capability be used in the context?
  3. Opportunity: What is the potential benefit?
  4. Corresponding risk: What risk accompanies this capability?
  5. Intended beneficiary: Who benefits from the use?
  6. Possible burden bearer: Who bears the verification work or risk?
  7. Contextual factors: What factors affect the suitability of this use?
  8. Evidence required: What evidence is needed to verify the output?
  9. Human‑review need: What human review is required?
  10. Preliminary safeguard: What safeguard would you propose?
  11. Remaining uncertainty: What questions remain unanswered?
  12. Reason to study further: Why should this use be studied further in Unit 2 or Unit 3?
Privacy and academic integrity:
  • Use only synthetic, public, or generalised information.
  • Your analysis is for your own learning and may be used as evidence for Assignment 1.
  • Do not copy this analysis from another student or from an AI system.
Show self‑review checklist
  • Have you selected a specific capability and a specific context?
  • Have you completed all 12 sections?
  • Have you identified at least one opportunity and one risk?
  • Have you identified a beneficiary and a burden bearer?
  • Have you proposed at least one safeguard?
  • Have you identified a question for Unit 2 or Unit 3?
Show solution architecture (staged guidance)
  1. Capability: State the capability (e.g., "summarisation").
  2. Intended use: Describe how it would be used (e.g., "to summarise research articles for a literature review").
  3. Opportunity: Describe the benefit (e.g., "faster identification of relevant articles").
  4. Risk: Describe the risk (e.g., "the summary may omit key findings or caveats").
  5. Beneficiary: Who benefits? (e.g., "the researcher").
  6. Burden bearer: Who bears the burden? (e.g., "the researcher must verify the summary against the original").
  7. Contextual factors: What affects suitability? (e.g., "the complexity of the articles, the stakes of the research").
  8. Evidence required: What must be verified? (e.g., "key findings, methodology, limitations").
  9. Human‑review need: What review is needed? (e.g., "compare summary to each original article").
  10. Safeguard: What safeguard would help? (e.g., "use a checklist to verify key elements").
  11. Uncertainty: What remains unknown? (e.g., "how well the system handles technical terminology").
  12. Study further: Why is more study needed? (e.g., "Unit 2 can explain how the model generates summaries; Unit 3 can provide verification procedures").
Show illustrative partial example (different capability)

Capability: Ideation (generating creative ideas).

Context: A writer using AI to generate plot ideas for a novel.

  • Intended use: The writer inputs a genre and some themes; the AI generates 10 plot ideas.
  • Opportunity: The writer overcomes creative block and explores possibilities they would not have considered.
  • Risk: The AI‑generated ideas may be derivative, clichéd, or culturally insensitive.
  • Beneficiary: The writer (who gains inspiration and saves time).
  • Burden bearer: The writer (who must evaluate, refine, and adapt the ideas).
  • Contextual factors: The genre, the writer's creative style, the audience.
  • Evidence required: Not applicable — ideas are creative, but the writer should ensure originality and appropriateness.
  • Human‑review need: The writer reviews each idea for potential, originality, and cultural sensitivity.
  • Safeguard: The writer treats the AI output as a starting point, not a final product, and adds their own creative input.
  • Uncertainty: How much of the AI's output is based on existing works, and how could that affect originality?
  • Study further: Unit 2 could explain how the AI generates ideas; Unit 3 could address copyright and originality concerns.

16. Self‑Check Quiz

This ungraded quiz is designed to help you check your understanding of the key concepts from this lesson.

Quiz 1 — Multiple choice

Which of the following is a major capability of generative AI?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Summarisation is a core capability of generative AI. Arithmetic (A), robotics (C), and database management (D) are not defining capabilities of generative AI.

Quiz 2 — Multiple choice

What is the primary risk associated with the capability of rapid generation?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Rapid generation can produce misinformation quickly, which is a significant risk. The other options are either opportunities (A) or unrealistic claims (C, D).

Quiz 3 — Multiple choice

Why does technical capability not establish contextual suitability?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: Suitability depends on the context, including the purpose, the stakes, the safeguards in place, and who is accountable. Just because a system can produce an output does not mean it should be used for that purpose.

Quiz 4 — Multiple choice

Which of the following features does not independently establish the reliability of an AI‑generated output?

Select one:
Show answer and explanation

Correct answer: D.

Explanation: Grammar, citations, and professional terminology can increase perceived credibility, but none independently establish truth, authenticity, or reliability. All outputs must be verified.

Quiz 5 — Multiple choice

In an AI‑supported activity, who is typically a burden bearer?

Select one:
Show answer and explanation

Correct answer: C.

Explanation: The human user typically bears the burden of verifying the output and taking responsibility for its use. The AI system (A) is not a burden bearer in the human sense.

Quiz 6 — Multiple choice

Which of the following is an opportunity associated with the capability of summarisation?

Select one:
Show answer and explanation

Correct answer: A.

Explanation: Faster orientation to lengthy material is an opportunity of summarisation. The other options are risks (B, C, D).

Quiz 7 — Multiple choice

Which of the following uses of generative AI is least likely to be contextually suitable without significant human oversight?

Select one:
Show answer and explanation

Correct answer: B.

Explanation: A legal contract has significant consequences and requires a qualified legal professional. It is not suitable for AI‑generated text without substantial human oversight and expertise.

Quiz 8 — True or false

If an AI‑generated output is fluent, detailed, and includes citations, it is likely to be reliable.

Select one:
Show answer and explanation

Correct answer: False.

Explanation: Fluency, detail, and citations can increase perceived credibility but do not establish reliability. The output may be fabricated, biased, or outdated. Verification is required.

17. Assignment 1 Journal Connection

Unit 1 contributes to Assignment 1: AI Learning and Reflection Journal (15% of your final grade).

Journal prompts for Lesson 1.2

After completing this lesson, consider adding to your journal with entries that address some of the following prompts:

  1. Opportunity‑risk pair: Describe one opportunity‑risk pair you identified in this lesson. Why does this pair interest you?
  2. Fluency vs. reliability: Describe a situation where you encountered fluent‑looking information that you later realised was unreliable. What made it appear credible?
  3. Capability vs. suitability: Think of a task you perform regularly. Would generative AI be technically capable of assisting with it? Would it be contextually suitable? Why or why not?
  4. Beneficiaries and burden bearers: Consider an AI use you have encountered. Who benefits? Who bears the burdens?
  5. Personal reflection: How has your understanding of generative AI capabilities and risks changed after this lesson?

Your journal entries are personal and reflective. They do not need to be perfect; they are a record of your learning process.

Show journal planning checklist
  • Have you selected at least 2 prompts to address?
  • Have you written about your own experiences and perspectives?
  • Have you maintained privacy boundaries?
  • Have you kept a record of any AI interactions or observations?
  • Have you noted any questions for later units?

18. Key Takeaways

19. Key Terms

Capability
A function that an AI system can perform (e.g., summarisation, translation, generation).
Opportunity
A legitimate benefit that may be achieved through an AI‑supported activity.
Risk
A possible error, harm, cost, loss, or unintended consequence associated with an AI use.
Hallucination
When an AI system generates content that is not supported by evidence or is entirely fabricated.
Fluency
The quality of being smooth, grammatically correct, and professionally written — which does not guarantee reliability.
Reliability
The quality of being accurate, truthful, and appropriate for use. Reliability must be established through verification.
Technical capability
Whether a system can produce a requested type of output.
Contextual suitability
Whether the output or use is appropriate for a specific purpose, context, and set of stakeholders.
Beneficiary
A person or group who receives benefits from an AI‑supported activity.
Burden bearer
A person or group who performs verification work, bears risks, or experiences negative consequences.
Safeguard
A measure put in place to reduce risk, such as human review, verification, or information boundaries.
Automation bias
The tendency to trust AI‑generated outputs more than they deserve, simply because they come from a system.

[See Appendix A for the complete Generative AI Glossary.]

20. Looking Ahead

In Lesson 1.2, you examined the capabilities, opportunities, and risks of generative AI. You learned that every capability carries corresponding risks, and that technical capability does not establish contextual suitability.

In Lesson 1.3, you will integrate these insights into the AI Paradox — the course's organising concept. You will learn the ORBIT framework (Opportunities, Risks, Beneficiaries and Burden Bearers, Human Intervention and Oversight, Transparency and Accountability) and apply it to analyse AI use cases in a structured way.

You will also begin to establish your personal AI boundaries and prepare for deeper study in Unit 2 (technical mechanisms) and Unit 3 (responsible‑use decision‑making).

Distinguishing this lesson from later content:
  • Lesson 1.2 (this lesson): Focuses on what generative AI can do, the opportunities, and the risks. Introduces the distinction between capability and suitability.
  • Lesson 1.3: Introduces the AI Paradox and the ORBIT framework for analysing opportunities and risks together.
  • Unit 2: Explains how generative AI works technically — training, models, tokenisation, and more.
  • Unit 3: Addresses how to determine whether an AI use is effective and responsible, with formal suitability assessment and decision‑making procedures.
Continue to Lesson 1.3: The AI Paradox and ORBIT Analysis Return to Unit 1 Home

21. Sources & Further Reading

Primary source:

Supporting materials:

Additional references (for further exploration):

Note: All references and citations will be verified and updated in the final course materials. Do not treat unverified references as authoritative sources.

👤