COMP 3270 / Unit 3 / Unit Home / Lesson 1

Lesson 3.1 · Week 2 · Reading: Chapter 3

Human Purpose, Need, and AI Suitability

Learn to begin with the human problem — not the technology — and to assess whether generative AI is genuinely suitable for a bounded task.

Suggested time: 120–150 minutes Opens Unit 3 Builds on Units 1 & 2

Table of Contents

1. Lesson Overview

This lesson is the first of four in Unit 3. It establishes the foundational habit of beginning with a human problem rather than with a technology. Many discussions about generative AI start with the question "What can this system do?" — but effective and responsible use requires a different starting point: "What human need or problem are we trying to address, and is generative AI the right way to address it?"

You will learn to distinguish the underlying problem from a proposed AI solution, identify intended outcomes and affected stakeholders, compare AI with reasonable non‑AI alternatives, and evaluate whether a task is suitable for AI assistance — not just whether it is technically possible. By the end of this lesson, you will have a structured method for screening proposed AI uses and a clear sense of the information you need before proceeding to risk assessment (Lesson 3.2).

Why this matters

A technically capable AI system can produce a fluent, plausible output for almost any input. But capability is not suitability. Using AI for a task that is unclear, poorly scoped, or better handled by a non‑AI approach can waste time, introduce errors, and undermine trust. By learning to define the human problem first, you gain control over when and how you use generative AI — rather than letting the technology drive the decision.

Responsible‑use reminder: Do not enter personal, confidential, proprietary, privileged, or security‑sensitive information into any public or unapproved AI system. Use only synthetic, public, or generalised information for all activities.

2. Learning Outcomes

By the end of this lesson, you should be able to:

  1. Define effective AI use and distinguish it from efficiency or speed.
  2. Distinguish an underlying human problem from a proposed AI solution.
  3. Identify a legitimate human purpose for an AI‑supported activity.
  4. Identify intended outcomes, users, and affected stakeholders.
  5. Distinguish technical capability from contextual suitability.
  6. Compare an AI‑supported approach with feasible non‑AI alternatives.
  7. Determine whether AI is relevant and proportionate to a bounded task.
  8. Assess preliminary suitability and identify missing information.

3. Before You Begin

4. Retrieval Practice

These brief questions activate knowledge from Units 1 and 2. Try to answer them before expanding the feedback.

Q1. In your own words, what is the AI Paradox?

Show feedback

Sample answer: The AI Paradox describes how a system's most impressive capabilities — such as generating fluent, context‑sensitive language — are also the source of its most significant risks. The same features that make generative AI useful (fluency, generality, adaptability) can produce outputs that are plausible but incorrect, misleading, or inappropriate.

Q2. What is the difference between a model and a system in the context of generative AI?

Show feedback

Sample answer: A model is the computational component — the trained neural network that maps inputs to outputs. A system includes the model plus the surrounding application, user interface, prompts, context, retrieval mechanisms, tools, and any human‑review processes. The system is what users interact with; the model is one part of that system.

Q3. What is a technical limitation of generative AI that matters for responsible use?

Show feedback

Sample answer: One key limitation is that generative AI produces outputs probabilistically — it predicts the next token based on patterns in training data, not on truth or fact. This means that fluency and confidence are not evidence of accuracy. Another limitation is the context window: the system can only process a fixed amount of text, which may exclude relevant information.

Q4. What is the ORBIT framework, and what is its purpose?

Show feedback

Sample answer: ORBIT stands for Opportunities, Risks, Beneficiaries and Burden Bearers, Intervention and Oversight, and Transparency and Accountability. It is a framework for structuring an initial responsible‑use analysis by identifying what the AI can do, what could go wrong, who is affected, how humans will oversee it, and how the use will be transparent and accountable.

5. Opening Scenario

📋 Scenario: "We need an AI tutor"

A university department is exploring ways to improve student support in a large introductory course. A staff member proposes: "We should use a generative AI system as a 24/7 tutoring assistant. Students can ask it questions about the course material, and it will provide instant explanations."

The proposal is well‑intentioned, but the department has not yet defined the specific problem they are trying to solve. Some instructors are concerned that students may rely on the AI for answers rather than developing their own understanding. Others worry about privacy if students upload their own work or personal questions.

Questions to consider:

  1. What is the actual human need or problem? Is it the same as "we need an AI tutor"?
  2. Who are the intended users, and who else might be affected (directly or indirectly)?
  3. What non‑AI alternatives should be considered before selecting an AI system?
  4. What information is missing from the proposal that would help determine whether AI is suitable?
Show guided analysis

1. What is the actual human need?
The underlying problem might be: students in a large course need timely, high‑quality support for understanding course concepts, but teaching resources are stretched. The proposed AI tutor is one possible solution — but it is not the problem itself. The problem should be stated in terms of student learning outcomes, resource constraints, and accessibility, not in terms of a particular technology.

2. Who are the intended users and affected stakeholders?
Intended users: students (especially those who need help outside office hours). Affected stakeholders: instructors (who may need to verify the AI's explanations), teaching assistants, the department, the institution, and potentially future employers if students rely on incorrect information. There may also be burden bearers: the staff who must monitor, update, and correct the AI's responses.

3. What non‑AI alternatives should be considered?
Alternatives include: expanded office hours, peer tutoring, improved online discussion forums, structured FAQs, study guides, better use of existing learning management systems, or a combination of these. Each alternative has different costs, scalability, and quality trade‑offs.

4. What information is missing?
The proposal does not specify: what course content the AI would be trained on or given access to; how the AI would be configured to avoid giving incorrect answers; who would review and correct the AI's outputs; how student privacy would be protected; what the success criteria are (e.g., improved grades, reduced support requests, better student satisfaction); and what the baseline is (how well do students currently get support?).

6. The Human Problem

The first step in any responsible AI use is to define the human problem independently of any proposed solution. This means describing the situation in terms of needs, gaps, or challenges that affect people — not in terms of "we should use AI."

🔑 Principle: Problem before technology

"The objective is not to maximise AI use. The objective is to improve a legitimate human outcome." — COMP 3270 course materials

To define the problem, ask:

  • What is the current situation?
  • What evidence shows that a problem exists?
  • Who is affected, and how?
  • What is the current process or method?
  • What would improvement look like?
  • What strengths of the current approach should be preserved?

A common mistake is to conflate the solution with the problem. For example, "We need to summarise these reports using AI" is a solution‑framed statement. The underlying problem might be "managers do not have time to read lengthy reports and therefore miss important insights." AI summarisation is one way to address that problem, but it is not the problem itself.

⚠️ Common misconception

"If the AI can produce an output, the problem is solved." This confuses output with outcome. An AI can generate a 500‑word summary in seconds, but if the summary is inaccurate, misses key points, or requires extensive correction, it may not improve the outcome. Effectiveness depends on whether the output actually serves the human need.

7. Legitimate Human Purpose

Once the problem is defined, the next step is to articulate the legitimate human purpose of the activity. A purpose is legitimate when it is:

✅ Examples of legitimate purposes
  • Supporting practice and skill development
  • Improving access to public information
  • Organising authorised notes or materials
  • Producing a provisional template or draft
  • Reducing repetitive low‑risk formatting tasks
  • Identifying questions or issues for human review
❌ Questionable or illegitimate purposes
  • Concealing prohibited AI use
  • Fabricating evidence or citations
  • Manipulating a vulnerable person
  • Avoiding required professional judgment
  • Processing information without authority
  • Bypassing institutional safeguards

A purpose may be potentially legitimate but underspecified. For example, "help students learn" is a broad purpose; it needs to be made concrete enough to evaluate whether the AI use actually serves that purpose.

8. Intended Outcomes and Stakeholders

Effective AI use requires clarity about what should improve, for whom, and under what conditions. This means identifying:

📊 Success is not just speed or volume

Do not measure success only through number of outputs, speed, cost reduction, system adoption, or user engagement. These may be useful indicators, but they do not tell you whether the human outcome has improved. A fast, cheap output that is wrong or misleading is not a success.

Consider burden bearers as well as beneficiaries. Who does additional work because of the AI system? Who has to verify, correct, or explain its outputs? These are legitimate stakeholders whose interests must be considered.

9. AI Suitability

Suitability is about whether a particular task is a good fit for generative AI, given its capabilities, limitations, and the context of use. A task may be technically possible for AI but not suitable in practice.

Suitability depends on factors such as:

Table 1. Indicators of AI suitability
Characteristic Favours bounded AI assistance Requires caution May suggest unsuitability
Purpose Clear, legitimate, bounded Broad, underspecified Illegitimate, prohibited
Information Public or authorised Personal, internal Unauthorised, highly sensitive
Output Reversible, easy to check Public‑facing, moderate consequence Irreversible, high‑impact
Review Qualified human reviewer available Review difficult or time‑consuming No qualified reviewer, no appeal
Consequence Low consequence, minor inconvenience Moderate financial, reputational Safety, rights, legal, high‑stakes
🧭 Technical capability ≠ suitability

A generative AI system may be able to produce a draft legal contract, a medical summary, or a financial analysis. But that does not mean it is suitable to do so in a real‑world context. Suitability requires considering the context, the stakeholders, the risks, and the human oversight available — not just whether the AI can generate a plausible output.

10. Non‑AI Alternatives

A responsible decision about AI use requires comparing the proposed AI approach with feasible non‑AI alternatives. This prevents the "solution in search of a problem" trap and ensures that the chosen approach is genuinely the best one for the human purpose.

Alternatives to consider include:

💡 Do not assume AI is superior

A non‑AI alternative is not automatically "old‑fashioned" or inferior. In many cases, a well‑designed database, a clear FAQ, or additional human support can be more reliable, more transparent, and more cost‑effective than a generative AI system. The comparison should be honest and balanced.

When comparing alternatives, consider: accuracy, reliability, cost, time, scalability, privacy, accessibility, fairness, transparency, and the ability to correct errors. Each alternative has trade‑offs, and the best choice depends on the specific context and purpose.

11. Decision‑Process Walkthrough

The diagram below illustrates the initial stages of the responsible‑use decision process that we focus on in this lesson.

[INSERT VERIFIED RESPONSIBLE‑USE DECISION DIAGRAM – Stage 1: Problem, Purpose, Alternatives, Suitability]
Figure 1. Stage 1 of the introductory responsible‑use decision process: from human problem to preliminary suitability.

Stages in this lesson:

  1. Define the human problem — describe the need without naming a specific technology.
  2. Establish a legitimate purpose — articulate the lawful, ethical, and clear reason for the activity.
  3. Identify intended outcomes and stakeholders — specify what should improve and for whom.
  4. Consider non‑AI alternatives — compare the proposed AI use with other feasible approaches.
  5. Assess AI suitability — evaluate whether the task is a good fit for generative AI, considering capability, context, and safeguards.
  6. Identify missing information — determine what additional facts or analysis are needed before proceeding.
  7. Make a preliminary recommendation — decide whether to proceed, proceed with safeguards, redesign, use an alternative, or defer.
✅ Key insight

At this stage, you are not making a final decision about whether to use AI. You are screening the proposed use to see if it passes the initial test of legitimacy, clarity, and suitability. If it does not, you may need to revise the proposal, gather more information, or consider a different approach before proceeding to risk assessment (Lesson 3.2).

12. Worked Example

The following example shows how to apply the concepts from this lesson to a specific use case. All information is synthetic and for instructional purposes only.

📝 Use case: Drafting meeting minutes

A project manager regularly spends 2–3 hours after each meeting preparing detailed minutes and action items. She is considering using a generative AI tool to produce a first draft from a meeting recording or notes.

1. Human problem

Current situation: The project manager spends significant time on manual minute‑taking, which reduces time available for other project work.

Evidence of need: The manager has tracked her time and finds that minute‑taking consumes 25% of her weekly work hours. The team has also noted that action items are sometimes missed or unclear.

Affected stakeholders: The project manager (direct user), team members who rely on accurate minutes, and other stakeholders who need to track project progress.

Strengths to preserve: The manager's judgment about which points are important, her ability to rephrase and clarify, and her accountability for the accuracy of the minutes.

2. Legitimate purpose

Purpose: To reduce the time required to produce accurate, clear meeting minutes and action items, while maintaining human accountability for the final document.

Legitimacy check: The purpose is lawful, relevant, and proportionate. It does not involve sensitive personal information (beyond ordinary meeting content) and the manager retains authority over the final output.

3. Intended outcomes and stakeholders

Outcome: Meeting minutes are produced within one business day, with clear action items and assigned owners, requiring no more than 30 minutes of human revision.

Success criteria: Team members can identify their action items; no critical decisions are omitted; the manager approves the minutes before distribution.

Users: The project manager (primary user), team members (consumers of the minutes), and possibly administrative staff.

Burden bearers: The project manager must review and correct the draft; team members may need to clarify items if the AI misrepresents them.

4. Non‑AI alternatives

Each alternative has trade‑offs. The AI approach could save time, but requires careful prompt design and human review to ensure accuracy. It may be appropriate if the manager has time to review and correct the draft.

5. Proposed AI role

The AI system would take the meeting recording or a rough set of notes and generate a structured draft of minutes, including key decisions, discussion points, and action items. The draft would be reviewed and revised by the project manager before distribution.

6. Suitability assessment

Preliminary conclusion: The use is suitable, with the requirement that the manager must perform meaningful review and correction before distribution.

7. Missing information

8. Preliminary recommendation

Proceed with safeguards. The use is suitable and the purpose is legitimate, but the manager should pilot the approach with a few meetings, establish clear review and correction procedures, and ensure that the AI system meets privacy requirements.

Show discussion of the worked example

This example illustrates the importance of defining the human problem before selecting a technology. The manager started with a time‑management problem, not with "we need AI minutes." By considering alternatives and assessing suitability, she arrived at a conditional recommendation that preserves human authority and accountability.

Notice that the recommendation is not "yes" or "no" but "proceed with safeguards." This reflects the reality that responsible AI use is often about how and under what conditions the technology is used, not just whether it is used at all.

13. Guided Learning Activities

🧩 Activity 1: Problem Before Technology
  • Purpose: Practise rewriting product‑centred proposals as problem‑centred statements.
  • Learning outcome: Distinguish an underlying human problem from a proposed AI solution.
  • Estimated time: 15 minutes.

Instructions: Rewrite each of the following statements to focus on the human problem rather than the proposed AI solution. Identify the current problem, affected stakeholders, evidence of need, intended improvement, strengths to preserve, and constraints.

  1. "We should use generative AI to automate student support."
  2. "Let's deploy an AI chatbot for customer service."
  3. "We need an AI system to summarise our internal reports."
Show model analysis for #1

Problem‑centred rewrite:

  • Current problem: Students in a large course need timely, accurate support, but the current support model (office hours, email) is insufficient to meet demand.
  • Affected stakeholders: Students (especially those with scheduling constraints), instructors, teaching assistants, and administrative staff.
  • Evidence of need: High volume of support requests, student feedback indicating frustration, or data showing that students who cannot attend office hours have lower performance.
  • Intended improvement: Students receive accurate, helpful responses to their questions within a reasonable time, reducing frustration and improving learning outcomes.
  • Strengths to preserve: The quality and accuracy of explanations, the opportunity for students to ask follow‑up questions, and the pedagogical relationship between students and instructors.
  • Constraints: Budget for additional staff, privacy of student data, and the need to maintain academic integrity.

Key insight: The original statement assumes that AI is the right solution. The problem‑centred statement leaves the solution open and focuses on the human need.

🧩 Activity 2: Legitimate‑Purpose Test
  • Purpose: Evaluate the legitimacy and clarity of proposed AI uses.
  • Learning outcome: Identify a legitimate human purpose and recognise underspecified or questionable purposes.
  • Estimated time: 15 minutes.

Instructions: Classify each proposed use as (a) clearly legitimate, (b) potentially legitimate but underspecified, (c) questionable, (d) inappropriate, or (e) insufficient information. Justify your classification.

  1. Using AI to generate a first draft of a blog post about a public topic.
  2. Using AI to provide mental health advice to users who disclose personal struggles.
  3. Using AI to help a student organise their personal notes and identify questions for their instructor.
  4. Using AI to replace a professional engineer's final sign‑off on structural calculations.
  5. Using AI to generate a personalized motivational message for each employee based on their performance data.
Show model analysis
  1. Clearly legitimate. Generating a first draft for a public blog post is a low‑risk, common use. The author retains responsibility for reviewing and editing.
  2. Questionable / inappropriate. Providing mental health advice is a high‑risk activity requiring professional qualifications and a trusted relationship. AI lacks the judgment, empathy, and accountability required for such a role.
  3. Clearly legitimate. The AI is assisting with organisation and question identification, not making decisions or judgments. The student remains in control.
  4. Inappropriate. Professional engineering sign‑off requires qualified human judgment, legal accountability, and professional liability. AI cannot take on these responsibilities.
  5. Potentially legitimate but underspecified. The purpose is to motivate employees, but the use of performance data raises privacy concerns, and personalised messages may be perceived as intrusive or manipulative. More information is needed about consent, transparency, and the content of the messages.
🧩 Activity 3: AI Suitability Screening
  • Purpose: Assess whether a given task is suitable for generative AI assistance.
  • Learning outcome: Determine whether AI is relevant and proportionate to a bounded task.
  • Estimated time: 20 minutes.

Instructions: For each task, assess the following factors: clarity of purpose, bounded scope, authorised information, verifiable output, reversible effect, manageable consequence, and appropriate human review. Decide whether the task is suitable, requires caution, or is unsuitable.

  1. Generating a list of potential research questions for a literature review (the researcher will evaluate and select).
  2. Translating a patient's medical records from one language to another for use in a clinical decision.
  3. Drafting a standard email response to a routine customer inquiry.
  4. Identifying potential candidates for a job posting based on resume summaries.
Show model analysis
  1. Suitable. The purpose is clear, the scope is bounded, the output is verifiable by the researcher, and the consequence of error is low (the researcher selects the final questions).
  2. Unsuitable / requires caution. Medical records are highly sensitive, and translation errors could have serious consequences for patient health. This task requires professional human translation or a verified, high‑assurance system with qualified oversight.
  3. Suitable. Standard email responses are low‑risk and reversible. The human can review and edit before sending. However, caution is needed if the email contains sensitive information.
  4. Requires caution. Resume screening affects employment opportunities, which is a high‑consequence area. Bias in AI systems could lead to unfair outcomes. Human review is essential, and the AI should be used only as a screening aid, not a final decision‑maker.
🧩 Activity 4: Non‑AI Alternative Comparison
  • Purpose: Compare AI and non‑AI approaches for a given task.
  • Learning outcome: Compare an AI‑supported approach with feasible non‑AI alternatives.
  • Estimated time: 20 minutes.

Instructions: For the scenario below, identify at least three non‑AI alternatives and compare them with the AI approach on accuracy, reliability, cost, time, privacy, and transparency.

Scenario: A university department wants to create a personalised study plan for each student in a large introductory course, based on their performance on quizzes and assignments.

Show model comparison
Comparison of approaches for creating personalised study plans
Approach Accuracy Time Privacy Transparency Cost
AI‑generated plans Moderate; depends on data quality Very fast Data privacy concerns Limited; AI reasoning is opaque Low marginal cost
Human advisor (one‑on‑one) High; can understand context Slow; limited scale Good; direct human relationship High; advisor can explain High staffing cost
Rules‑based algorithm Good for defined rules Fast Depends on implementation High; rules are explicit Moderate development cost
Self‑directed online resources Depends on student Student‑driven High (no data collection) High Low

Analysis: The AI approach offers speed and low marginal cost but raises privacy concerns and lacks transparency. A hybrid approach (AI‑assisted advisor) may balance the trade‑offs: the AI generates a draft plan that the advisor reviews and personalises.

14. Exercises

Apply the concepts from this lesson to the following scenarios. Each exercise includes hints and sample solutions.

Exercise 3.1.1 – Human problem, purpose, AI task, output, and outcome
  • Learning outcome: Distinguish the human problem, purpose, AI task, output, and intended outcome.
  • Estimated time: 10 minutes.
  • Scenario: A small business owner wants to use generative AI to write social media posts for their business. They believe this will save time and help them maintain a consistent online presence.
  • Task: Identify the human problem, the purpose, the AI task, the output, and the intended outcome. Are any of these missing or unclear?
Show Hint 1: Define the human problem

Describe the need without naming an AI product or assuming that automation is the solution. What challenge is the business owner facing?

Show Hint 2: Distinguish task from outcome

The AI task is what the AI does (e.g., generate text). The intended outcome is what should improve for the business owner or their audience (e.g., increased engagement). The output is the AI‑generated content itself.

Show sample solution
  • Human problem: The business owner lacks time to create consistent, engaging social media content, and their online presence is irregular, which may reduce customer engagement.
  • Purpose: To maintain a consistent social media presence that attracts and engages customers, without consuming excessive owner time.
  • AI task: Generate draft social media posts based on provided themes, product information, or prompts.
  • Output: The AI‑generated text for each post (and possibly suggested images or hashtags).
  • Intended outcome: Increased customer engagement, improved brand visibility, and reduced time burden on the owner.

Missing or unclear: The owner has not defined what success looks like (e.g., engagement metrics), nor have they considered the need for human review to ensure posts are accurate, on‑brand, and appropriate.

Exercise 3.1.2 – Legitimacy and clarity of purpose
  • Learning outcome: Evaluate the legitimacy and clarity of proposed purposes.
  • Estimated time: 10 minutes.
  • Scenario: A high school teacher proposes using generative AI to provide "personalised feedback" on student essays. The AI would generate comments and suggestions for each student.
  • Task: Evaluate the legitimacy and clarity of this purpose. What information is needed to determine whether this use is appropriate?
Show Hint 1: Check legitimacy factors

Consider: Is the purpose lawful? Ethically defensible? Relevant? Clear enough to evaluate? Proportionate? Consistent with human authority?

Show Hint 2: Consider the educational context

Feedback on student work is a pedagogical activity. What is the teacher's role? What are the students' expectations? How would the AI's feedback affect student learning and trust?

Show sample analysis

Legitimacy: The purpose is potentially legitimate (supporting student learning) but underspecified. The teacher needs to clarify:

  • What kind of feedback will the AI provide? (e.g., grammar, structure, content, creativity?)
  • How will the teacher review and personalise the AI's feedback before sharing it with students?
  • What is the educational rationale for using AI rather than the teacher's own feedback?
  • How will student privacy be protected? (Essays may contain personal information.)
  • Will students be informed that AI was used, and will they have the opportunity to challenge the feedback?

Conclusion: The purpose is potentially legitimate but underspecified. The teacher should develop a clear policy and pedagogical rationale before proceeding.

Exercise 3.1.3 – Compare AI and non‑AI alternatives
  • Learning outcome: Compare an AI‑supported approach with feasible non‑AI alternatives.
  • Estimated time: 15 minutes.
  • Scenario: A non‑profit organisation wants to analyse public comments from a community consultation to identify key themes and concerns.
  • Task: Identify the proposed AI role and compare it with at least two non‑AI alternatives. Which approach would you recommend and why?
Show Hint 1: Define the AI role

What would the AI do? Would it categorise comments, extract themes, generate a summary, or something else?

Show Hint 2: Consider trade‑offs

Compare alternatives on: accuracy, reliability, cost, time, transparency, and the ability to handle nuance or minority viewpoints.

Show sample comparison

Proposed AI role: The AI would process the text of public comments and generate a summary of key themes, along with a count of how many comments relate to each theme.

Alternative 1: Manual thematic analysis — researchers read all comments, identify themes, and code them. This is time‑consuming but allows for deep understanding and nuance.

Alternative 2: Hybrid approach — use AI to suggest initial themes and then have researchers review, refine, and interpret the results. This balances speed and accuracy.

Alternative 3: No analysis — publish the raw comments without analysis. This is transparent but may not help the organisation understand the community's priorities.

Recommendation: The hybrid approach (Alternative 2) is likely the most appropriate. It leverages the AI's speed while preserving human judgment and accountability. The organisation should pilot the approach with a sample of comments and compare the results with manual analysis to validate the AI's performance.

Exercise 3.1.4 – Preliminary suitability and missing information
  • Learning outcome: Assess preliminary suitability and identify missing information.
  • Estimated time: 15 minutes.
  • Scenario: A financial advisor is considering using a generative AI system to generate personalised investment recommendations for clients. The advisor would review and approve each recommendation before sharing it with the client.
  • Task: Assess the preliminary suitability of this use. What factors favour AI assistance? What factors require caution? What information is missing?
Show Hint 1: Consider the consequence

Investment recommendations affect clients' financial well‑being. What happens if the AI's recommendation is wrong?

Show Hint 2: Check the authority

Who is the advisor? What professional obligations do they have? Is the advisor the final decision‑maker, or would the AI make the final recommendation?

Show sample analysis

Factors favouring AI assistance:

  • AI can process large amounts of data quickly.
  • AI can identify patterns or suggest options the advisor might not consider.
  • The advisor retains final review and approval authority.

Factors requiring caution:

  • High consequence of error: financial loss, regulatory penalties, reputational harm.
  • Personal information: the system would process sensitive financial data.
  • Professional regulation: the advisor may have a fiduciary duty that cannot be delegated to AI.
  • Accountability: the advisor remains responsible for the final recommendation, but the AI's reasoning may be opaque.

Missing information:

  • What specific AI system would be used, and what is its accuracy on financial recommendations?
  • How would the AI's recommendations be validated?
  • What is the advisor's regulatory obligation regarding AI‑assisted advice?
  • How would clients be informed about the AI's role?
  • What is the process for correcting or appealing an AI‑generated recommendation?

Preliminary conclusion: This use is high risk and should be approached with extreme caution. It may be suitable only with strong safeguards, including independent validation, regulatory review, and full transparency with clients. The advisor should consult professional and regulatory guidance before proceeding.

15. Homework Problems

Formative · Ungraded · Recommended preparation for later course units and Assignment 1 reflection.

Problem 1 – Problem vs. solution
  • Learning outcome: Distinguish an underlying human problem from a proposed AI solution.
  • Scenario: A manager says, "We need to implement an AI system to handle all routine customer inquiries."
  • Task: Rewrite this as a problem‑centred statement, identifying the current problem, affected stakeholders, evidence of need, intended improvement, strengths to preserve, and constraints.
Show hint

What is the underlying problem? Is it the volume of inquiries? The time staff spend on them? The quality of responses?

Show solution design
  • Current problem: Customer service staff spend a significant portion of their time on routine, repetitive inquiries, which reduces time available for complex issues.
  • Affected stakeholders: Customers (who may experience delays), customer service staff (who may be overworked), and management (who want efficient operations).
  • Evidence of need: Data on inquiry volume, staff time allocation, customer satisfaction scores.
  • Intended improvement: Staff have more time for complex inquiries, and routine inquiries are handled quickly and consistently.
  • Strengths to preserve: The quality and empathy of human‑handled inquiries, the ability to handle unusual cases, and the staff's professional judgment.
  • Constraints: Budget, privacy, system integration, and the need to maintain customer trust.
Problem 2 – Legitimate purpose
  • Learning outcome: Identify a legitimate human purpose.
  • Scenario: A journalist wants to use generative AI to help research a story by summarising publicly available documents and identifying potential leads.
  • Task: Evaluate the legitimacy of this purpose. Is it clearly legitimate, potentially legitimate but underspecified, questionable, or inappropriate?
Show hint

Consider the journalist's role, the nature of the documents, the public interest, and the need for accuracy and verification.

Show sample answer

Clearly legitimate. The journalist is using AI to assist with research on public documents, which is a legitimate journalistic purpose. However, the journalist must verify all AI‑generated information before publication and maintain editorial control. The use of AI should be disclosed to the audience in accordance with the news organisation's policy.

One caveat: If the documents contain sensitive personal information (even if publicly available), the journalist must ensure that their use of AI does not violate privacy norms or the terms of access.

Problem 3 – Stakeholder analysis
  • Learning outcome: Identify intended outcomes, users, and affected stakeholders.
  • Scenario: A hospital is considering using generative AI to draft patient discharge summaries from clinical notes.
  • Task: Identify the intended users, the patients (as stakeholders), and any other affected stakeholders. What are their interests, and who might bear additional work or risk?
Show hint

Consider: doctors, nurses, patients, family members, administrators, and anyone who relies on the discharge summary for continuity of care.

Show solution design
  • Intended users: Doctors or nurses who would review and approve the discharge summary.
  • Patients: Directly affected by the accuracy and clarity of the summary; they need to understand their care plan.
  • Other stakeholders: Family members (who may rely on the summary), community healthcare providers (who receive the summary), hospital administrators (who manage resources and quality), and legal/compliance officers (who ensure records are accurate).
  • Burden bearers: Clinicians who must review and correct the AI‑generated drafts; patients if the summaries are inaccurate or confusing; other providers if the information is incomplete.

Key insight: The burden of review falls on clinicians, who are already busy. If the AI requires extensive correction, it may not be effective.

Problem 4 – Intended outcome
  • Learning outcome: Identify intended outcomes and success criteria.
  • Scenario: A university library wants to use an AI chatbot to answer common reference questions.
  • Task: Define specific, observable success criteria for this use. What would improve, for whom, and how would we know?
Show hint

Consider: response time, accuracy, user satisfaction, staff workload, and the range of questions the chatbot can handle.

Show sample answer
  • Success criterion 1: The chatbot correctly answers at least 80% of common reference questions (verified by a librarian).
  • Success criterion 2: The average response time for common questions decreases from 2 hours (email) to under 1 minute.
  • Success criterion 3: User satisfaction with the chatbot is at least 4 out of 5 on a satisfaction survey.
  • Success criterion 4: Librarians report a measurable reduction in time spent on routine questions, allowing more time for in‑depth research consultations.
  • Success criterion 5: The chatbot consistently provides correct citation and location information for library resources.

Note: These criteria should be measured over a pilot period before full deployment.

Problem 5 – Non‑AI alternatives
  • Learning outcome: Compare an AI‑supported approach with feasible non‑AI alternatives.
  • Scenario: A municipal government wants to use generative AI to draft responses to routine public inquiries, such as questions about garbage collection schedules or building permit requirements.
  • Task: Identify at least three non‑AI alternatives and compare them with the AI approach. Under what conditions would the AI approach be appropriate?
Show hint

Think about: existing FAQs, a searchable website, staff‑answered emails, phone lines, and self‑service portals.

Show solution design
  • Alternative 1: Maintain a comprehensive, searchable FAQ on the city website. Low cost, reliable if maintained, but may not cover all inquiries.
  • Alternative 2: Staff‑answered email responses. High quality, human empathy, but costly and slow.
  • Alternative 3: Interactive voice response (IVR) system. Good for simple inquiries, but frustrating for complex questions.

AI approach: A generative AI system could draft responses that are personalised to the specific inquiry. It could be appropriate if:

  • The responses are reviewed by a human before being sent.
  • The AI is trained on accurate, up‑to‑date municipal information.
  • There is a fallback mechanism for inquiries the AI cannot handle.
  • Users are informed that the response was AI‑assisted and can request human follow‑up.
Problem 6 – Task suitability
  • Learning outcome: Determine whether AI is relevant and proportionate to a bounded task.
  • Scenario: A researcher wants to use generative AI to help write the literature review section of a grant proposal, but the AI would be generating the text based on the researcher's notes and summaries of the literature.
  • Task: Assess the suitability of this use. What factors favour AI assistance? What factors require caution?
Show hint

Consider: the researcher's expertise, the importance of the grant proposal, the accuracy of the AI's output, and the need for proper citation.

Show sample answer

Factors favouring AI assistance:

  • The AI can help structure the review and generate fluent prose.
  • The researcher provides the substantive content (notes and summaries), so the AI is working from authoritative input.
  • The researcher can review and correct the output.

Factors requiring caution:

  • Grant proposals are high‑stakes; errors or misinterpretations could affect funding decisions.
  • The AI may not accurately represent the nuance of the literature.
  • Citation management: the AI may generate incorrect or hallucinated citations.

Conclusion: The use is suitable if the researcher treats the AI output as a draft to be heavily reviewed and revised, and if the researcher verifies all citations and substantive claims. The researcher should also consider the funder's policy on AI use in proposals.

Problem 7 – Scope boundaries
  • Learning outcome: Identify what should remain outside the scope of AI assistance.
  • Scenario: A law firm is using generative AI to help draft contract clauses from a library of precedents. The AI suggests language based on the firm's templates and the specific facts of the case.
  • Task: What activities should remain outside the scope of AI assistance in this context? What must remain under human authority?
Show hint

Consider: legal judgment, professional responsibility, client relationships, and the final interpretation and approval of the contract.

Show sample answer
  • Outside scope (human only): Final interpretation of legal obligations, professional legal judgment about the appropriateness of clauses, approval of the final contract, and client advice and communication.
  • Outside scope (human only): Decisions about legal strategy, risk tolerance, and the negotiation approach.
  • Outside scope (human only): The ultimate accountability for the contract's legal sufficiency.

Reasoning: The AI can assist with drafting and suggesting language, but legal expertise, judgment, and accountability cannot be delegated to AI. The lawyer must review, approve, and take responsibility for the final contract.

Problem 8 – Information authority
  • Learning outcome: Identify whether information authority exists for a proposed AI use.
  • Scenario: A team of researchers wants to use a generative AI system to analyse interview transcripts from a research study. The interviews contain personal stories and identifiable information.
  • Task: What information authority is required? What steps should the researchers take before using AI?
Show hint

Consider: research ethics, participant consent, data protection policies, and the AI system's privacy and data handling practices.

Show sample answer
  • Information authority required: Research ethics board approval, participant consent that covers AI analysis, and institutional data‑protection policies.
  • Steps before using AI:
    • Check the AI system's privacy policy: does it retain data, use it for training, or expose it to others?
    • Anonymise or de‑identify the transcripts before processing.
    • Consider using a secure, on‑premises AI system rather than a public service.
    • Document the AI use in the research protocol and obtain any required approvals.

Key principle: Technical ability to process the information does not create authority to do so. The researchers must ensure they have the legal and ethical right to use the data with AI.

Problem 9 – Transfer problem
  • Learning outcome: Apply the problem‑before‑technology approach to a new context.
  • Scenario: A community organisation wants to use generative AI to translate public health information into multiple languages for a diverse population.
  • Task: Apply the problem‑before‑technology approach to this scenario. What is the human problem? What alternatives exist? What information is needed to assess suitability?
Show hint

Consider: the purpose of the translation, the audience, the accuracy required, the consequences of error, and the community's trust in the translation.

Show sample solution
  • Human problem: Members of a diverse community do not have access to public health information in their preferred language, which may lead to health inequities and poor health outcomes.
  • Alternatives: Human translation (professional or community‑based), use of existing translated materials, simplified language versions, or partnerships with community organisations.
  • Proposed AI role: Generate initial translations of public health messages, to be reviewed and corrected by a professional translator or community expert.
  • Information needed: What is the AI's accuracy for the target languages? What is the process for human review? How will the community be informed about the use of AI translation? What is the fallback if the AI output is unusable?

Preliminary conclusion: The use could be suitable if human review is robust, the AI is accurate enough, and the community is involved in the translation and review process. However, if the translation is for critical health information, professional human translation may be the only acceptable option.

16. Mini‑Project: AI Suitability Brief

📁 AI Suitability Brief
  • Purpose: Apply the concepts from Lesson 3.1 to a bounded AI use case.
  • Learning outcomes: Define a human problem, establish a legitimate purpose, identify stakeholders, compare alternatives, assess suitability, and identify missing information.
  • Estimated effort: 60–90 minutes.

Instructions: Select a bounded AI use from one of the following contexts (or choose your own with instructor approval):

  • Education / learning
  • Professional practice
  • Organisational activity
  • Communication / creative work
  • Public information / government
  • Everyday life

Develop a preliminary suitability brief that addresses the following sections. Use the worked example from this lesson as a model.

Required sections:

  1. Human problem – describe the current situation, evidence of need, affected stakeholders, and strengths to preserve.
  2. Legitimate purpose – articulate the lawful, ethical, and clear reason for the activity.
  3. Intended outcome – specify what should improve, for whom, and how success would be observed.
  4. Intended user – who will directly use the AI system or its outputs?
  5. Affected stakeholders – who else is affected, including burden bearers?
  6. Current or non‑AI process – what is currently done, or what would be done without AI?
  7. Proposed AI role – what would the AI do? (e.g., generate, summarise, recommend, etc.)
  8. Activities within scope – what specific tasks would the AI handle?
  9. Activities outside scope – what remains exclusively human?
  10. Required information – what information would the AI need, and do you have authority to use it?
  11. Initial suitability factors – assess the task against the suitability criteria (clarity, purpose, information, consequence, verifiability, reversibility, scale, stakeholders, expertise, safeguards).
  12. Questions still requiring answers – what additional information is needed?
  13. Preliminary conclusion – proceed, proceed with safeguards, redesign, use a non‑AI alternative, or defer.

Deliverable: Submit a written brief (approximately 1,000–1,200 words or an equivalent structured document).

Privacy and integrity boundaries:
  • Use only synthetic, public, or authorised information.
  • Do not use real personal, confidential, or proprietary information.
  • Do not use a use case that involves a high‑impact decision (e.g., medical, legal, financial, safety‑critical) unless you have the appropriate expertise and authority.
  • Do not submit a completed personal reflection or personal AI charter as part of this brief.
Show self‑review checklist
  • ☐ I have defined the human problem without naming a specific AI product.
  • ☐ My purpose statement is lawful, ethical, relevant, clear, proportionate, and consistent with human authority.
  • ☐ I have identified intended outcomes and success criteria that go beyond speed or volume.
  • ☐ I have considered both beneficiaries and burden bearers.
  • ☐ I have compared the AI approach with at least two non‑AI alternatives.
  • ☐ I have assessed suitability using the criteria from the lesson.
  • ☐ I have identified what is outside the scope of AI.
  • ☐ I have identified missing information.
  • ☐ My preliminary conclusion is justified and proportionate.
Show solution architecture

Structure for a strong brief:

  1. Executive summary (optional): One‑paragraph overview of the use case and preliminary conclusion.
  2. Context and problem: Describe the situation with enough detail to understand the need.
  3. Purpose and intended outcomes: State the purpose clearly and define observable success criteria.
  4. Stakeholder analysis: Identify users, affected parties, and burden bearers.
  5. Alternative comparison: Briefly describe 2–3 alternatives and compare them to the AI approach.
  6. Proposed AI role and scope: Be specific about what the AI will and will not do.
  7. Suitability assessment: Walk through the suitability criteria (you can use a table or bulleted list).
  8. Missing information: What do you still need to know?
  9. Preliminary recommendation: State your recommendation and justify it.
  10. Next steps: What should happen next (e.g., gather more information, pilot the use)?

Note: The brief is preliminary — it does not replace a full risk assessment (Lesson 3.2) or oversight plan (Lesson 3.3). It is a first‑pass screening to determine whether the proposal is worth developing further.

Show illustrative partial example (different context)

Use case: A small non‑profit organisation wants to use generative AI to draft thank‑you letters to donors.

  • Human problem: The organisation's development team spends significant time drafting personalised thank‑you letters, reducing time available for donor cultivation and program work.
  • Legitimate purpose: To produce timely, personalised thank‑you letters that acknowledge donor contributions and encourage future giving, while freeing staff time for higher‑value activities.
  • Intended outcome: All donors receive a personalised thank‑you within one week of their contribution, with staff time on letters reduced by 50%.
  • Proposed AI role: Generate a draft thank‑you letter using donor information (name, amount, and any specific notes about the donation).
  • Suitability assessment: Clear purpose, authorised information (donor data with consent), reversible output (can be edited), manageable consequence (low risk).
  • Preliminary conclusion: Proceed with safeguards: human review of every letter before sending, ensure donor data is protected, and pilot the approach with a small group of donors first.

Note: This is a partial example. Your brief should include all sections listed above.

17. Self‑Check Quiz

Test your understanding of the concepts from this lesson. Each question includes feedback.

Q1. Which of the following is the best definition of effective AI use?

Select one:
Show answer and explanation

Correct answer: b)

Explanation: Effective AI use is defined by its contribution to a legitimate human purpose and the quality of its outcomes, not by speed, frequency, or the degree of automation. Option (a) encourages indiscriminate use; option (c) confuses speed with effectiveness; and option (d) assumes substitution is the goal.

Q2. Which of the following is an example of a product‑centred statement rather than a problem‑centred statement?

Select one:
Show answer and explanation

Correct answer: b)

Explanation: Statement (b) prescribes a specific technological solution (AI chatbot) without first defining the human problem. Statements (a), (c), and (d) all describe the problem or need without assuming a particular solution.

Q3. A legitimate human purpose should be all of the following except:

Select one:
Show answer and explanation

Correct answer: c)

Explanation: A legitimate purpose must be lawful, ethical, clear, proportionate, and consistent with human authority. Being "technically impressive" is not a requirement — a purpose may be technically impressive but still illegitimate if it violates privacy, fairness, or professional standards.

Q4. Which of the following is not a valid reason to measure success beyond speed or output volume?

Select one:
Show answer and explanation

Correct answer: b)

Explanation: Speed is not the most important factor; accuracy, relevance, and usability matter more. A fast output that is wrong or misleading can waste time, undermine trust, and cause harm. Option (b) is a misconception.

Q5. A task is more likely to be suitable for generative AI when:

Select one:
Show answer and explanation

Correct answer: b)

Explanation: Tasks that are easy to verify and have low consequences are well‑suited for generative AI assistance. High‑stakes decisions (a), tasks requiring empathy (c), and unauthorised sensitive information (d) are generally less suitable or require strong safeguards.

Q6. Which of the following is not a non‑AI alternative to consider?

Select one:
Show answer and explanation

Correct answer: d)

Explanation: Using a different generative AI model is still an AI approach, not a non‑AI alternative. Non‑AI alternatives include human work, conventional software, improved processes, and other approaches that do not involve generative AI.

Q7. True or false: If a generative AI system can produce a plausible output, the task is suitable for AI use.

Select one:
Show answer and explanation

Correct answer: False.

Explanation: Technical capability does not establish suitability. A task may be technically possible for AI but still unsuitable due to the consequences of error, the sensitivity of information, the need for human judgment, or the lack of appropriate oversight.

Q8. Which of the following is the primary reason to begin with the human problem rather than with the AI solution?

Select one:
Show answer and explanation

Correct answer: b)

Explanation: Beginning with the human problem ensures that the purpose is clear, that alternatives are considered, and that the AI use is genuinely needed. It prevents the common error of adopting AI because it is available rather than because it serves a legitimate purpose.

18. Assignment 1 Journal Connection

Unit 3 prepares you for Assignment 1: AI Learning and Reflection Journal. The following prompts are designed to help you reflect on the concepts from this lesson and generate material for your journal entries.

📓 Reflective prompts
  1. How do you distinguish effective from efficient AI use? Provide an example from your own experience or from a scenario you encountered in this lesson.
  2. Why should a human problem be defined before selecting an AI tool? What are the risks of starting with the technology?
  3. Describe one use case from this lesson for which a non‑AI alternative may be preferable. What makes the alternative more appropriate?
  4. How does considering affected stakeholders (including burden bearers) change the way you evaluate an AI use?

Note: You are not required to submit these reflections now. They are intended to help you build material for your journal. You may wish to save your responses in a private document.

Show reflection planning checklist
  • ☐ I have described a specific example or scenario.
  • ☐ I have explained my reasoning using concepts from the lesson.
  • ☐ I have considered both opportunities and risks.
  • ☐ I have reflected on my own assumptions or biases.
  • ☐ I have identified what I have learned or how my thinking has changed.
  • ☐ I have connected the concept to my own context (educational, professional, or everyday).

19. Key Takeaways

20. Key Terms

Effective AI use
An AI‑supported activity that contributes to a legitimate human purpose and produces an outcome of adequate quality for its intended use.
Legitimate human purpose
A purpose that is lawful, ethically defensible, relevant, clear, proportionate, and consistent with human or community authority.
Intended outcome
The specific improvement or benefit that the AI‑supported activity is expected to produce for the intended users and other stakeholders.
Affected stakeholder
Any person or group who is impacted by an AI use, including both beneficiaries and burden bearers.
Burden bearer
A stakeholder who bears additional work, responsibility, or risk as a result of an AI use (e.g., human reviewers, verifiers, or those who must correct errors).
Suitability (AI suitability)
The degree to which a given task is a good fit for generative AI assistance, considering purpose, information, consequence, verifiability, reversibility, and other contextual factors.
Product‑centred statement
A proposal or description that assumes a specific technological solution (e.g., "we should use AI") without first defining the human problem.
Problem‑centred statement
A description that defines the human need, gap, or challenge independently of any particular solution.
Non‑AI alternative
A feasible approach to addressing the human problem that does not involve generative AI, such as direct human work, conventional software, or process improvement.
Preliminary recommendation
An initial decision, made after assessing purpose, alternatives, and suitability, about whether to proceed with a proposed AI use, and under what conditions.

21. Looking Ahead

In Lesson 3.2, you will build on the foundation from this lesson by assessing risk, consequence, proportionality, and alternatives. You will learn to identify what could go wrong, how serious the consequences could be, and what safeguards are proportionate to the level of risk. The human problem and suitability assessment you develop in this lesson will provide the starting point for that analysis.

After completing Lesson 3.2, you will move on to Lesson 3.3 (Human Authority, Oversight, Transparency, and Accountability) and Lesson 3.4 (Integrated Responsible‑Use Decisions and Personal Boundaries), which together complete Unit 3.

The skills you develop in Unit 3 will prepare you for Module 2: Prompt Engineering and Human‑AI Collaboration, beginning with Unit 4: Prompt Design Fundamentals.

Continue to Lesson 3.2: Risk, Consequence, Proportionality, and Alternatives Return to Unit 3 Home

22. Sources & Further Reading

Note: All references to the textbook are to the COMP 3270 course materials. Page numbers to be confirmed.

👤