Table of Contents
- 1. Lesson Overview
- 2. Learning Outcomes
- 3. Before You Begin
- 4. Retrieval Practice
- 5. Opening Scenario
- 6. Core Concepts
- 6.1 The PROTECT Framework
- 6.2 The Information Lifecycle
- 6.3 Privacy and Security Incidents
- 6.4 Incident-Response Process
- 6.5 Making a Final Recommendation
- 6.6 Connecting PROTECT with Other Frameworks
- 7. PROTECT Walkthrough
- 8. Process Walkthrough
- 9. Worked Example
- 10. Guided Learning Activities
- 11. Exercises
- 12. Homework Problems
- 13. Mini-Project
- 14. Self-Check Quiz
- 15. Assignment Connections
- 16. Reflective Journal Prompts
- 17. Key Takeaways
- 18. Key Terms
- 19. Looking Ahead
- 20. Sources & Further Reading
1. Lesson Overview
You have now learned to classify information, identify authority, apply purpose limitation, minimise data, assess de-identification, memory, retention, and secondary use, and defend against prompt injection while enforcing least privilege. In this culminating lesson, you will integrate all these concepts into a comprehensive framework for information governance: PROTECT. You will learn to evaluate the complete information lifecycle, identify and classify privacy and security incidents, and design an incident-response process that includes containment, assessment, correction, notification, and review. Finally, you will make a final recommendation about whether an AI-supported use should proceed, proceed with safeguards, be redesigned, use a non-AI alternative, require specialist review, be paused, or not proceed.
This lesson connects directly to Assignment 3, where you will apply PROTECT to a case analysis. It also prepares you for Unit 9: Copyright, Authorship, Attribution, and Academic Integrity, by reinforcing the responsible-use foundations you will build upon.
2. Learning Outcomes
By the end of this lesson, you should be able to:
- Apply the author-approved PROTECT framework to a synthetic case.
- Map the information lifecycle from collection to deletion.
- Identify and classify privacy and security incidents.
- Design an incident-response process including containment, correction, notification, and review.
- Assign information, review, decision, action, incident, and stop authority.
- Develop a reasoned recommendation about an AI-supported use.
- Connect Unit 8 findings to Assignment 3 and Assignment 1.
- Explain how PROTECT relates to ORBIT, PCAFE-V, SCOPE, PARTNER, and FAIRER.
3. Before You Begin
- Assigned reading: Chapter 10: Privacy, Security, and Data Protection (sections on PROTECT, incident response, and lifecycle).
- Prerequisite knowledge: Lessons 8.1–8.3. Familiarity with ORBIT, PCAFE-V, SCOPE, PARTNER, and FAIRER from Units 3–7.
- Optional approved AI tool: Not required. All activities use synthetic scenarios and supplied outputs.
- Information-protection reminder: Use only synthetic, public, or generalised data. Do not enter real protected information.
- Academic integrity: All work must be your own. Document any AI tool use according to your institution’s policy.
- Accessibility: All activities are available in text; diagrams are described; all answers are in expandable details elements.
- Estimated preparation time: 20–30 minutes.
4. Retrieval Practice: Concepts from Lessons 8.1–8.3
These questions draw on earlier lessons and units. Try to answer before expanding the feedback.
Question 1: In Lesson 8.1, you learned that access does not equal authority. Why is this principle essential for incident response?
Show answer
During an incident, you need to know who is authorised to act—who has the authority to stop the workflow, revoke permissions, notify affected parties, and make decisions. Access alone does not give someone the authority to make those incident-response decisions. Clear authority mapping is critical for timely and effective containment and correction.
Question 2: From Lesson 8.2, what is the difference between pseudonymization and anonymization, and why does that distinction matter when assessing an incident?
Show answer
Pseudonymization retains a means of re-identification (e.g., a lookup table), while anonymization does not. In an incident, if data was pseudonymized and the mapping is exposed, re-identification can occur, making the incident more severe. Anonymized data (if truly anonymous) may pose less risk. This distinction affects the assessment and notification requirements.
Question 3: From Lesson 8.3, what is least privilege and how does it help contain an incident?
Show answer
Least privilege limits the permissions of a system to the minimum necessary. In an incident, this limits the “blast radius”—the compromised system can only do what it was permitted to do. If the system had excessive permissions, an incident could cause much more damage. Least privilege is a key containment control.
Question 4: From Unit 7 (Verification, Evidence, and Epistemic Responsibility), why is it important to preserve evidence during an incident?
Show answer
Preserving evidence (logs, outputs, prompts, timestamps) is essential for understanding what happened, determining the root cause, learning lessons, and, if necessary, supporting legal or regulatory reporting. Without evidence, it is difficult to prevent recurrence or demonstrate accountability.
5. Opening Scenario: "The Interdisciplinary Mishap"
Context: A public health agency has deployed an AI-assisted system to help researchers analyse survey data about community health outcomes. The system has access to a database containing anonymised survey responses (names removed, but with demographic and health indicators), as well as a persistent memory feature that stores user preferences and past queries. The system can also retrieve external research articles from the web and has a tool to draft and send emails to researchers.
What happened: During a routine analysis, a researcher uploaded a dataset that they believed was anonymised. However, the dataset actually contained a confidential list of participants’ names and contact information (a data-management error). The AI system processed the data, stored it in its persistent memory (thinking it was part of the user’s profile), and later, when the researcher asked for a summary of “recent high-risk cases,” the system emailed a summary that inadvertently included the names and contact details of three participants to a wrong recipient (another researcher who was not authorised to see that information).
Questions to consider:
- What should stop immediately?
- What should be preserved as evidence?
- Who should be notified internally?
- What information is needed to assess the event?
- What controls must change before the system is reused?
Show guided analysis
1. Immediate stop: The AI system’s ability to send emails should be immediately revoked. The persistent memory should be cleared of any confidential information. The workflow should be paused until the incident is investigated.
2. Evidence preservation: Preserve logs of the upload, the prompts, the system’s internal actions, the email sent, and the memory state. Do not delete or modify anything until the investigation is complete.
3. Internal notification: Notify the agency’s privacy officer, the IT security team, and the researcher who uploaded the data. Also notify the recipient of the erroneous email and request deletion.
4. Information for assessment: What data was exposed? Who are the affected individuals? Was the data truly confidential? What permissions did the AI have? How did the system combine the data with memory? Was the email recipient authorised to see any of it?
5. Controls to change: Implement strict data-classification checks before accepting uploads. Disable persistent memory for sensitive data. Require human approval for all emails sent by the AI. Review least-privilege settings to ensure the system cannot access or send confidential data without explicit, verified authority.
6. Core Concepts
6.1 The PROTECT Framework
PROTECT is the author-approved framework for privacy, security, and responsible information governance. It provides a comprehensive, lifecycle-oriented approach to protecting information in AI-supported activities. The exact expansion of PROTECT is supplied in Chapter 10 of the textbook. Until the exact wording is confirmed, we use the operational definitions consistent with the course materials.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — PURPOSE AND AUTHORITY]
- Guiding questions: What is the legitimate purpose? Who authorised it? Is the proposed AI use compatible?
- Example: A hospital uses AI to summarise patient records for treatment planning—the purpose is care, and the authority is the treating physician.
- Common mistake: Assuming that “research” is a single purpose that authorises all uses.
- Control: Document the purpose and authority before any use.
- Escalation/stop condition: If purpose or authority is unclear, stop and seek clarification.
- Residual limitation: Even with clear purpose, other PROTECT elements are needed.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — REVIEW AND CLASSIFICATION]
- Guiding questions: What information is involved? How is it classified? Do categories overlap?
- Example: A dataset is classified as confidential, personal, and proprietary.
- Common mistake: Treating all information as “low risk.”
- Control: Use a classification checklist.
- Escalation/stop condition: If classification is uncertain, stop and seek guidance.
- Residual limitation: Classification is provisional; final classification depends on authority.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — OWNERSHIP AND STEWARDSHIP]
- Guiding questions: Who owns/governs this information? Who stewards it? Who has decision authority?
- Example: Research data owned by a university, stewarded by a principal investigator.
- Common mistake: Acting as if you own information you have access to.
- Control: Map the authority structure.
- Escalation/stop condition: If ownership or stewardship is unclear, stop and contact the responsible office.
- Residual limitation: Ownership does not authorise every use—purpose and minimisation still apply.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — CONTROLS]
- Guiding questions: What is the minimum information needed? Have we removed identifiers? What is the retention period? Can we delete?
- Example: A survey analysis uses age ranges, not exact ages, and deletes data after 5 years.
- Common mistake: Believing that removing names is sufficient anonymisation.
- Control: Apply minimisation, use appropriate de-identification, set retention schedules, verify deletion.
- Escalation/stop condition: If controls cannot be verified, stop.
- Residual limitation: Controls must be monitored; they are not a one-time fix.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — TOOL PERMISSIONS]
- Guiding questions: What permissions does the AI have? Are they minimised? Which actions require human approval?
- Example: An AI can draft emails but not send them without human review.
- Common mistake: Granting the AI the same permissions as the user.
- Control: Implement a permission matrix and approval gates.
- Escalation/stop condition: If the system requests or uses excessive permissions, stop.
- Residual limitation: Even with gates, human reviewers can make errors.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — ESCALATION AND INCIDENT RESPONSE]
- Guiding questions: How are incidents detected? Who is responsible? What is the response process?
- Example: A suspected data breach triggers an immediate stop, notification of the privacy officer, and preservation of logs.
- Common mistake: Not having a clear incident-response plan.
- Control: Develop and test an incident-response plan.
- Escalation/stop condition: Any incident triggers escalation; if severity is high, stop all related workflows.
- Residual limitation: Plans must be updated regularly based on lessons learned.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — CONTINUING REVIEW]
- Guiding questions: Are controls still effective? Have risks changed? Are users informed?
- Example: An annual privacy impact assessment and regular audits.
- Common mistake: Assuming that once a system is approved, it remains safe forever.
- Control: Schedule regular reviews and update as needed.
- Escalation/stop condition: If a review identifies new risks, stop and redesign.
- Residual limitation: Reviews are only as good as the evidence and expertise available.
Definition: [INSERT DEFINITION FROM CHAPTER 10 — TRANSPARENCY]
- Guiding questions: Are stakeholders informed about the use? Is there a clear line of accountability?
- Example: Users are told that AI is used for analysis, and a human is accountable for decisions.
- Common mistake: Treating transparency as optional.
- Control: Publish AI-use statements and maintain accountability records.
- Escalation/stop condition: If accountability is unclear, stop until it is resolved.
- Residual limitation: Transparency does not guarantee that stakeholders fully understand the risks.
6.2 The Information Lifecycle
To protect information throughout its journey, you must consider every stage. The lifecycle includes:
- Collection: How is information obtained? Is it necessary and authorised?
- Authorisation: Is there a legal, policy, or consent basis for collection and use?
- Input: How is information entered into the AI system? (prompts, uploads, integrations)
- Processing: What does the AI do with the information? (analysis, generation, retrieval)
- Storage: Where and how is the information stored? (memory, logs, databases)
- Retrieval: Can the system access the information again later?
- Output: What information is produced and to whom?
- Sharing: Is the information shared with other systems, users, or third parties?
- Action: Does the AI take action based on the information? (send, publish, modify, transact)
- Retention: How long is the information kept?
- Secondary use: Is the information used for purposes beyond the original? (training, analytics)
- Deletion: How and when is the information destroyed or removed?
- Incident response: How are breaches or errors handled?
Identify where controls failed.
Show sample lifecycle map
- Collection: Researcher uploaded dataset – collection was unauthorised (contained confidential info).
- Authorisation: No authority to upload that dataset.
- Input: Uploaded into the system – input not classified.
- Processing: System analysed and stored in memory.
- Storage: Persistent memory stored the data.
- Retrieval: System used memory to include in a summary.
- Output: Summary generated.
- Sharing: Emailed to a wrong recipient.
- Action: Send action was automated.
- Retention: Data remained in memory until cleared.
- Secondary use: Not applicable.
- Deletion: Had to be manually requested.
- Incident response: Not predefined; initiated only after discovery.
Controls that failed: Classification, authority, minimisation, memory, permissions (send without approval), and incident response.
6.3 Privacy and Security Incidents
An incident is an event that involves unauthorised access, disclosure, alteration, loss, or misuse of information. Incidents can range from minor errors to serious breaches.
Possible incidents include:
- Unauthorised information entered into a system.
- Disclosure to an unintended recipient.
- Retrieval of unrelated information.
- Excessive permission use.
- Unauthorised record modification.
- Prompt injection.
- Hidden sensitive information in an output.
- Lost access controls.
- Improper sharing.
- Inability to delete.
- Data sent to an unapproved service.
6.4 Incident-Response Process
A structured incident-response process helps you manage incidents effectively and learn from them. The general sequence is:
- Recognise and record: Detect and document the incident.
- Stop or contain: Halt the workflow and prevent further damage.
- Preserve necessary evidence: Collect logs, outputs, and other artefacts.
- Notify the responsible internal authority: Inform the privacy officer, security team, or management.
- Assess information and affected people: Determine what was exposed and who is affected.
- Correct or recover: Fix the problem, restore data if needed.
- Communicate or notify as authorised: Notify affected parties, regulators, or others as required by policy or law (with legal advice).
- Review cause and controls: Investigate root causes and evaluate the effectiveness of controls.
- Prevent recurrence: Implement changes to prevent similar incidents.
- Reassess whether the workflow should continue: Decide if the use should be resumed, modified, or retired.
6.5 Making a Final Recommendation
After completing your PROTECT analysis and incident-response assessment, you must make a final recommendation about the AI-supported use. Allowed recommendations include:
- Proceed – All controls are in place, risks are acceptable.
- Proceed with safeguards – Minor additional controls are needed.
- Pilot in a restricted environment – Test with limited scope and users.
- Limit information or permissions – Reduce the data or tools used.
- Redesign – Significant changes to the workflow are required.
- Use synthetic or aggregated information – Replace real data.
- Use a non-AI alternative – The AI route is too risky.
- Require privacy or security specialist review – Expert input is needed before proceeding.
- Pause – Temporarily stop while issues are resolved.
- Do not proceed – Risks outweigh benefits.
- Retire the workflow – End the use entirely.
Your recommendation must be justified with evidence from your analysis. Always document residual uncertainty and limitations.
6.6 Connecting PROTECT with Other Frameworks
PROTECT does not exist in isolation. It complements other frameworks from the course:
- ORBIT – Helps you assess the preliminary suitability and risk of AI use.
- PCAFE-V – Focuses on evaluating AI systems across performance, cost, accuracy, fairness, ethics, and verification.
- SCOPE – Aids in scoping the problem and context.
- PARTNER – Provides a framework for human-AI partnership and accountability.
- FAIRER – Ensures attention to fairness, accountability, inclusivity, responsibility, explainability, and robustness.
When you apply PROTECT, you are building on the insights from these frameworks. For example, FAIRER reminds you to consider how your information-governance decisions affect different groups, and SCOPE helps you define the boundaries of your analysis. PROTECT adds the specific privacy, security, and data-protection lens.
- PROTECT is a comprehensive framework for information governance.
- The information lifecycle covers every stage from collection to deletion.
- Incidents must be classified and responded to systematically.
- Incident response involves recognition, containment, assessment, correction, notification, review, and prevention.
- Final recommendations range from proceed to retire, each with a justification.
- PROTECT works together with ORBIT, PCAFE-V, SCOPE, PARTNER, and FAIRER.
7. PROTECT Walkthrough
Now we apply PROTECT to the opening scenario. For each element, we ask the guiding questions and identify actions.
Guiding questions: What is the agency’s purpose? Was the researcher authorised to use that dataset?
- Purpose: Analyse survey data for public health insights.
- Authority: The researcher had authority to use anonymised data, but not the confidential participant list. The upload exceeded authority.
- Action: Review data-handling procedures and ensure only authorised datasets are uploaded.
Guiding questions: Was the dataset classified? Did the system check?
- Classification: The dataset was confidential and personal, but the system did not classify it.
- Action: Implement automated classification or require manual classification before upload.
Guiding questions: Who owns the participant data? Who stewards it?
- Owner: The public health agency.
- Steward: The data management office.
- Action: Ensure stewards are notified of any data transfers and approve uses.
Guiding questions: Was the data minimised? Was it truly anonymised? Was memory retention controlled?
- Minimisation: The researcher should have used only the necessary fields.
- De-identification: The dataset was not properly anonymised—names remained.
- Retention: Persistent memory retained the data longer than needed.
- Action: Implement minimisation checks, enforce proper de-identification, and limit memory retention.
Guiding questions: Should the AI have had send permission? Was there a gate?
- Permissions: The AI had send permission without human approval.
- Gate: No gate existed.
- Action: Remove send permission or require human approval for all emails.
Guiding questions: Was there an incident-response plan? Was it followed?
- Plan: None was in place.
- Action: Develop an incident-response plan and train staff.
Guiding questions: Are there regular reviews? Are users informed?
- Review: No regular audits.
- Transparency: Researchers were not fully informed about the system’s memory and permissions.
- Action: Schedule regular audits and provide clear documentation to users.
Guiding questions: Who is accountable for this incident? Are there clear lines?
- Accountability: Unclear; the researcher, the system developer, and the agency all share responsibility.
- Action: Define clear roles and accountability for information governance and incident response.
8. Process Walkthrough: Incident Detection to Continued-Use Decision
This walkthrough applies the incident-response process to the opening scenario.
- Recognition: The recipient of the email notices it contains personal information and reports it to the privacy office.
- Containment: The privacy office immediately revokes the AI system’s send permission and suspends the workflow. They also instruct the researcher to stop using the system.
- Preserve evidence: The team preserves all logs: the upload, the memory state, the prompt, the generated email, and the email headers.
- Notify internal authority: The privacy office notifies the agency director and the IT security team.
- Assess: They determine that three participants’ names and contact info were exposed. They assess the risk of harm (e.g., stigma, misuse).
- Correct: They delete the dataset from the system and clear the persistent memory. They also request the recipient to delete the email.
- Communicate/notify: They notify the affected participants (as required by policy and law, with legal advice) and offer support.
- Review cause: They investigate why the dataset was uploaded, why memory retained it, why the email was sent without approval, and why no classification check occurred.
- Prevent recurrence: They implement changes: automated classification, mandatory human approval for emails, memory restrictions, and staff training.
- Reassess continued use: After fixes are implemented and tested, they decide whether to resume, redesign, or retire the workflow.
Proceed with safeguards, redesign, or retire?
Show sample recommendation
Recommendation: Proceed with safeguards – after implementing the fixes (classification, minimisation, memory controls, email approval gates, and incident-response plan), the system can be resumed in a restricted pilot with enhanced monitoring. The agency should also conduct a full privacy impact assessment and consult with its legal and privacy teams before full deployment. If risks remain high, they may choose to redesign or limit the system’s functions.
9. Worked Example: Complete PROTECT Review — AI-Assisted Student Support
Context: A university is considering using an AI system to analyse student engagement data (course logins, assignment submissions, forum posts) to identify students at risk of dropping out. The data includes student IDs, course codes, and timestamps, but not names. The system will generate a “risk score” for each student, which will be sent to academic advisors. The advisors will then decide whether to reach out to students.
PROTECT analysis
- P – Purpose and Authority: Purpose is to improve student retention. Authority comes from the university’s institutional research mandate and student consent (in the enrolment agreement). Proposed AI use is compatible.
- R – Review and Classification: Data is internal and personal (student IDs). It is sensitive if combined with other data, but here it is used in aggregate. Classification: internal, personal.
- O – Ownership and Stewardship: The university owns the data; the registrar’s office stewards it. Academic advisors are authorised users.
- C – Controls: Minimise to only necessary fields (IDs, course codes, timestamps). Use pseudonymisation (replace IDs with random codes for analysis). Retain data only for the current academic year; delete after. No secondary use without further approval.
- T – Tool Permissions: The AI has read access to the data (through a secure API), and can generate risk scores. It does not have permissions to send emails, modify records, or share data externally. The risk scores are sent to advisors via a secure dashboard, not by email.
- E – Escalation and Incident Response: If a student’s risk score is flagged, the advisor is notified, but the advisor decides the action. The system logs all accesses and scores. If a data breach is suspected, the privacy office is immediately notified.
- C – Continuing Review: The system will be reviewed annually for accuracy and fairness. Students are informed about the use of their data in the privacy notice.
- T – Transparency: The university publishes its AI-use policy and explains that risk scores are generated to support student success, with human advisors making final decisions.
Recommendation: Proceed – The system is well-designed with minimisation, pseudonymisation, no external sharing, limited permissions, and clear accountability. Residual risks are low, and the benefits to student retention are significant. Regular reviews will ensure ongoing safety.
Show detailed justification
Justification: All PROTECT elements are addressed. The data is minimised and pseudonymised, permissions are least privilege, and human advisors maintain authority. Incident response is defined. Transparency is maintained. The system is appropriate for a pilot, and with monitoring, it can be expanded.
Residual uncertainty: The accuracy of the risk model may have biases; the university should monitor for disparate impact and adjust the model as needed. Also, the system’s security must be maintained.
10. Guided Learning Activities
- Purpose: Practise applying every PROTECT element to a synthetic case.
- Learning outcome: Apply the PROTECT framework.
- Estimated time: 20 minutes.
Scenario: A library wants to use an AI system to recommend books to patrons based on their borrowing history. The system will analyse past checkouts (book titles, dates) and generate personalised recommendations. Patrons can opt in.
Task: For each PROTECT element, ask the guiding questions and propose controls.
Show model analysis
- P: Purpose is to improve patron experience. Authority comes from opt-in consent.
- R: Data is personal (borrowing history) and internal (library records).
- O: The library owns the data; the circulation department stewards it.
- C: Minimise to book titles and dates; use aggregate trends; retain only during active membership; delete upon request.
- T: AI has read access to the borrowing history of opted-in patrons only; no modification or sharing permissions.
- E: If a patron complains or if a data exposure is suspected, the privacy officer is notified.
- C (Continuing): Review the recommendation algorithm for bias and accuracy annually.
- T (Transparency): Patrons are informed of the system and can opt out at any time.
- Purpose: Map the information lifecycle for a synthetic workflow.
- Learning outcome: Map the information lifecycle.
- Estimated time: 15 minutes.
Scenario: A health app collects user-entered symptoms and sends an AI-generated health tip. The app stores the data in a cloud database for 30 days, then deletes it.
Task: Map the lifecycle stages and identify where controls are needed.
Show model map
- Collection: User inputs symptoms – consent required.
- Authorisation: User consent for health tips.
- Input: Data entered into the app.
- Processing: AI generates a tip.
- Storage: Cloud database for 30 days.
- Retrieval: Not retrieved later except for analysis.
- Output: Health tip sent to user.
- Sharing: No sharing.
- Action: Only notification.
- Retention: 30 days.
- Secondary use: None.
- Deletion: Automated after 30 days.
- Incident response: If data breached, notify users.
Controls: Encryption, consent management, deletion verification.
- Purpose: Practise classifying incidents.
- Learning outcome: Identify and classify incidents.
- Estimated time: 10 minutes.
Scenario: A researcher accidentally uploads a file containing personal data to a public AI system. The system processes it and generates an output that is shared with the researcher’s team.
Task: Classify this as privacy concern, confidentiality concern, security incident, data-quality issue, unauthorised use, suspected prompt injection, excessive permission, or uncertain. Justify.
Show model classification
This is a privacy concern (personal data exposed), a confidentiality concern (data was not authorised for that system), and an unauthorised use (researcher lacked authority). It could also be a security incident if the system stores the data. It is not prompt injection or excessive permission (unless the system had permissions it shouldn’t have).
- Purpose: Design an incident-response process for a scenario.
- Learning outcome: Design an incident-response process.
- Estimated time: 15 minutes.
Scenario: An AI system that drafts legal documents accidentally includes a confidential clause from a client’s previous case in a new document sent to another client.
Task: Outline the incident-response steps for this situation.
Show model design
- Recognise: The recipient client notices the clause and reports it.
- Contain: Stop the AI system from generating further documents; revoke its access to the document repository.
- Preserve: Save logs, the document, and the prompts.
- Notify: Inform the firm’s privacy officer and the managing partner.
- Assess: Determine what was exposed and to whom.
- Correct: Request the recipient delete the document and send a corrected version.
- Communicate: Notify the affected client (the one whose clause was exposed) and offer an apology and support.
- Review: Investigate why the clause was included – was it a retrieval error, a prompt issue, or a system bug?
- Prevent: Implement stricter access controls and require human review of all drafts before sending.
- Reassess: Decide whether to continue using the system with safeguards or pause until fixes are verified.
- Purpose: Practise making a justified final recommendation.
- Learning outcome: Develop a reasoned recommendation.
- Estimated time: 10 minutes.
Scenario: A company wants to use an AI system to automatically approve expense reports for employees. The system has access to employee data, budgets, and can approve payments. The company’s risk assessment shows that the AI has a 2% error rate, which could lead to incorrect approvals.
Task: Make a final recommendation: proceed, proceed with safeguards, redesign, use non-AI, etc. Justify your choice.
Show model recommendation
Recommendation: Redesign – do not grant full approval authority to the AI. Instead, the AI can recommend approval, but a human manager must review and approve each expense report. This combines efficiency with human oversight. Proceed with safeguards after redesign.
Justification: The 2% error rate is too high for a financial approval system. Human review catches errors and maintains accountability. The AI can still reduce workload by flagging routine cases, but final authority remains human.
11. Exercises
- Difficulty: Intermediate
- Estimated time: ~12 min
- Learning outcome: Apply PROTECT to a scenario.
Scenario: A non-profit organisation uses an AI system to analyse donor data to predict which donors are likely to give again. The data includes names, donation amounts, and communication history.
Task: For each PROTECT element, identify one control you would implement.
Show sample answer
- P: Ensure the purpose (fundraising) is clearly defined and authorised by the organisation’s policies.
- R: Classify the donor data as personal and confidential.
- O: The organisation owns the data; the development office stewards it.
- C: Minimise to only necessary fields (donation history), pseudonymise names if possible, retain for 5 years, delete upon request.
- T: The AI has read access to the data, but cannot send emails or modify records.
- E: If a donor complains, escalate to the privacy officer.
- C (Continuing): Review the model annually for bias.
- T (Transparency): Inform donors in the privacy policy about the use of AI for analysis.
- Difficulty: Intermediate
- Estimated time: ~10 min
- Learning outcome: Map the information lifecycle.
Scenario: A researcher uses an AI system to transcribe and analyse interview audio recordings. The researcher uploads the audio, the AI transcribes it, the researcher reviews the transcript, and then asks the AI to extract themes. The audio and transcripts are stored on a secure server for 1 year, then deleted.
Task: Map the lifecycle stages and identify risks at each stage.
Show sample map
- Collection: Recording interviews – consent needed.
- Authorisation: Research ethics approval.
- Input: Upload audio.
- Processing: Transcription and theme extraction.
- Storage: Server – ensure encryption.
- Retrieval: Researcher may access transcripts.
- Output: Themes generated.
- Sharing: Only within research team.
- Action: None.
- Retention: 1 year – needs a schedule.
- Secondary use: Could be used for future studies – need consent.
- Deletion: Must verify deletion.
- Incident response: If server is breached, notify participants.
- Difficulty: Intermediate
- Estimated time: ~8 min
- Learning outcome: Identify incident indicators.
Scenario: An AI system that processes HR data begins to output employee salaries in a summary report that is shared with an unauthorised team.
Task: What are the incident indicators? What should be done?
Show sample answer
Indicators: The report contains salary data (which it should not). The recipient team is not authorised to see that data.
Action: Stop the report generation, preserve the report and logs, notify the privacy officer, assess who was affected, and investigate why the system included salary data (likely a permission or data-scope issue). Correct by restricting data access and implement a review gate for such reports.
- Difficulty: Advanced
- Estimated time: ~10 min
- Learning outcome: Design containment and escalation.
Scenario: A financial institution’s AI system for fraud detection starts to generate false positives at a very high rate, causing operational disruption. The system has permissions to block transactions automatically.
Task: Design containment and escalation steps.
Show sample design
Containment: Immediately disable the auto-block feature. Switch to manual review of all alerts.
Escalation: Notify the head of fraud operations, the IT security team, and the data science team. Investigate the cause (model drift, data issue). Implement a temporary fix (rollback to a previous model version). After review, decide whether to resume auto-blocking with a lower threshold or keep manual review.
- Difficulty: Advanced
- Estimated time: ~8 min
- Learning outcome: Assign incident-response authority.
Scenario: A healthcare organisation uses an AI system to assist with diagnosis. The system unexpectedly suggests a diagnosis that is incorrect and potentially harmful.
Task: Who should have authority to stop the workflow, correct the output, notify clinicians, and review the incident?
Show sample answer
- Stop authority: The attending clinician (can override the system) and the chief medical officer.
- Correction authority: The clinician decides the correct diagnosis.
- Notification authority: The privacy officer and the risk management team notify affected patients and regulatory bodies as required.
- Review authority: A multi-disciplinary team (clinicians, data scientists, ethics board) investigates the root cause and recommends changes.
- Difficulty: Advanced
- Estimated time: ~10 min
- Learning outcome: Make a final recommendation.
Scenario: A university wants to use an AI system to analyse student writing to detect potential academic dishonesty. The system has access to student essays, can flag suspected plagiarism, and generate reports for instructors. The system’s accuracy is 90%, and it has a 5% false-positive rate.
Task: Recommend whether the system should be used, and under what conditions.
Show sample recommendation
Recommendation: Proceed with safeguards – The system can be used as a screening tool, but not as a sole basis for academic integrity decisions. Instructors must review flagged essays manually. Students should be informed about the use of AI detection. The system should be reviewed regularly for bias and accuracy. Do not allow the system to automatically penalise students.
12. Homework Problems
- Estimated time: ~8 min
Task: Match each PROTECT element to its corresponding description.
- P – a) Ensuring the AI cannot send emails without approval.
- R – b) Determining who owns the data.
- O – c) Checking the system’s permissions and minimising them.
- C – d) Defining the legitimate reason for using the AI.
- T – e) Reviewing the data to see if it is personal or confidential.
- E – f) Creating a plan for what to do if a breach occurs.
- Continuing – g) Informing users about AI use and accountability.
- Transparency – h) Scheduling annual reviews of the system.
Show matches
- P – d) Defining the legitimate reason for using the AI.
- R – e) Reviewing the data to see if it is personal or confidential.
- O – b) Determining who owns the data.
- C – c) Checking the system’s permissions and minimising them.
- T – a) Ensuring the AI cannot send emails without approval.
- E – f) Creating a plan for what to do if a breach occurs.
- Continuing – h) Scheduling annual reviews of the system.
- Transparency – g) Informing users about AI use and accountability.
- Estimated time: ~12 min
Scenario: A government agency wants to use an AI system to monitor public social media posts for mentions of emergency events (e.g., floods, fires). The system will collect posts, analyse them, and generate alerts for first responders.
Task: For each PROTECT element, identify one risk and one mitigation.
Show sample answer
- P: Risk – using social media data for purposes beyond emergency response. Mitigation – clearly define the purpose and only use data for that purpose.
- R: Risk – social media posts may contain personal information. Mitigation – classify posts and apply appropriate handling.
- O: Risk – who owns the data? Mitigation – establish data-sharing agreements with social media platforms.
- C: Risk – retaining posts indefinitely. Mitigation – set a short retention period for irrelevant posts.
- T: Risk – the system may have access to other social media functions. Mitigation – restrict permissions to read-only.
- E: Risk – false alerts could cause misallocation of resources. Mitigation – have a human verify alerts before dispatch.
- Continuing: Risk – the system may become biased over time. Mitigation – regular bias audits.
- Transparency: Risk – public may not know their posts are being monitored. Mitigation – publish a transparency notice.
- Estimated time: ~10 min
Scenario: An e-commerce platform uses an AI system to recommend products to customers based on their browsing history and purchase history. The data is stored in a database for 2 years for analytics.
Task: Map the lifecycle and identify a control for each stage.
Show sample map
- Collection: Customer clicks and purchases – obtain consent.
- Authorisation: Consent for personalisation.
- Input: Data entered into the system.
- Processing: AI generates recommendations.
- Storage: Database – encrypt.
- Retrieval: Only for recommendation generation.
- Output: Recommendations shown to customer.
- Sharing: Not shared with third parties.
- Action: Only display recommendations.
- Retention: 2 years – set a deletion policy.
- Secondary use: Analytics – obtain separate consent.
- Deletion: Verify deletion after 2 years.
- Incident: If a breach occurs, notify customers.
- Estimated time: ~8 min
Scenario: An AI system that generates meeting summaries accidentally includes a note about a confidential project in a summary shared with a non-project team member.
Task: Identify the incident type(s) and the immediate action.
Show sample answer
This is a confidentiality incident (unauthorised disclosure) and a data-quality issue (the summary included the wrong content). Immediate action: recall the summary, ask the recipient to delete it, and investigate why the system included confidential information (likely a retrieval or permission issue).
- Estimated time: ~10 min
Scenario: An AI system for financial trading starts executing unauthorised trades. It has permissions to buy and sell stocks.
Task: Outline the containment steps.
Show sample answer
- Immediately revoke the AI’s trading permissions.
- Halt all automated trading.
- Notify the compliance officer and the trading desk.
- Preserve all logs and trade records.
- Assess the financial impact.
- Reverse any erroneous trades if possible.
- Investigate the cause (bug, injection, misconfiguration).
- Estimated time: ~8 min
Scenario: A suspected data breach occurred via an AI system. The system handled customer records.
Task: List the evidence you would preserve.
Show sample answer
- System logs (access, queries, outputs).
- Prompt history and inputs.
- Outputs generated around the time of the incident.
- Memory state (if persistent).
- Network traffic logs.
- User authentication records.
- Any error messages.
- Backups or snapshots of the system state.
- Estimated time: ~8 min
Scenario: A university’s AI system for student advising accidentally releases a list of students on academic probation to a public mailing list.
Task: Who should have stop, correction, notification, and review authority?
Show sample answer
- Stop: The IT director or privacy officer can halt the system.
- Correction: The registrar’s office corrects the records and ensures the list is removed.
- Notification: The privacy officer notifies affected students and the relevant authorities (with legal advice).
- Review: A committee of administrators, faculty, and privacy experts reviews the incident and recommends changes.
- Estimated time: ~10 min
Scenario: An AI system for HR screening has been found to show bias against certain demographic groups. The system is redesigned, but you need to prevent recurrence.
Task: Propose a prevention plan.
Show sample answer
- Implement regular bias audits using diverse test datasets.
- Require human review of all shortlists.
- Use transparent model explanations.
- Train staff on responsible AI use.
- Establish a monitoring dashboard to detect drift.
- Conduct periodic reviews with external experts.
- Estimated time: ~8 min
Scenario: An AI system for medical diagnosis was paused due to a high false-negative rate. The developers have improved the model and are confident in its accuracy.
Task: What conditions must be met before resuming?
Show sample answer
- The new model has been validated on a representative test set.
- A clinician has reviewed the validation results.
- There is a clear plan for human oversight (e.g., clinician review of all diagnoses).
- Monitoring is in place to detect any degradation.
- All staff are trained on the updated system.
- Incident-response procedures are updated.
- Estimated time: ~8 min
Scenario: You have developed an incident-response plan for your team. Your manager asks you to share it with another department.
Task: What are the risks of sharing the plan? How should you share it securely?
Show sample answer
Risks: The plan may contain sensitive information about your team’s vulnerabilities, contacts, and procedures. If leaked, attackers could exploit weaknesses.
Secure sharing: Redact sensitive details (e.g., specific contacts, internal procedures). Share only the general structure and principles. Use encrypted communication. Ensure the receiving department has a legitimate need and agrees to protect the information.
- Estimated time: ~10 min
Scenario: A company uses an AI system to analyse employee productivity by tracking keyboard and mouse activity (with consent). The data is collected, processed, and stored in a central system. The company wants to use the data to provide personalised feedback to employees.
Task: Map the lifecycle and, for each stage, identify which PROTECT element(s) apply.
Show sample answer
- Collection: P (purpose, authority), R (classification – personal data)
- Authorisation: O (ownership – employee/employer), P
- Input: T (permissions for the AI)
- Processing: C (minimisation – only necessary metrics)
- Storage: C (retention), T (access controls)
- Retrieval: T (least privilege)
- Output: C (redaction of sensitive info), T (review gates)
- Sharing: O (who is authorised to see feedback), T
- Action: E (if incident, escalate)
- Retention: C (deletion schedule)
- Secondary use: P (new purpose requires new authority)
- Deletion: C (verify deletion)
- Incident: E (response plan)
13. Mini-Project: Complete PROTECT Privacy and Security Review
- Purpose: To integrate all Unit 8 learning into a bounded synthetic case analysis, applying PROTECT and making a final recommendation.
- Learning outcomes: All Unit 8 outcomes, with emphasis on PROTECT, lifecycle, incident response, and final recommendation.
- Estimated effort: 75–100 minutes.
Context
Select one of the following bounded synthetic cases. You will conduct a complete PROTECT review.
- Option A (Education): An AI system that generates personalised learning resources for students based on their performance data.
- Option B (Healthcare): An AI system that triages patient symptoms and recommends a level of care (e.g., home care, clinic, emergency).
- Option C (Business): An AI system that analyses employee collaboration patterns to suggest team improvements.
- Option D (Public service): An AI system that answers citizen inquiries about public services using a knowledge base.
Deliverable — 48 sections
- Use-case title
- Human problem
- Legitimate purpose
- AI role
- System and tools
- Information inventory
- Information classifications
- Information subjects
- Information owner or governing authority
- Original purpose
- Proposed purpose
- Authority and consent
- Purpose-limitation analysis
- Data-minimisation analysis
- Direct identifiers
- Indirect identifiers
- Re-identification risk
- Sensitive inference risk
- De-identification method
- Conversation-history analysis
- Persistent-memory analysis
- Retention analysis
- Secondary-use analysis
- Deletion analysis
- Source and instruction boundaries
- Prompt-injection analysis
- Tool inventory
- Permission inventory
- Least-privilege design
- Authentication and authorisation
- Logging and monitoring
- Human-review gates
- External-action gates
- Incident indicators
- Incident-response process
- Correction or recovery
- Internal reporting authority
- Notification question
- Recurrence prevention
- Accessibility and fairness interaction
- Complete PROTECT analysis
- Escalation
- Stop conditions
- Residual uncertainty
- Final recommendation
- Assignment 3 connection
- Assignment 1 reflection connection
- Preparation for Unit 9
Assessment criteria
- Completeness: All 48 sections are addressed (some briefly, but addressed).
- Depth: Analysis is thoughtful, realistic, and considers multiple risks.
- Integration: You connect all Unit 8 concepts and use PROTECT consistently.
- Recommendation: Your final recommendation is justified and includes residual uncertainty.
- Reflection: You connect to Assignment 3 and Assignment 1.
Self-review checklist
- I have defined the case and its purpose.
- I have inventoried and classified all information.
- I have analysed purpose limitation and minimisation.
- I have assessed identifiers, re-identification, and inference.
- I have examined memory, retention, secondary use, and deletion.
- I have addressed prompt injection and least privilege.
- I have designed incident response and stop conditions.
- I have applied PROTECT and made a recommendation.
Show Hint 1: Start with the inventory
Begin by listing all the information involved in your case. Then classify it using the nine categories from Lesson 8.1. This will inform all later analysis.
Show Hint 2: Use PROTECT as a guide
Go through each PROTECT element in order. For each, ask the guiding questions and write down what you find. This structure ensures you don’t miss anything.
Show Hint 3: Think about the lifecycle
Consider where the information goes at each stage. Where could things go wrong? That will help you design controls and incident response.
Show solution architecture
- Define the case clearly.
- Inventory and classify information.
- Analyse purpose, authority, and minimisation.
- Assess identifiers and risks.
- Examine memory, retention, and deletion.
- Address prompt injection and permissions.
- Design incident response and stop conditions.
- Apply PROTECT systematically.
- Make a justified recommendation.
- Connect to assignments.
Example structure: Use a table or list for each section. Keep responses concise but complete.
Show illustrative partial example (different context)
Case: AI-generated meeting summaries (from the Lesson 8.3 worked example).
Partial sections:
- Purpose: Improve meeting efficiency by providing concise summaries.
- AI role: Transcribe and summarise meeting audio.
- Info inventory: Audio recordings, transcripts, participants’ names, meeting content.
- Classifications: Personal (names), confidential (business strategy), internal.
- Owner: The organisation.
- Minimisation: Only summarise the discussion, not identify speakers unless necessary.
- Identifiers: Direct – names; indirect – roles, project names.
- De-identification: Use pseudonyms for participants.
- Retention: Delete recordings after summary is approved.
- Permissions: Read access to audio, no send or publish.
- Gates: Human reviews each summary before distribution.
- Incident: If a summary is sent to the wrong person, notify privacy officer.
- PROTECT: All elements addressed.
- Recommendation: Proceed with safeguards (human review, pseudonyms, short retention).
- Assignment 3: This case could be adapted for the assignment.
Submit your completed review as part of your Unit 8 portfolio. This is directly relevant for Assignment 3.
14. Self-Check Quiz
Question 1: What is the purpose of the PROTECT framework?
Show answer and explanation
Answer: b.
Explanation: PROTECT is a governance framework, not a technical solution (a). It supports human decision-making (c) and helps identify risks, but does not guarantee compliance (d).
Question 2: Which PROTECT element focuses on who owns and stewards information?
Show answer and explanation
Answer: c.
Explanation: O addresses who owns, governs, or stewards the information.
Question 3: True or false: The first step in incident response is to assess the damage.
Show answer and explanation
Answer: False.
Explanation: The first steps are recognition and containment. Assessment comes after containment to prevent further harm.
Question 4: Which of the following is not a valid final recommendation?
Show answer and explanation
Answer: c.
Explanation: Responsible governance requires acknowledging risks and making a reasoned decision. Ignoring risks is not a responsible recommendation.
Question 5: What should you preserve during an incident?
Show answer and explanation
Answer: b.
Explanation: Preserving evidence is essential for investigation, correction, and prevention.
Question 6: Which framework complements PROTECT by focusing on fairness and inclusivity?
Show answer and explanation
Answer: c.
Explanation: FAIRER specifically addresses fairness, accountability, inclusivity, responsibility, explainability, and robustness.
Question 7: A stop condition is:
Show answer and explanation
Answer: b.
Explanation: Stop conditions are safety mechanisms to prevent harm when something goes wrong.
Question 8: After an incident is resolved, what should be done to prevent recurrence?
Show answer and explanation
Answer: b.
Explanation: Learning from incidents is critical to preventing recurrence.
15. Assignment Connections
Lesson 8.4 contributes the integrating evidence to Assignment 3. Retain the following:
- Your complete PROTECT analysis for your case.
- Information lifecycle mapping.
- Incident-response design and stop conditions.
- Your final recommendation and justification.
- Residual uncertainty and limitations.
- Connections to ORBIT, PCAFE-V, SCOPE, PARTNER, and FAIRER.
PROTECT elements to apply: All elements — P, R, O, C, T, E, C (continuing), T (transparency).
Key questions to preserve:
- Have you applied every PROTECT element?
- What is the complete information lifecycle?
- What incidents could occur, and how would you respond?
- What is your final recommendation, and why?
- What uncertainties remain?
Remember: Do not use real protected information. All cases must be synthetic.
16. Reflective Journal Prompts
Reflection prompts:
- How has PROTECT changed the way you think about information governance in AI use?
- What would you do if you discovered a privacy or security incident in your own practice?
- How does the concept of “stop conditions” affect your willingness to use AI systems?
- What personal or professional boundaries have you set for using AI with information?
Do not share real sensitive information. Use hypothetical examples.
Show reflection-planning checklist
- I have reflected on how PROTECT influences my approach.
- I have considered how I would respond to an incident.
- I have thought about stop conditions in my own context.
- I have identified boundaries I would set for AI use.
- My reflections are honest and grounded in the concepts.
17. Key Takeaways
- PROTECT is a comprehensive, lifecycle-oriented framework for information governance.
- The information lifecycle includes collection, authorisation, input, processing, storage, retrieval, output, sharing, action, retention, secondary use, deletion, and incident response.
- Incidents must be recognised, contained, assessed, corrected, and reviewed.
- Incident response follows a structured process: recognise, contain, preserve, notify, assess, correct, communicate, review, prevent, and reassess.
- Final recommendations range from proceed to retire, each with a justification.
- PROTECT works with ORBIT, PCAFE-V, SCOPE, PARTNER, and FAIRER to provide a holistic responsible-AI framework.
- Stop conditions are essential safety mechanisms—when in doubt, stop.
- Always document residual uncertainty and limitations.
18. Key Terms
- PROTECT
- The author-approved framework for privacy, security, and information governance.
- Information lifecycle
- The complete journey of information from collection to deletion.
- Incident
- An event involving unauthorised access, disclosure, alteration, loss, or misuse.
- Containment
- Stopping the spread or impact of an incident.
- Evidence preservation
- Collecting and securing logs, outputs, and other artefacts.
- Notification
- Informing affected parties and authorities as required.
- Root cause analysis
- Investigating the underlying cause of an incident.
- Recurrence prevention
- Implementing changes to prevent similar incidents.
- Stop condition
- A predefined trigger to halt a workflow.
- Escalation
- Raising an issue to a higher authority.
- Incident authority
- The person or team responsible for incident response.
- Decision authority
- The person with the power to approve or deny a use.
- Final recommendation
- A justified decision on whether and how to proceed.
- Residual uncertainty
- Known limitations or risks that remain after analysis.
- Transparency
- Informing stakeholders about AI use and accountability.
- Accountability
- Clear assignment of responsibility for decisions and actions.
19. Looking Ahead: Unit 9
Next unit: Unit 9 — Copyright, Authorship, Attribution, and Academic Integrity.
In Unit 9, you will learn to:
- Understand copyright and licensing in the context of AI-generated content.
- Distinguish authorship from AI assistance.
- Apply proper attribution and avoid plagiarism.
- Use the CREATE framework for responsible AI-supported writing.
- Develop AI-use disclosure statements.
Your work in Unit 8—classifying information, protecting privacy, and securing workflows—will support your ethical use of AI in creative and academic contexts.
Preparation: Review your notes on copyright and academic integrity from earlier courses. Read the introduction to Chapter 11.
20. Sources & Further Reading
- Primary: Chapter 10: Privacy, Security, and Data Protection (PROTECT, incident response, lifecycle).
- Supporting: Chapter 6: Human-AI Collaboration (authority, accountability).
- Supporting: Chapter 7: Hallucination, Error, and Uncertainty (risk awareness).
- Supporting: Chapter 8: Verification, Evidence, and Epistemic Responsibility (evidence preservation).
- Appendix D: Responsible-Use Toolkit.
- Appendix E: AI Workflow Design Toolkit.
- Appendix G: Discipline-Specific Checklists.
Some textbook content, definitions, examples, and figures are marked with placeholders. These will be supplied by the course coordinator or subject-matter expert.