AI-Assisted Analysis, Interpretation, and Research Ethics

1. Lesson Overview

This lesson moves from literature work to the analysis and interpretation of research data. You will learn to distinguish between data preparation, analysis, interpretation, and conclusion—and to identify where generative AI may legitimately assist and where it must be excluded or tightly constrained. The emphasis is on preserving human interpretive authority, verifying AI-generated outputs (codes, themes, calculations, code), and upholding research ethics, privacy, consent, and community governance.

You will encounter automation bias (the tendency to over-trust AI outputs) and methodological drift (unplanned deviations from the approved research plan). You will also learn to recognise when analysis must stop, escalate, or be re-examined. The lesson uses mixed-methods examples—qualitative (thematic analysis) and quantitative (statistical analysis)—to illustrate appropriate and inappropriate AI roles.

2. Lesson Learning Outcomes

By the end of this lesson, you will be able to:

1. Distinguish data preparation, analysis, interpretation, and conclusion.
2. Identify appropriate and inappropriate AI roles in qualitative and quantitative analysis.
3. Verify AI-generated code, calculations, categories, themes, assumptions, and outputs.
4. Recognise automation bias and methodological drift.
5. Protect research data, participant information, and community-governed knowledge.
6. Identify ethics, consent, confidentiality, Indigenous data governance, and community-authority concerns.
7. Preserve human interpretive and methodological responsibility.
8. Apply TRACE, CLAIM, PROTECT, FAIRER, and CREATE to analysis workflows.

3. Before You Begin

• Assigned reading: Read the sections of Chapter 14 on data analysis, interpretation, and research ethics. Review Chapter 9 (Bias, Fairness, and Inclusion) and Chapter 10 (Privacy, Security, and Data Protection).
• Prerequisite knowledge: You should be able to apply TRACE and CLAIM, and you should understand the importance of verification and source authority from Lessons 13.1 and 13.2.
• No programming required: This lesson explains code and calculation verification conceptually. You will not be required to write or run code.
• No real data: All examples use synthetic data or supplied datasets. You must not upload real participant data or confidential information to AI systems.
• Optional AI tool: If you have access to an approved AI system, you may use it for low-risk tasks such as brainstorming code structure or suggesting verification checks—but do not use it on real data without explicit authorisation.
• Non-platform alternative: Every activity includes a fully supplied alternative so that you are not disadvantaged if you choose not to use an AI system.
• Research-ethics reminder: AI-assisted analysis does not replace ethics approval. Always consult your institution's research ethics board or equivalent authority.
• Privacy and confidentiality: Use only synthetic, public, supplied, or explicitly authorised information.
• Accessibility: All diagrams have text descriptions. If you need further accommodations, contact your accessibility office.
• Preparation time: Approximately 15–20 minutes for reading and retrieval practice.

4. Retrieval Practice

These questions activate knowledge from Lessons 13.1 and 13.2. Attempt each one before revealing the feedback.

5. Opening Scenario

6. Core Concepts

6.1 Data Preparation, Analysis, Interpretation, and Conclusion

These four stages form a logical progression in research:

Key principle: AI may support the first two stages (preparation and analysis) with strong safeguards, but interpretation and conclusion must remain entirely human.

6.2 Qualitative Analysis Risks

When using AI for qualitative analysis, be aware of:

• Invented quotations: AI may fabricate participant quotes or attribute statements to the wrong person.
• Decontextualisation: AI may extract a quote without its surrounding context, changing its meaning.
• Homogenisation: AI may merge distinct perspectives into a single theme, erasing minority voices.
• Lost minority perspectives: AI may over-represent dominant patterns and miss rare but important views.
• Inappropriate categories: AI may apply categories that do not fit the cultural or contextual setting.
• False thematic consensus: AI may create an illusion of agreement when there is disagreement.
• Disclosure of participant information: AI may inadvertently reproduce identifiable information.
• Cultural misinterpretation: AI may lack cultural competence, leading to erroneous labels or meanings.
• Weakened reflexivity: AI does not have a positionality, so relying on it reduces the researcher's critical self-examination.

Mitigation: Always read original transcripts, verify every theme against data, include contradictory examples, and document your reflexive decisions.

6.3 Quantitative Analysis Risks

• Inappropriate method: AI may suggest a test that does not fit the data structure or research question.
• Incorrect code: AI-generated code may contain syntax errors, logical errors, or use packages incorrectly.
• Incorrect units: Misinterpreting units (e.g., meters vs. feet) can invalidate results.
• Data leakage: Using information from the test set during training or preprocessing.
• Omitted assumptions: Failing to check normality, homogeneity of variance, etc.
• Incorrect denominator: Using the wrong denominator in rate calculations.
• p-hacking suggestions: AI may suggest multiple tests without correcting for multiple comparisons.
• False precision: Reporting many decimal places when estimates are uncertain.
• Overfitting: Creating models that fit the training data but do not generalise.
• Invalid causal claims: AI may imply causality from correlational results.
• Incorrect interpretation: Misreading statistical output (e.g., confusing p-value with effect size).
• Fabricated results: AI may generate plausible but false numbers.

Mitigation: Verify all code against known test cases, check assumptions, use multiple methods, and interpret with caution.

6.4 Code and Calculation Verification

For AI-generated code (or any code), review for:

• Correctness: Does the code implement the intended method?
• Security: Does it handle data safely (no injection, no unauthorised access)?
• Data handling: Are data read, transformed, and output correctly?
• Package and version assumptions: Are dependencies compatible?
• Reproducibility: Will the code run on another system?
• Missing values: Are missing values handled appropriately?
• Edge cases: What happens with empty input or extreme values?
• Units: Are units consistent?
• Comments: Is the code well-documented?
• Hidden transformations: Are any transformations applied silently?
• Expected outputs: Does the output match known examples?

For calculations, use independent recomputation, trusted software, known test cases, unit checks, range checks, and alternative methods.

6.5 Automation Bias

Automation bias is the tendency to over-rely on automated systems, accepting their outputs without independent verification. In research, this can lead to:

• Accepting AI-generated themes as valid without checking.
• Trusting AI-generated statistics without verifying calculations.
• Overlooking errors because the AI output appears polished.
• Deferring judgment to the AI, especially when the researcher lacks expertise.

Counteraction: Always verify AI outputs independently, maintain a critical stance, and document your verification process.

6.6 Methodological Drift

Methodological drift occurs when the actual analysis deviates from the approved or stated research method without justification. Examples:

• Adding unplanned analysis criteria after seeing results.
• Changing coding rules without documentation.
• Replacing human interpretation with AI output.
• Changing inclusion/exclusion criteria.
• Using a new dataset not originally planned.
• Accepting an AI-recommended method without justification.
• Changing the research question after viewing results.

Prevention: Document all decisions, justify any deviations, and maintain version control of methods. If drift is significant, re-evaluate with the research ethics board.

6.7 Research Ethics, Consent, and Data Governance

Key principles:

• Respect for persons: Participants must give informed consent, and their autonomy must be respected.
• Concern for welfare: Research must not cause harm; benefits must outweigh risks.
• Justice: Fair distribution of benefits and burdens; avoid exploitation.
• Consent: Participants must understand how their data will be used, including AI processing.
• Privacy and confidentiality: Protect identifiable information; use de-identification, but recognise it is not foolproof.
• Data governance: Control who has access, how data are stored, and for how long.
• Indigenous data governance: Some data are governed collectively. Individual consent may not override community authority.
• AI-specific ethics: Disclose AI use, ensure data are not uploaded to unapproved systems, and respect terms of service.

Rule: Technical access to data does not create ethical authority. Always obtain proper approvals and permissions.

6.8 Reflexivity

Reflexivity is the process of examining how the researcher's assumptions, position, choices, tools, prompts, and relationships influence the research process. AI does not replace researcher reflexivity. You must ask:

• How have my own experiences and biases shaped my analysis?
• Have I considered alternative interpretations?
• How has my use of AI influenced my findings?
• What would I do differently if I were a member of the community I am studying?

Reflexivity is a human responsibility that cannot be delegated.

7. RIGOUR Walkthrough

In Lesson 13.3, we focus on the RIGOUR elements of Verification of analysis and interpretation and Method selection and methodological fit, as they relate to analysis.

8. Research-Process Walkthrough: From Authorised Data to Interpretation

This walkthrough follows a researcher through data preparation, analysis, verification, and interpretation, with AI used only in bounded ways.

Stage 1: Authorise data

Human activity: The researcher obtains ethics approval, informed consent, and community permission. Data are stored securely.

Stage 2: Data preparation

Human activity: The researcher cleans survey data (removes duplicates, checks for outliers) and transcribes interviews.

Bounded AI activity: The researcher may use AI-assisted transcription (e.g., automated speech-to-text) but must check transcripts against audio for accuracy. AI may also suggest ways to handle missing values, but the researcher decides the method.

Verification: Check a random sample of transcripts for transcription errors.

Stage 3: Qualitative analysis (thematic coding)

Human activity: The researcher reads all transcripts, develops a provisional codebook, and codes data manually.

Bounded AI activity: The researcher may ask AI: "What are some possible themes in this transcript?" and use that as a brainstorming aid. AI does not create the final codebook or assign codes.

Verification: The researcher compares AI-suggested themes with her own reading. She retains only those that are supported by data.

Stage 4: Quantitative analysis (statistical testing)

Human activity: The researcher decides to use a t-test to compare means between groups.

Bounded AI activity: The researcher may ask AI: "Write a script in R to perform a t-test on these variables." The researcher reviews the code, tests it on synthetic data, and then applies it to the real data.

Verification: The researcher independently calculates the t-statistic using a calculator and checks assumptions (normality, homogeneity of variance).

Stage 5: Interpretation

Human activity: The researcher interprets the themes and statistical results in light of the research question, theory, and context. She considers alternative explanations and acknowledges limitations.

AI role: None. Interpretation is a human responsibility. AI may suggest possible interpretations, but the researcher must critically evaluate and decide.

Stage 6: Reflexivity and documentation

Human activity: The researcher reflects on how her own position, choices, and AI use may have influenced the analysis. She documents all human decisions, AI prompts and outputs, and verification steps.

9. Worked Example

This worked example shows how a researcher applies the concepts from this lesson to a synthetic mixed-methods study.

Step 1: Data preparation

• Interviews were audio-recorded with consent.
• AI-assisted transcription: The researcher used an automated transcription service but checked every transcript against the audio for accuracy (5% error rate found and corrected).
• Survey data were cleaned: missing values were identified and handled using mean imputation for numeric variables (a decision documented).
• AI suggestion: The researcher asked AI for best practices in handling missing values; she chose mean imputation after verifying it was appropriate for her analysis.

Step 2: Qualitative analysis

• The researcher read all transcripts and developed an initial codebook based on the research question and literature.
• AI brainstorming: She asked AI to suggest themes from one transcript. AI suggested "trust," "ease of use," "accuracy concerns," and "cultural fit." The researcher compared these with her own reading and added "training needs" and "workflow integration."
• Verification: She coded all transcripts manually, then compared a sample of her codes with the AI's suggestions. She found that AI had misclassified two quotes as "trust" when they were actually "frustration." She corrected the codebook and re-coded.

Step 3: Quantitative analysis

• Hypothesis: Higher technological literacy is associated with more positive perceptions of AI tools.
• AI-assisted code: The researcher asked AI to write R code for a correlation test and a regression model. She reviewed the code, tested it on synthetic data, and confirmed it produced expected results.
• Verification: She manually calculated the correlation coefficient for a random subset and confirmed the AI's result. She checked assumptions: normality (Q-Q plots), linearity (scatterplots), and homoscedasticity (residual plots).
• Result: Significant positive correlation (r = 0.52, p < 0.01).

Step 4: Interpretation and conclusion

• Human interpretation: The researcher interpreted the quantitative result as supporting the hypothesis, but she noted that the correlation was moderate and did not imply causation. The qualitative themes added nuance: even participants with low literacy valued the tools but expressed greater concerns about accuracy.
• AI suggestion: AI suggested that "the study proves that literacy drives positive perceptions," but the researcher rejected this causal language and instead wrote: "The data show a positive association, consistent with prior research. However, qualitative findings reveal that even low-literacy users perceive benefits, suggesting that other factors are also at play."
• Limitations: The researcher acknowledged the small sample size, the rural context (limiting generalisability), and the potential for self-report bias.

Step 5: Reflexivity and documentation

• The researcher reflected on her own background (urban, highly educated) and how it might shape her interpretation of rural health workers' experiences.
• She documented all AI prompts, outputs, and her verification steps.
• She disclosed the AI use in her final report.

10. Guided Learning Activities

data = read.csv("participants.csv")
mean_age = sum(data$age) / length(data$age)
print(mean_age)

11. Exercises

Complete these eight exercises. Each includes a solution or solution design inside a details element.

12. Homework Problems

13 problems · Formative · Ungraded · Use as possible evidence for Assignment 1.

chisq.test(data$gender, data$satisfaction)

13. Mini-Project: AI-Assisted Analysis and Ethics Review

Supplied synthetic workflow

A researcher conducts a mixed-methods study on the impact of AI-powered mental health chatbots on university students. The study includes:

• 20 semi-structured interviews with students.
• A survey of 200 students measuring anxiety (GAD-7) and satisfaction.

The researcher uses an AI system to:

• Transcribe interviews.
• Generate themes from transcripts.
• Write R code to analyse survey data (correlation between chatbot usage and anxiety).
• Interpret the results and write the conclusion.

Required sections

1. Research question – state it.
2. Proposed method – describe the mixed-methods design.
3. Data type – what data were collected?
4. Data authority – who has authority over the data?
5. Ethics approval question – was approval obtained? What would be required?
6. Consent scope – did consent cover AI processing?
7. Confidentiality – how are data protected?
8. Privacy and security – where are data stored? Who has access?
9. Community or Indigenous governance – not applicable, but consider if it were.
10. AI role – what AI tasks are proposed?
11. Data-preparation tasks – what preparation is needed?
12. Analysis tasks – what analysis is planned?
13. Interpretation tasks – who will interpret?
14. Human-only tasks – what must remain human?
15. Qualitative codes or quantitative procedures – describe them.
16. Verification plan – how will you verify AI outputs?
17. Code-review plan – how will code be reviewed?
18. Calculation-review plan – how will calculations be checked?
19. Contradictory evidence – how will you handle it?
20. Bias and fairness concerns – what biases might arise?
21. Traceability – how will you document the analysis?
22. Methodological drift – what could go wrong?
23. Reproducibility questions – can the analysis be reproduced?
24. Human interpretation – what will be interpreted by the researcher?
25. Specialist-review requirement – do you need a statistician or qualitative expert?
26. Escalation – when would you escalate?
27. Stop conditions – when would you stop?
28. Residual uncertainty – what remains uncertain?
29. Preliminary recommendation – proceed, proceed with verification, redesign, or stop?

14. Self-Check Quiz

8 questions · Formative · Ungraded. Attempt each from memory before revealing the answer.

15. Assessment Connection

16. Assignment 1 Journal Connection

17. Key Takeaways

18. Key Terms

Data preparation

Cleaning, coding, formatting, and organising data for analysis.

Analysis

Applying systematic methods to generate findings from data.

Interpretation

Assigning meaning to analysis results within the research question, theory, and context.

Conclusion

Summarising implications, limitations, and recommendations.

Automation bias

The tendency to over-trust automated outputs without independent verification.

Methodological drift

Unplanned deviations from the approved or stated research method.

Reflexivity

Examining how the researcher's assumptions, choices, and position influence the research.

Qualitative analysis risks

Errors such as invented quotations, decontextualisation, homogenisation, and cultural misinterpretation.

Quantitative analysis risks

Errors such as incorrect code, inappropriate methods, false precision, and invalid causal claims.

Code verification

Checking AI-generated (or any) code for correctness, security, and reproducibility.

Calculation verification

Independently checking numerical results using alternative methods or test cases.

Research ethics

Principles governing respect, welfare, justice, consent, privacy, and confidentiality in research.

Data governance

Policies and authority controlling data collection, use, storage, and sharing.

Indigenous data governance

Collective authority over data and knowledge held by Indigenous communities.

19. Looking Ahead

20. Sources and Further Reading

• Primary: COMP 3270 Textbook, Chapter 14: Research and Scholarly Practice (sections on analysis, interpretation, and ethics).
• Supporting: Chapter 7 (TRACE), Chapter 8 (CLAIM), Chapter 9 (Bias, Fairness, and Inclusion), Chapter 10 (Privacy, Security, and Data Protection), Chapter 12 (LEARN).
• Further reading: [INSERT VERIFIED RESEARCH-METHODS REFERENCE].
• Institutional policies: Consult your institution's research ethics board, privacy office, and data-management policies.
• Indigenous research: Refer to TCPS 2 Chapter 9 and community-specific governance protocols.