Learning, Teaching and Leadership

Abstract

This teaching resource explores an innovative pedagogical approach for developing AI literacy in a postplagiarism era. The document outlines a method of teaching fact-checking skills by having students critically evaluate AI-generated content containing deliberate errors. It provides practical guidance for educators on creating content with strategic inaccuracies, structuring verification activities, teaching source evaluation through a 5-step process, understanding AI error patterns, and implementing these exercises throughout courses. By engaging students in systematic verification processes, this approach helps develop metacognitive awareness, evaluative judgment, and appropriate skepticism when consuming AI-generated information. The resource emphasizes assessing students on their verification process rather than solely on error detection, preparing them to navigate an information landscape where distinguishing fact from fiction is increasingly challenging yet essential.

Here is a downloadable .pdf of this teaching activity:

Introduction

In a postplagiarism era, one of the most valuable skills we can teach students is how to critically evaluate AI-generated content. This can help them to cultivate meta-cognition and evaluative judgement, which have been identified as important skills for feedback and evaluation (e.g., Bearman and Luckin, 2020; Tai et al., 2018). Gen AI tools present information with confidence, regardless of accuracy. This characteristic creates an ideal opportunity to develop fact-checking competencies that serve students throughout their academic and professional lives.

Creating Content with Strategic Errors

Begin by generating content through an AI tool that contains factual inaccuracies. There are several approaches to ensure errors are present:

• Ask the AI about obscure topics where it lacks sufficient training data
• Request information about recent events beyond its knowledge cutoff
• Pose questions about specialized fields with technical terminology
• Combine legitimate questions with subtle misconceptions in your prompts

For example, ask a Large Language Model (LLM), such as ChatGPT (or any similar tool) to ‘Explain the impact of the Marshall-Weaver Theory on educational psychology’. There is no such theory, at least to the best of my knowledge. I have fabricated it for the purposes of illustration. The GenAI will likely fabricate details, citations, and research.

Structured Verification Activities

Provide students with the AI-generated content and clear verification objectives. Structure the fact-checking process as a systematic investigation.

First, have students highlight specific claims that require verification. This focuses their attention on identifying testable statements versus general information.

• Next, assign verification responsibilities using different models:
• Individual verification where each student investigates all claims
• Jigsaw approach where students verify different sections then share findings
• Team-based verification where groups compete to identify the most inaccuracies

Require students to document their verification methods for each claim. This documentation could include:

• Sources consulted
• Search terms used
• Alternative perspectives considered
• Confidence level in their verification conclusion

Requiring students to document how they verified each claim can help them develop meta-cognitive awareness about their own learning and experience how GenAI’s outputs should be treated with some skepticism and gives them specific strategies to verify content for themselves.

Teaching Source Evaluation: A 5-Step Process

The fact-checking process creates a natural opportunity to reinforce source evaluation skills.

As teachers, we can guide students to follow a 5-step plan to learn how to evaluate the reliability, truthfulness, and credibility of sources.

• Step 1: Distinguish between primary and secondary sources. (A conversation about how terms such as ‘primary source’ and ‘secondary source’ can mean different things in different academic disciplines could also be useful here.)
• Step 2: Recognize the difference between peer-reviewed research and opinion pieces. For opinion pieces, editorials, position papers, essays, it can be useful to talk about how these different genres are regarded in different academic subject areas. For example, in the humanities, an essay can be considered an elevated form of scholarship; however, in the social sciences, it may be considered less impressive than research that involves collecting empirical data from human research participants.
• Step 3: Evaluate author credentials and institutional affiliations. Of course, we want to be careful about avoiding bias when doing this. Just because an author may have an affiliation with an ivy league university, for example, that does not automatically make them a credible source. Evaluating credentials can — and should — include conversations about avoiding and mitigating bias.
• Step 4: Identify publication date and relevance. Understanding the historical, social, and political context in which a piece was written can be helpful.
• Step 5: Consider potential biases in information sources. Besides bias about an author’s place of employment, consider what motivations they may have. This can include a personal or political agenda, or any other kind of motive. Understanding a writer’s biases can help us evaluate the credibility of what they write.

Connect these skills to your subject area by discussing authoritative sources specific to your field. What makes a source trustworthy in history differs from chemistry or literature.

Understanding Gen AI Error Patterns

One valuable aspect of this exercise goes beyond identifying individual errors to recognizing patterns in how AI systems fail. As educators, we can facilitate discussions about:

• Pattern matching versus genuine understanding
• How training data limitations affect AI outputs
• The concept of AI ‘hallucination’ and why it occurs
• Why AI presents speculative information as factual
• How AI systems blend legitimate information with fabricated details

Connect these skills to your subject area by discussing authoritative sources specific to your field. What makes a source trustworthy in history differs from chemistry or literature.

Practical Implementation

Integrate these fact-checking exercises throughout your course rather than as a one-time activity. Start with simple verification tasks and progress to more complex scenarios. Connect fact-checking to course content by using AI-generated material related to current topics.

Assessment should focus on the verification process rather than simply identifying errors. Evaluate students on their systematic approach, source quality, and reasoning—not just error detection.

As AI-generated content becomes increasingly prevalent, fact-checking skills are an important academic literacy skill. By teaching students to approach information with appropriate skepticism and verification methods, we prepare them to navigate a postplagiarism landscape where distinguishing fact from fiction becomes both more difficult and more essential.

References

Eaton, S. E. (2023). Postplagiarism: Transdisciplinary ethics and integrity in the age of artificial intelligence and neurotechnology. International Journal for Educational Integrity, 19(1), 1-10. https://doi.org/10.1007/s40979-023-00144-1

Edwards, B. (2023, April 6). Why ChatGPT and Bing Chat are so good at making things up. Arts Technica. https://arstechnica.com/information-technology/2023/04/why-ai-chatbots-are-the-ultimate-bs-machines-and-how-people-hope-to-fix-them/

Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: enabling students to make decisions about the quality of work. Higher Education, 76(3), 467-481. https://doi.org/10.1007/s10734-017-0220-3

Disclaimer: This content is crossposted from: https://postplagiarism.com/2025/04/23/teaching-fact-checking-through-deliberate-errors-an-essential-ai-literacy-skill/

________________________

Share this post: Teaching Fact-Checking Through Deliberate Errors: An Essential AI Literacy Skill – https://drsaraheaton.com/2025/04/23/teaching-fact-checking-through-deliberate-errors-an-essential-ai-literacy-skill/

Sarah Elaine Eaton, PhD, is a Professor and Research Chair in the Werklund School of Education at the University of Calgary, Canada. Opinions are my own and do not represent those of my employer.