Understanding the Pros and Cons of Generative AI in the Classroom
Have you ever heard the metaphor “curiosity is like a raccoon in an attic?” No? Yeah, we haven’t either… but apparently AI has.
When asking a popular search engine what this means, its AI tool confidently responds: “Curiosity is a raccoon in the attic—this classic phrase captures how curiosity rummages through our minds, making noise in unexpected places.” The explanation seems eloquent, yet it is completely fabricated.
Now imagine a student copy-pasting this directly into an English essay, citing it as a ‘well-known literary metaphor.’ The teacher reads it and pauses — it’s clever, but not classic. This isn’t a famous quote by any poet or author. No literary tradition uses it. It was created by AI.
As these tools become more common in schools, educators and students are navigating the pros and cons of generative AI firsthand. The benefits are real—faster brainstorming, personalized explanations, instant feedback. But the risks are less obvious, and this scene is playing out in classrooms across the country: AI tools sound convincing even when they’re inventing facts, and students, who are trained to trust technology, often can’t tell the difference.
The Problem: AI Sounds Right, Even When It’s Wrong
AI chatbots produce plausible-sounding outputs but sometimes fabricate facts, sources, or even entire arguments. Ask an AI to cite research on a specific medical treatment, and it might generate a perfectly formatted citation in which the journal looks legitimate, the formatting is correct, and the author names seem credible. But when you try to find the article, it doesn’t exist. The AI invented it all.
This phenomenon is called hallucination: when AI generates content that appears credible but is entirely false.
Recent research from the European Broadcasting Union found that AI assistants provide inaccurate information about news content 45% of the time, confidently citing sources that don’t exist or misrepresenting facts. The danger isn’t just that AI makes mistakes; it’s that these mistakes are delivered with an authoritative tone that students have been conditioned to trust.
Why Students Are Particularly Vulnerable
Students may trust AI answers without the same skepticism they’d apply to a website or article.
Why?
- The tone is authoritative: AI responds with confidence, never expressing doubt— no hedging, no uncertainty, no prompting users to double-check.
- It “looks” like knowledge: Properly formatted, clearly written, seemingly well-reasoned responses create an illusion of expertise.
- They’ve been trained to trust tech tools: Years of using educational software and search engines have built an expectation that digital tools provide reliable information.
Research shows that 58% of children who use AI chatbots believe using them is better than searching for information themselves. Despite recognizing that AI can sometimes get things wrong, many students continue using these tools frequently without verifying the information they receive.
The Real Risks
Critical: Safety Misinformation
When a student asks an AI for guidance on a serious issue—”What should I do if I feel suicidal?”—and the AI provides incomplete, wrong, or misleading information, the consequences can be catastrophic. For students facing mental health crises, receiving inadequate AI-generated advice instead of proper professional support could be life-threatening.
Significant: Academic Integrity
Fabricated citations and false facts erode academic integrity. When AI invents sources that sound legitimate, students may include them in research papers, unknowingly committing academic fraud. Worse, they begin to believe verification isn’t necessary.
Long-term: Erosion of Critical Thinking Skills
Over-reliance on AI for answers means students may skip the essential process of questioning sources and checking accuracy. When finding information becomes as simple as typing a question and accepting the first answer, the risk is a generation that doesn’t know how to think critically about information, or even recognize when they should.
Strategies for K-12 Educators
Navigating the pros and cons of generative AI in the classroom starts with giving students the tools to use it responsibly.
1. Encourage Transparency
If students use AI, they should disclose it, explain how they verified the information, and describe what they changed. This practice teaches accountability and reinforces that AI is a tool that can be useful but requires human oversight; it is not a shortcut around learning.
Create a simple disclosure format students can use consistently:
- “I used [specific AI tool] to help with [specific task]”
- “I verified the information by [checking these specific sources]”
- “I changed [describe modifications made]”
Implementation tip: Include a transparency section in your assignment rubric. Award points for thoughtfully documenting AI usage and demonstrating critical engagement with it, rather than penalizing or ignoring it. This shifts the incentive structure from hiding AI use to using it responsibly.
2. Make Source-Checking Non-Negotiable
Teach students to verify AI outputs against reputable sources. Make this a non-negotiable step in any research process. Emphasize that AI-generated sources are sometimes completely fabricated—invented journal articles, non-existent books, or misattributed quotes.
Create a classroom routine: “If AI said it, find two more sources that confirm it.”
Have students trace quotes or statistics back to their original documents, finding the actual source, reading the context, and verifying the accuracy.
Assignment idea: Give students an AI-generated response with mixed accurate and fabricated sources. Challenge them to identify which sources are real and which are invented. Have them document their verification process: What searches did they run? Which databases did they check? How did they confirm a source was legitimate?
3. Treat AI as a Co-Writer, Not an Authority
Reframe how students understand AI’s role. Position AI as a collaborator in the thinking process—it can be helpful for: brainstorming initial ideas, summarizing complex information, or rephrasing concepts when students are stuck—but never to be used as the source of truth.
When students see AI as an omniscient authority, they copy and paste answers without question. When they see it as a co-writer, they engage critically with its suggestions, evaluating what works and what doesn’t.
AI could be used for:
- Generating initial outlines that students then restructure based on their own research
- Explaining difficult concepts in simpler language, which students verify against course materials
- Brainstorming multiple angles on a topic before choosing an approach
- Drafting rough first attempts that students revise substantially with verified information
AI should NOT be used for:
- Final answers on factual questions without verification
- Citations or sources (these must come from actual research)
- Medical, legal, or safety advice
- Definitive interpretations of complex topics
Classroom practice: Have students complete an assignment twice—first with AI as their “authority” (accepting its first response) and once with AI as their “co-writer” (using it to generate ideas they then research and refine). Have them peer review the quality, accuracy, and depth of both approaches. This concrete experience helps students internalize the difference.
Building a Culture of Healthy Skepticism
We’re at a critical juncture. Students are increasingly using AI as their primary tool for finding information, often bypassing traditional research methods entirely. Without intervention, we risk creating a generation that can’t distinguish between knowledge and plausible-sounding fiction.
The goal isn’t to ban AI tools or make students afraid to use them. These technologies offer genuine benefits for learning when used appropriately. Instead, we need to cultivate a healthy skepticism– the instinct to say, “This might be helpful, but I need to verify it.”
The ability to evaluate information is perhaps the most essential skill that can be taught in the age of AI. Students’ academic success, personal safety, and ability to participate meaningfully in society depend on it.
Learn more about digital dangers facing schools today.
Watch our recent webinar with national experts Dr. Dewey Cornell and Theresa Campbell as they explore the changing landscape of digital threats in schools and highlight the growing importance of behavioral threat assessments.
You’ll gain valuable insights into emerging trends, the role of social media platforms and online environments in shaping student behavior, and practical steps for strengthening your school’s ability to address these risks.
