* This blog post is a summary of this video.

ChatGPT for Teachers: AI's Impact on Education and Assessment

Table of Contents

Introducing ChatGPT: A Preview of its Capabilities

ChatGPT is an artificial intelligence system that has gained a lot of attention recently for its ability to generate human-like text. It was created by OpenAI and released in November 2022. With a simple prompt, ChatGPT can provide lengthy and coherent responses on a wide range of topics, from answering test questions to writing essays and articles.

To demonstrate some of ChatGPT's capabilities, we can look at examples of how it handles common educational tasks:

Answering Multiple Choice Questions: When presented with a multiple choice test question on educational technology, ChatGPT was able to select the correct answer. It seems capable of reasoning through many test-style questions across different subject areas. This could significantly impact assessment integrity if students have unmonitored access to ChatGPT during exams.

Composing Open-Ended Responses: When asked to write a response comparing technology-enhanced learning design models, ChatGPT produced a detailed essay with correct formatting, references, and domain-specific vocabulary. The depth of knowledge across many niches allows it to convincingly write on specialized topics. However, the references it cites are often fabricated rather than based on real sources.

Critiquing Peer Forum Posts: ChatGPT can also provide critical feedback on writing samples, like critiquing a classmate's online discussion post. It gives reasonably thoughtful feedback while touching on key assignment criteria. However, the critique lacks the personalized, contextual feedback a real teacher would provide.

Answering Multiple Choice Questions

When presented with authentic multiple choice questions from an educational technology course, ChatGPT scored 8 out of 10 correct. This is a surprisingly high score and concerning in terms of assessment security for tests with a multiple choice format, which are still common in many subject areas. With continued training on academic content, we can expect ChatGPT's performance on multiple choice questions to improve even further. Teachers will need to account for this risk when developing and administering assessments.

Composing Open-Ended Responses

When given an open-ended essay prompt asking to discuss technology-enhanced learning models, ChatGPT produced a relevant and logically structured response. It correctly incorporated APA formatting, academic vocabulary, and even fabricated references to support its points. While the surface-level quality is high, close inspection reveals the lack of authentic personal perspective and contextualization that a student response would demonstrate. The cited references are also fabricated rather than based on real scholarly sources, revealing its limitations.

Critiquing Peer Forum Posts

As online discussion forums are common in education, ChatGPT was asked to provide a critical peer response to another student's hypothetical forum post. The AI gave reasonably insightful feedback while meeting key assignment criteria. However, its critique lacks the nuanced, individualized perspective a real teacher or peer reviewer could provide. The evaluation is overly generic rather than situating comments within the context of the person and class environment.

How ChatGPT Works: Its Architecture and Training

Understanding ChatGPT's underlying architecture provides insight into both its capabilities and limitations. Its full name of 'Chat Generative Pre-trained Transformer' reveals key aspects of its technical core:

  • Generative - ChatGPT is designed to generate brand new text responses to prompts, not just select between pre-written options.

  • Pre-trained - It builds on a long language model 'training period' analyzing massive datasets scraped from the internet.

  • Transformer - This is a type of neural network architecture used for processing sequences like text.

When responding to a given prompt, ChatGPT analyzes the input text to understand context. It then uses probability analysis across its intimate knowledge of language patterns to determine the most likely words to generate next. This knowledge comes from digesting enormous volumes of text during its pre-training phase.

Both supervised learning (feedback on its responses) and reinforcement learning (improving responses to achieve goals) allow it to continually enhance its performance. Combined with rapid advances in computing power to support resource-intensive neural network processing, ChatGPT has reached new heights in its ability to generate human-like text.

ChatGPT's Potential Impact on Education

It's clear that ChatGPT has tremendous potential to support and disrupt education in equal measure. Students currently have free access to an AI system capable of providing detailed explanations, answering test questions, and generating original arguments on complex topics - with varying degrees of accuracy.

Where does reliable information end and creative fabrication begin? Teachers will need to carefully re-examine assessments and learning activities in light of such technology. Rote memorization and multiple choice testing seem primed for disruption given ChatGPT's skills.

More broadly, what combination of knowledge, skills and values should a contemporary education now focus on imparting? With AI systems treating information as flexible building blocks rather than static facts, higher-order thinking around critical use and interpretation of information will grow in importance.

A Case Study: Could ChatGPT Pass an Undergrad Course?

To better understand ChatGPT's capabilities within an educational context, we conducted a case study experiment:

  • We created a mock undergraduate course covering key topics in educational technology. The grading scheme included quizzes (10%), online discussion posts (20%), a technology design project (40%), and final exam (30%). These assessments aim to evaluate understanding from multiple perspectives.

  • ChatGPT was then used to attempt each part of the course without any modifications or additional training beyond what OpenAI released publicly.

  • Its responses were graded objectively based on the course rubric to quantify performance.

On the 10% quiz portion, ChatGPT scored 8 out of 10 points. This reveals a high risk for many existing test bank questions at the undergraduate level. Less formulaic assessments will be needed to evaluate true student understanding.

When responding to online discussion prompts, ChatGPT earned 11 out of 20 points. It made coherent arguments but struggled to situate comments contextually or provide credible supporting citations. There is risk for students to rely heavily on ChatGPT for more open-ended written tasks, though some level of original analysis remains important.

For the 40% design project requiring application of technology skills, ChatGPT was unable to match a student's ability to actively build digital artifacts. However, it received 9 out of 40 points for articulating a strong justification of design decisions in writing. So risk remains for misrepresenting this creative application work through skillful writing.

On the final written exam worth 30 points, ChatGPT scored 19 out of 30 points with its answer. This continues the pattern...solid understanding of concepts was demonstrated, but fabricated details and a lack of original insight reduced scoring compared to a student response.

Overall in the mock course, ChatGPT scored 47 out of 100 points. While not definitively passing based on the grading thresholds used, the AI models reasonable competence across assessment formats relative to an average undergraduate student.

Recommendations for Teachers and Next Steps

While ChatGPT does not necessarily represent an imminent existential threat to education as we know it, responsible steps should be taken to get out ahead of risks related to AI generative text capabilities:

  • Experiment with ChatGPT yourself to deeply understand its current strengths and limitations based on tasks relevant to your teaching context.

  • Continue developing assessments emphasizing formative feedback, nuanced evaluation of skill demonstration, and validation of information sources.

  • Guide students to reflect on the appropriate and ethical role AI could play in augmenting their learning rather than replacing core knowledge building.

  • Stay informed on emerging detection technologies able to identify AI-generated text with increasing accuracy.

  • Participate in professional learning on effective teaching strategies in an age of increasingly capable machine learning systems.

Rather than an all-out ban on AI assistance tools or reliance on detecting misuse after-the-fact, teachers will serve students best by cultivating project-based learning emphasizing original work. Mastering computational skills to harness technology, while anchoring efforts firmly to community values of accuracy and ethics, is key.

This balanced approach - neither demonizing nor worshiping new tools - allows progress to unfold aligned to human priorities. ChatGPT itself has incredible potential to make expert knowledge on many subjects far more accessible if used judiciously rather than as an automatic cheat sheet substitute for deeply engaged learning.

Share Your Perspective in Our ChatGPT Teacher Survey

We would greatly appreciate hearing your perspective on AI developments like ChatGPT directly from the classroom trenches. Please consider contributing 5 minutes of your precious time to complete our brief confidential ChatGPT Teacher Survey:

  • You'll have the option to share your observations on opportunities and risks posed as assessments, teaching strategies, and student learning habits evolve in an increasingly AI-influenced world.

  • The voluntary questionnaire contains about a dozen multiple choice and short answer questions.

  • We plan to analyze patterns in collective feedback without identifying individual responses. General aggregated findings will be made freely available to help guide educators at this unique juncture.

Beyond just hearing from teachers across diverse subjects and grade levels, we hope to stimulate deeper discussion around AI's emerging role in the classroom. What might future assessment models look like? How should curriculum continue adapting to new realities enabled by technology while preserving timeless basics? What attitudes will best serve students in this climate of change?

By pooling our classroom experiences and visions for education's future, the survey provides one small avenue for progress rooted in practical wisdom. We encourage you to add your voice to the conversation.

FAQ

Q: What is ChatGPT?
A: ChatGPT is an AI chatbot that uses large language models to provide natural language responses to user requests. It was created by OpenAI and released in November 2022.

Q: How could ChatGPT be used to cheat on assessments?
A: ChatGPT has shown it can answer multiple choice questions correctly, compose open-ended written responses, and even critique peer forum posts. This means it could be used by students to cheat on a variety of assessments.

Q: What are some ways teachers can detect ChatGPT cheating?
A: Emerging AI detection tools like AI content detector and GPTzero can help teachers identify text written by ChatGPT. Teachers can also use plagiarism checkers and look for hallmark signs of AI writing like erroneous references.

PreNext