EduAI sounds promising, but what does the research say?
7 early findings from experiments and meta-analyses
As educators grapple with the potential and limitations of AI integration in a school context, scientific research can give us some much needed perspective and an objective measure of the effectiveness of these technologies.
Of course, these findings should be considered with caution: while researchers did not wait for the release of ChatGPT to study EduAI, the topic remains very new. Long-term effects are still hard to assess, for instance, and it is important to keep in mind that this revolution has barely begun. More powerful technologies become available by the day, and best practices around their use are still being explored.
With this in mind, early indications are that:
1) AI integration can have significant positive effects on learning outcomes such as long-term retention and reasoning ; but it does not automatically serve critical thinking [Deng & Yu (2023)] (which does not mean it cannot enhance it if intentionally designed and explicitly used for that purpose).
2) These effects are due to improvements in motivation and engagement, notably through individualization, active learning, feedback, and adapted pacing [Ait Baha et alia (2023), Chaiprasurt et alia (2022), Fernoaga et alia (2018)].
3) The benefits of AI integration increase with students’ age, and are greater for shorter and focused interventions, especially the ones automating tasks such as answering FAQs [Alemdag (2023), Wu & Yu (2023)].
4) Enduring benefits will require “humanizing” chatbots, e.g., equipping them with “emotional intelligence” features, and introducing elements of gamification [Wu & Yu (2023)].
These first 4 ideas confirm what many would have expected. However, research also indicates that:
5) AI can actually create “emotional connections” with students and enhance their self-efficacy [Zhang et alia (2022)].
It would thus be inaccurate to oppose AI-assisted and collaborative learning. AI technologies might actually enable teachers to combine the benefits of personalized and social learning. More generally, and most importantly, research also suggests that:
7) AI integration is not a zero-sum game. AI-based and human-based teaching are not at odds. Rather, AI technologies can “superpower” teachers and make it easier for them to be more effective mentors [Fernoaga et alia (2018)].
Experiment based on the introduction of an educational chatbot designed to teach a programming language to secondary schools students. Findings indicates that, compared to a class using traditional methods, AI integration enhanced students’ learning experience by allowing them to study at their own speed, with less stress, and by keeping them more motivated and engaged.
Meta-analysis of 28 studies reviewing the effects of chatbots on learning outcomes. Findings revealed a significant medium effect, with larger effects for short and focused interventions, especially those automating FAQs.
Experiment based on the introduction of an educational chatbot, Tubo, designed to assist student group work in middle school. Feedback suggested positive outcomes from the students’ and teachers’ point of view.
Experiment based on the introduction of an educational chatbot designed to enhance students’ experience and support their motivation, including through feedback. Results indicates that this communication technology helped remediate the isolation-effect of online learning environments
Meta-analysis of 32 studies finding that chatbots had a medium-to-high effect on overall learning outcomes. Particular improvements were noted in memorization, reasoning, and learning interest, but the opposite was true for critical thinking.
Experiment based on the introduction of chatbots designed to act as learning assistants and adapt to students’ varied needs and pace. Interactivity, individualization, and access to real-time data on students’ understanding supported and improved the role of teachers as mentors.
Experiment based on the introduction of an chatbot, Miguel, designed to engage in educational conversations and facilitate peer-to-peer audio feedback. 90% of students reported a desire to repeat the experience as way to demonstrate their knowledge.
Meta-analysis of 24 studies synthesizing the effects of AI chatbots on students’ learning outcomes. Overall, these tools had large positive effects - which increased with students’ age, and decreased over the course of the intervention. To go beyond the novelty effects, the authors recommend “humanizing” chatbots, equipping them with “emotional intelligence” features, and introducing elements of gamification.
Study based on the introduction of a chatbot designed to inspire and establish emotional connections with students. Results indicated heightened levels of emotional confidence and self-efficacy (belief in their own abilities as learners).