Abstract

This meta-analysis examined the effectiveness of artificial intelligence (AI) technologies in educational settings through a systematic review of 13 empirical studies conducted across eight countries. We analysed the impact of various AI technologies on educational outcomes using PRISMA guidelines and multiple analytical approaches, including novel applications of Naive Bayes, TF-IDF, and BERT-based algorithms. The overall analysis revealed a significant positive effect size (Hedges' g = 0.86, 95% CI [0.45, 1.27], p < 0.0001), indicating substantial benefits of AI integration in education. Particularly noteworthy were the effects of chatbots and generative AI (effect size = 1.02, 95% CI [0.45, 1.59], p < 0.0001), which demonstrated the most substantial positive impact on student learning outcomes. Online learning and virtual reality applications showed moderate positive effects (effect size = 0.79, 95% CI [-0.04, 1.62], p < 0.07) while learning management systems and AI platforms demonstrated promising but more modest impacts (effect size = 0.62, 95% CI [0.03, 1.21], p < 0.05). Although significant heterogeneity was observed across studies (I² ranging from 54.03% to 93.23%), the consistent positive effects across different educational contexts suggest the robust potential of AI technologies in enhancing educational practices. Implementing a novel weighted hybrid model, combining random and fixed effects approaches, provided additional methodological insights for analysing educational technology effectiveness. These findings provide empirical support for integrating AI technologies in educational settings while highlighting the importance of considering specific contextual factors and implementation strategies for optimal outcomes.

This work is licensed under a Creative Commons Attribution 4.0 License.