Beyond Recommendation: A Critical Review of Generative AI in Educational Recommender Systems
Keywords:
generative ai, large language models, educational technology, content generation, recommender systemsAbstract
The emergence of Generative AI (GenAI) promises to shift educational recommender systems from static content curation to dynamic experience creation. But is this technical promise translating into demonstrable educational value? This paper presents a critical review, grounded in a comprehensive search of 1,223 articles across four major databases (ACM, IEEE, ScienceDirect, MDPI), which yielded only 16 foundational studies meeting our specific criteria, to argue that a significant “evaluation gap” is hindering the field’s progress. This scarcity itself is a key finding, confirming the field’s nascent state. Our analysis of this core literature reveals a field focused on technical innovation. Applications are dominated by Learning Path Recommendation (63%) and Content Generation (31%), driven by accessible techniques like Prompt Engineering (44%) and Fine-tuning (38%). These systems are successfully being reframed as generative co-pilots. However, we expose a critical misalignment: the methods used to evaluate these systems have not evolved with the technology. The literature is overwhelmingly reliant on technical and user-perception metrics (found in 82% of studies), with a near-total absence of research measuring direct improvements in student learning outcomes. This evaluation gap is the single most significant barrier to creating truly effective systems. This review’s primary contribution is the evidence-based critique of this gap. We conclude by proposing a concrete research agenda focused on validating educational efficacy, ensuring pedagogical integrity, and building trustworthy systems to guide the field from its current state of technical potential toward a future of proven learning impact.
References
Abu-Rasheed, H., Jumbo, C., Al Amin, R., Weber, C., Wiese, V., Obermaisser, R., & Fathi, M. (2025). Llm-assisted knowledge graph completion for curriculum and domain modelling in personalized higher education recommendations. In 2025 ieee global engineering education conference (educon) (pp. 1–5).
Andrade-Ruiz, G., Carrasco, R.-A., Porcel, C., Serrano-Guerrero, J., Mata, F., &
Arias-Oliva, M. (2024). Emerging perspectives on the application of recommender systems in smart cities. Electronics, 13 (7), 1249.
Barrera, F. J., Brown, E. D., Rojo, A., Obeso, J., Plata, H., Lincango, E. P., . . . Shekhar, S. (2023). Application of machine learning and artificial intelligence in the diagnosis and classification of polycystic ovarian syndrome: a systematic review. Frontiers in endocrinology, 14 , 1106625.
Chen, X., Yin, C., Chen, H., Rong, W., Ouyang, Y., & Chai, Y. (2024). Course recommendation system based on course knowledge graph generated by large language models. In 2024 ieee international conference on teaching, assessment and learning for engineering (tale) (pp. 1–8).
Chun, H. W., Ong, R. K., & Khong, A. W. (2024). Reasonable sense of direction: Making course recommendations understandable with llms. In 2024 ieee 67th international midwest symposium on circuits and systems (mwscas) (pp. 1408–1412).
Dahal, P., Nugroho, S., Schmidt, C., & Sänger, V. (2025). Ai-based learning recommendations: Use in higher education. Future Internet, 17 (7), 285.
Deepa, D., Rajkumar, T. D., & Balakrishnan, D. (2024). Optimized medical recommendation system utilizing large language models for enhanced question answering performance. In 2024 global conference on communications and information technologies (gccit) (pp. 1–5).
Durrani, U., Akpinar, M., Togher, M., Malik, A., Dordevic, M., & Aoudi, S. (2024). Harnessing ai for personalized academic major recommendations an application of large language models in education. In 2024 international conference on artificial intelligence, metaverse and cybersecurity (icamac) (pp. 1–6).
Frej, J., Dai, A., Montariol, S., Bosselut, A., & Käser, T. (2024). Course recommender systems need to consider the job market. In Proceedings of the 47th international acm sigir conference on research and development in information retrieval (pp. 522–532).
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., . . . Wang, H. (2023).
Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 , 2 (1).
Grimm, V., & Rubart, J. (2024). Authoring educational hypercomics assisted by large language models. In Proceedings of the 35th acm conference on hypertext and social media (pp. 88–97).
Harrathi, M., & Braham, R. (2021). Recommenders in improving students’ engagement in large scale open learning. Procedia Computer Science, 192 , 1121–1131.
Hirnyi, M., & Levus, Y. (2024). Educational and competitive algorithmic platform providing personalized recommendations. In 2024 ieee 19th international conference on computer science and information technologies (csit) (pp. 1–4).
Jamet, H., Manderlier, M., Shrestha, Y. R., & Vlachos, M. (2024). Evaluation and simplification of text difficulty using llms in the context of recommending texts in french to facilitate language learning. In Proceedings of the 18th acm conference on recommender systems (pp. 987–992).
Javed, U., Shaukat, K., Hameed, I. A., Iqbal, F., Alam, T. M., & Luo, S. (2021). A review of content-based and context-based recommendation systems. International Journal of Emerging Technologies in Learning (iJET), 16 (3), 274–306.
Ko, H., Lee, S., Park, Y., & Choi, A. (2022). A survey of recommendation systems: recommendation models, techniques, and application fields. Electronics, 11 (1), 141.
Krouska, A., Troussas, C., Voyiatzis, I., Mylonas, P., & Sgouropoulou, C. (2024). Chatgpt-based recommendations for personalized content creation and instructional design with a tailored prompt generator. In 2024 2nd international conference on foundation and large language models (fllm) (pp. 295–299).
Law, L. (2024). Application of generative artificial intelligence (genai) in language teaching and learning: A scoping literature review. Computers and Education Open, 100174.
Marcus, G., Davis, E., & Aaronson, S. (2022). A very preliminary analysis of dall-e 2. arXiv preprint arXiv:2204.13807 .
Neyem, A., Sandoval Alcocer, J. P., Mendoza, M., Centellas-Claros, L., Gonzalez, L. A., & Paredes-Robles, C. (2024). Exploring the impact of generative ai for standup report recommendations in software capstone project development. In Proceedings of the 55th acm technical symposium on computer science education v. 1 (pp. 951–957).
Papadakis, H., Papagrigoriou, A., Panagiotakis, C., Kosmas, E., & Fragopoulou, P. (2022). Collaborative filtering recommender systems taxonomy. Knowledge and Information Systems, 64 (1), 35–74.
Peñalvo, F. J. G., & Ingelmo, A. V. (2023). What do we mean by genai? a systematic mapping of the evolution, trends, and techniques involved in generative ai. IJIMAI , 8 (4), 7–16.
Roy, D., & Dutta, M. (2022). A systematic review and research perspective on recommender systems. Journal of Big Data, 9 (1), 59.
Seth, R., & Sharaff, A. (2022). A comparative overview of hybrid recommender systems: Review, challenges, and prospects. Data mining and machine learning applications, 57–98.
Song, Y., Wang, T., Cai, P., Mondal, S. K., & Sahoo, J. P. (2023). A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities. ACM Computing Surveys, 55 (13s), 1–40.
Valle-Cruz, D., Gil-Garcia, J. R., & Sandoval-Almazan, R. (2024). Artificial intelligence algorithms and applications in the public sector: a systematic literature review based on the prisma approach. Research Handbook on Public Management and Artificial Intelligence, 8–26.
Vishnumolakala, S. K., Sobin, C., Subheesh, N., Kumar, P., & Kumar, R. (2024). Ai-based research companion (arc): An innovative tool for fostering research activities in undergraduate engineering education. In 2024 ieee global engineering education conference (educon) (pp. 1–5).
Wahid, R., Mero, J., & Ritala, P. (2023). Written by chatgpt, illustrated by midjourney: generative ai for content marketing. Asia Pacific Journal of Marketing and Logistics, 35 (8), 1813–1822.
Wang, S. (2025). Intelligent recommendation of open education teaching resources based on hybrid collaborative recommendation algorithm with large language model. In 2025 international conference on intelligent systems and computational networks (iciscn) (pp. 1–9).
Welsby, P., & Cheung, B. M. (2023). Chatgpt (Vol. 99) (No. 1176). Oxford University Press.
Wu, W., Hu, J., Huang, Y., Zeng, W., & Shao, H. (2024). Intelligent teaching platform based on large models. In Proceedings of the 2024 international symposium on artificial intelligence for education (pp. 409–415).
Zhang, D., Zhao, X., & Zhou, G. (2024). Application of generative ai in the teaching of human resource management course for medical and health management majors. In 2024 14th international conference on information technology in medicine and education (itme) (pp. 735–739). Intelligence, 8–26.
Vishnumolakala, S. K., Sobin, C., Subheesh, N., Kumar, P., & Kumar, R. (2024). Ai-based research companion (arc): An innovative tool for fostering research activities in undergraduate engineering education. In 2024 ieee global engineering education conference (educon) (pp. 1–5).
Wahid, R., Mero, J., & Ritala, P. (2023). Written by chatgpt, illustrated by midjourney: generative ai for content marketing. Asia Pacific Journal of Marketing and Logistics, 35 (8), 1813–1822.
Wang, S. (2025). Intelligent recommendation of open education teaching resources based on hybrid collaborative recommendation algorithm with large language model. In 2025 international conference on intelligent systems and computational networks (iciscn) (pp. 1–9).
Welsby, P., & Cheung, B. M. (2023). Chatgpt (Vol. 99) (No. 1176). Oxford University Press.
Wu, W., Hu, J., Huang, Y., Zeng, W., & Shao, H. (2024). Intelligent teaching platform based on large models. In Proceedings of the 2024 international symposium on artificial intelligence for education (pp. 409–415).
Zhang, D., Zhao, X., & Zhou, G. (2024). Application of generative ai in the teaching of human resource management course for medical and health management majors. In 2024 14th international conference on information technology in medicine and education (itme) (pp. 735–739)
