The need to enable effective teaching and learning in online environments was exemplified best over the past year as many schools had to move their classes online all over the world (Li & Lalani, 2020). AI-enabled Adaptive Learning Systems (AI-ALS) have the potential to empower teachers and improve their efficiency with repetitive tasks such as assessment and to improve the quality of their teaching, as well as to support students in achieving their potential and improve the quality of learning (Alamri et al., 2020; Chen et al., 2020; Ghergulescu et al., 2016).
This project integrated two state-of-the-art engines, Adaptemy AI Engine[1] and AlgebraKiT Engine[2], to enhance AI-ALS with modelling sub-skills (i.e., micro-evidence within a content object like steps in a question). Modelling sub-skills enables a system to empower teachers with insights into the student’s (lack of) sub-skills, to create student awareness of skills’ mastery level, and to provide better learning recommendations. Mathematics is one subject where personalised learning could be improved through sub-skill modelling, as solving a maths problem is a multi-step process that requires good conceptual knowledge and procedural skills, but many students have misconceptions and gaps that can lead to errors (Hansen et al., 2020; Feldman et al., 2018).
Adaptemy AI Engine creates and updates accurate learner models and provides multi-layered adaptation and recommendations that encapsulate effective learning strategies. Adaptemy’s AI Engine is developed based on existing research in the areas of Intelligent Tutoring Systems and Adaptive E-Learning. It makes use of a curriculum model, a content model and a learner model. The rich information from the three models enables the AI engine to personalize the learning and to accurately update the models. For each student, the Adaptemy AI Engine maintains an ability profile on all the concepts in the curriculum. The Adaptemy AI Engine contains several layers of adaptation and personalization. Through this, a system that integrates the Adaptemy AI Engine can provide immediate personalized feedback to the student, engaging content sequencing that adapts to the student’s performance, adaptive assessment and scoring, learning paths recommendations and student motivation detection and learning loops.
AlgebraKiT Engine provides a solution that evaluates each step a student does when solving a maths problem, recognizes and explains errors automatically, and offers immediate hints to the student. The engine was extended to generate sub-skill tags that describe what maths sub-skills are required to solve a problem and to detect what sub-skills are related to mistakes. The sub-skills are defined in the sub-skill taxonomy, which exists separately from AlgebraKiT’s evaluation maths engine. So, the maths engine does not use or know the contents of the sub-skill taxonomy. Instead, the engine is built around a large collection of maths rules that are applied in sequence by the engine to solve a problem. This collection of rules represents the procedures a student (should) be able to use. Maths rules and sub-skills are therefore closely related, although multiple rules can be associated with the same sub-skill.
The Adaptemy AI Engine was extended to work with a sub-skills model. Furthermore, the learner model was extended with a sub-skills profile. When the student finishes the questions, multiple sub-skills are updated based on the sub-skills evidence using a customised IRT for sub-skills. The score of the question, question discriminant and difficulty are computed as functions of the weight of the tag and question metrics.
As part of the project, we proposed a conceptual framework for extending the AI-Enabled Adaptive Learning with Sub-skills Modelling and integrated it into BuildUp Algebra Tutor platform. BuildUp Algebra Tutor incorporates the proposed framework and integrates the two AI engines: the Adaptemy AI Engine for learner modelling, adaptation and personalisation, and the AlgebraKiT Engine for sub-skill detection and step-by-step feedback.
A pilot study with 5th year students was conducted to evaluate the benefits of BuildUp Algebra Tutor. The results have showed that the step-by-step scaffolding improved the student success rate by 27.43%. The sub-skill prediction performance of the learner model is high with an AUC of up to 0.944. Moreover, survey results showed an increase in student’s self-reported metrics such as confidence. Students were also asked to rate if they liked to practice maths using traditional methods (pre-survey) and the BuildUp platform (post-survey), how easy to use they find them, and if they think they can improve their maths using traditional methods / BuildUp. A 5-point Likert scale (i.e., 1 – ‘strongly disagree’ to 5 – ‘strongly agree’) was used. The mean ratings are higher for BuildUp Algebra Tutor than traditional methods.
The proposed framework and the results of the pilot were presented at the 13th International Conference on Computer Supported Education (Ghergulescu et al., 2021), while the BuildUp Algebra Tutor integration architecture was presented at the 11th International Learning Analytics and Knowledge Conference (Ghergulescu & O’Sullivan, 2021).
[1] Adaptemy – www.adaptemy.com
[2] AlgebraKiT – https://algebrakit-learning.com/home
References:
Alamri, H. A., Watson, S., & Watson, W. (2020). Learning Technology Models that Support Personalization within Blended Learning Environments in Higher Education. TechTrends, 65(2), 62–78.
Chen, L., Chen, P., & Lin, Z. (2020). Artificial Intelligence in Education: A Review. IEEE Access, 8, 75264–75278.
Feldman, M. Q., Cho, J. Y., Ong, M., Gulwani, S., Popović, Z., & Andersen, E. (2018). Automatic Diagnosis of Students’ Misconceptions in K-8 Mathematics. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–12.
Ghergulescu, I., Flynn, C., & O’Sullivan, C. (2016). Learning Effectiveness of Adaptive Learning in Real World Context. EdMedia 2016-World Conference on Educational Media and Technology, 1391–1396.
Ghergulescu, I., Flynn,C., O’Sullivan, C., van Heck, I., Slob, M. (2021), A Conceptual Framework for Extending Domain Model of AI-enabled Adaptive Learning with Sub-skills Modelling, 13th International Conference on Computer Supported Education (CSEDU21)
Ghergulescu, I., O’Sullivan, C. (2021), Enhancing AI-enabled Adaptive Learning System with Sub-skills Modelling, The 11th International Learning Analytics and Knowledge Conference (LAK21)
Hansen, A., Drews, D., Dudgeon, J., Lawton, F., & Surtees, L. (2020). Children′s Errors in Mathematics (5th ed.). London, UK: Learning Matters, SAGE.
Li, C., & Lalani, F. (2020). The rise of online learning during the COVID-19 pandemic. World Economic Forum. https://www.weforum.org/agenda/2020/04/coronavirus-education-global-covid19-online-digital-learning/
