Multi-Armed Bandits for Intelligent Tutoring Systems

Abstract

We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activities to maximize skills acquired by students, taking into account the limited time and motivational resources. At a given point in time, the system proposes to the students the activity which makes them progress faster. We introduce two algorithms that rely on the empirical estimation of the learning progress, RiARiT that uses information about the difficulty of each exercise and ZPDES that uses much less knowledge about the problem. The system is based on the combination of three approaches. First, it leverages recent models of intrinsically motivated learning by transposing them to active teaching, relying on empirical estimation of learning progress provided by specific activities to particular students. Second, it uses state-of-the-art Multi-Arm Bandit (MAB) techniques to efficiently manage the exploration/exploitation challenge of this optimization process. Third, it leverages expert knowledge to constrain and bootstrap initial exploration of the MAB, while requiring only coarse guidance information of the expert and allowing the system to deal with didactic gaps in its knowledge. The system is evaluated in a scenario where 7-8 year old schoolchildren learn how to decompose numbers while manipulating money. Systematic experiments are presented with simulated students, followed by results of a user study across a population of 400 school children.

References in corpus (3)

• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots
• RAPID: A Reachable Anytime Planner for Imprecisely-sensed Domains

Cited by in corpus (16)

• Combining Difficulty Ranking with Multi-Armed Bandits to Sequence Educational Content
• Safe Reinforcement Learning via Curriculum Induction
• Analysis of the apparent nuclear modification in peripheral Pb-Pb collisions at 5.02 TeV
• Accuracy-based Curriculum Learning in Deep Reinforcement Learning
• Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
• The Impact of Flow in an EEG-based Brain Computer Interface
• Improving Artificial Teachers by Considering How People Learn and Forget
• Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study
• Getting too personal(ized): The importance of feature choice in online adaptive algorithms
• Beyond A/B Testing: Sequential Randomization for Developing Interventions in Scaled Digital Learning Environments
• Meta Automatic Curriculum Learning
• Diversity-Driven Selection of Exploration Strategies in Multi-Armed Bandits
• Teaching to Learn: Sequential Teaching of Agents with Inner States
• Class Teaching for Inverse Reinforcement Learners
• Trying AGAIN instead of Trying Longer: Prior Learning for Automatic Curriculum Learning
• Prerequisite Structure Discovery in Intelligent Tutoring Systems