A jacket costs $80 and is reduced by 25%. What is the sale price?
Lucy is 14 years old and working on percentages. She answers this question correctly and clicks Enter.
In most assessment systems, that’s the end of the story:
- Lucy answered correctly.
- Her score is updated and the result is stored.
- She is shown the next question in the pre-determined quiz.
The interaction has generated a score, but very little insight.
At Adaptemy, that same button click triggers a very different process: Within milliseconds, the Adaptemy AI engine updates mastery estimates, recalculates confidence, evaluates psychometric evidence, propagates information across related concepts, and reassesses what Lucy should see next.
So…How Much Can You Really Learn From a Question Response?
In this article, I’ll share a little about what happens under the hood with Adaptemy.
Behind a simple learner experience (Left) a multi-layered sequence of modelling is triggered under the hood (Right)
The First Layer: Item Response Theory
One of the first processes triggered inside the Adaptemy engine is Item Response Theory (IRT).
IRT is a psychometric framework that recognises that not all questions contribute equally to measurement. A difficult, highly discriminating question provides more information about learner proficiency than a straightforward recall question. Rather than treating all responses equally, IRT helps determine how much evidence a particular interaction contributes to the learner model.
If Lucy answered a question that almost every learner answers correctly, the response provides relatively little evidence about her proficiency.
If she answered a question that weaker learners consistently struggle with, the same correct response provides considerably stronger evidence that she understands the concept.
The score remains identical, but the amount of information extracted from the interaction increases dramatically.
The Second Layer: Guessing and Slippage
Educational measurement has long recognised that learner responses are imperfect observations of knowledge.
A learner may answer correctly despite lacking genuine understanding. Equally, a knowledgeable learner may occasionally answer incorrectly because of distraction, fatigue, misreading the question or simple carelessness. These phenomena are commonly referred to as guessing and slippage.
When Lucy clicks Submit, the engine does not simply assume that a correct answer equals mastery.
Instead, it evaluates the response within the context of her broader performance history.
- Has Lucy consistently demonstrated understanding of related concepts?
- Has she struggled previously?
- Is this response aligned with the rest of the evidence available?
- The objective is not to model the answer.
The outcome is to model Lucy’s knowledge
The Third Layer: Learner Mastery Estimation
At this stage, the response begins contributing directly to Lucy’s learner model.
Unlike traditional assessment systems, which store assessment records, Adaptemy maintains a dynamic representation of learner knowledge. Every interaction becomes evidence that influences the platform’s estimate of what the learner knows and does not know.
When Lucy’s response arrives, the engine updates its estimate of her mastery of percentages.
Importantly, mastery is not treated as a binary state. The platform does not simply conclude that Lucy either knows or does not know the concept. Instead, it continuously estimates the probability that mastery has been achieved based on all available evidence. This distinction is fundamental:
- The output of the interaction is not a score.
- The output is an updated learner model.
The Fourth Layer: Confidence Estimation
Knowing what the system believes about Lucy’s mastery is only part of the story. Equally important is how confident the system is in its own estimate.
At Adaptemy, we maintain two separate but related values for every concept in the learner model.
- The first is our estimate of the learner’s mastery.
- The second is our confidence in the accuracy of that estimate.
In other words, we are not only modelling what we believe Lucy knows, we are also modelling how certain we are that our own conclusion is correct.
When Lucy clicks Submit, the engine updates both values.
Suppose Lucy answers a percentages question correctly. The platform may increase its estimate that she has mastered percentages. However, if this is the first piece of evidence we have seen for that concept, our confidence in that estimate may remain relatively low. While the response is encouraging, a single interaction rarely provides enough evidence to support a high-confidence mastery decision.
As Lucy continues learning, she generates additional evidence. She may answer related questions, complete activities involving percentages in different contexts, or demonstrate understanding of prerequisite and dependent concepts. With each interaction, the platform not only refines its mastery estimate, but also increases its confidence in the accuracy of that estimate.
This distinction is critically important. Two learners may both have an estimated mastery score of 80%, but if one estimate is based on a single question and the other is based on fifty consistent interactions collected over several weeks, they should not be treated the same way. The first estimate remains uncertain. The second is supported by a substantial body of evidence.
The Fifth Layer: Bayesian Knowledge Propagation
Knowledge rarely exists in isolation.
Curricula are built upon networks of interconnected concepts. Understanding percentages depends upon understanding fractions. Understanding fractions contributes to proportional reasoning. Similar relationships exist throughout almost every subject area.
Adaptemy’s learner modelling architecture explicitly represents these relationships through knowledge structures and Bayesian propagation techniques.
When Lucy’s mastery estimate for percentages increases, the engine does not stop there.
Evidence may also be propagated to connected concepts throughout the learner model. Confidence may increase in prerequisite areas. Predictions regarding related concepts may be updated. The platform’s understanding of Lucy’s broader knowledge state becomes more refined.
A single question is now contributing evidence well beyond the concept it was originally designed to assess.
The Sixth Layer: Predictive Learner Modelling
As the learner model evolves, the platform begins using it to make predictions. Machine learning and statistical modelling techniques allow the engine to estimate future performance, identify learners at risk of struggling, and determine which instructional interventions are most likely to improve outcomes.
When Lucy clicks Submit, the platform is not only updating its understanding of what she knows today.
It is improving its predictions about what she is likely to understand tomorrow.
This information helps determine whether she is ready for more advanced material, whether additional evidence is required, or whether targeted intervention is likely to be beneficial.
The Seventh Layer: Learning About the Assessment Question Itself
The interaction is not only teaching us about Lucy: It is also teaching us about the question.
Every learner response contributes evidence regarding the psychometric quality of the assessment itself. Over time, this allows the platform to calculate learned difficulty, item discrimination, distractor effectiveness, reliability contributions, response patterns and potential sources of bias.
When Lucy answers the percentages question, her response becomes part of a growing evidence base that helps us understand whether the question is functioning as intended.
- Assessment authors may believe a question is difficult. Learner behaviour may reveal something different.
- Assessment authors may believe a distractor is effective. Learner behaviour may reveal that nobody selects it.
The platform continuously learns about both the learner and the measurement instrument being used to assess the learner.
The Eighth Layer: Instructional Decisioning
All of this activity exists for one purpose. Better instructional decisions.
As the learner model becomes more accurate, the platform becomes better at determining what should happen next.
When Lucy clicks Submit, the engine may conclude that she is ready to progress. It may determine that additional evidence is required before a mastery decision can be made. It may identify a prerequisite gap that should be addressed before introducing more advanced content.
Assessment and instruction therefore become part of the same evidence-driven cycle.
Assessment generates evidence > Evidence updates the learner model > The learner model informs instruction. > Instruction generates new evidence.
The cycle repeats continuously.
What New Questions Become Possible?
Because Adaptemy extracts significantly more information from learner interactions than traditional systems, organisations gain the ability to answer questions that would otherwise remain invisible.
About the Learner
- Has Lucy genuinely mastered the concept?
- How confident are we in that conclusion?
- Is she demonstrating a misconception?
- Is she likely guessing?
- What prerequisite knowledge may be missing?
- Is she ready to progress?
- Which intervention is most likely to improve outcomes?
About the Assessment Item
- Is the question functioning as intended?
- Is its observed difficulty aligned with its authored difficulty?
- How discriminating is the item?
- Are the distractors effective?
- Is there evidence of guessing or bias?
- How much information does the item contribute to measurement?
About the Curriculum
- Which concepts act as bottlenecks?
- Which prerequisites are most predictive of future success?
- Where are misconceptions occurring most frequently?
- Which pathways lead to the strongest learning outcomes?
- Which concepts contribute most strongly to future mastery?
One Question Response: Hundreds of Realtime Calculations
Lucy has only answered a single question: Adaptemy generated a cascade of evidence-driven updates across a sophisticated learner modelling architecture.
Within milliseconds, the interaction contributed to mastery estimation, confidence estimation, Bayesian knowledge propagation, predictive modelling, psychometric analysis and instructional decision-making. The platform learned more about Lucy, more about the assessment item, and more about the curriculum itself.
None of this required additional questions.
None of it required additional effort from the learner.
It simply required extracting more valueand insight from the interaction.
That is ultimately what the Adaptemy AI engine is designed to do.
Because the quality of personalisation will always depend on the quality of learner mastery estimation.
And learner mastery estimation depends on how much we can learn from every single response.
