Adaptive Diagnostics

Why run a diagnostic?

A 40-hour course might have 80+ concepts. A student who already knows half the material shouldn't have to prove it by grinding through 40 hours of basics. The diagnostic solves this by asking a small number of strategically chosen questions and using the knowledge graph to infer mastery across related concepts.

The result is a personalized starting point. After the diagnostic, the student's knowledge frontier reflects what they actually know, not an assumption that they know nothing.

The BKT engine

The diagnostic uses the same Bayesian Knowledge Tracing model as the learning engine. Each concept starts with a prior P(Learned) and is updated after every response using Bayes' rule with slip and guess parameters.

The key difference from normal learning sessions: the diagnostic doesn't teach. There are no instructions, worked examples, or feedback. It's purely assessment — the goal is to gather information as efficiently as possible.

Update mechanics

For each question answered, the engine:

• - Computes the posterior P(Learned | response) using Bayes' rule with the concept's slip and guess parameters
• - Applies the learning transition P(T) — even in a diagnostic, the student might learn from the question itself
• - Propagates evidence to related concepts through the graph (see below)
• - Recalculates uncertainty across all concepts to inform the next question selection

MEPE: Maximum Expected Posterior Entropy reduction

The MEPE selector picks the next question by asking: "Which question would reduce my overall uncertainty about this student the most?" It's a greedy information-gain strategy.

How it works

For each candidate question, MEPE simulates two outcomes — correct and incorrect — and computes the expected reduction in total entropy (uncertainty) across all concepts. The question with the highest expected entropy reduction wins.

• - High-uncertainty concepts get priority: If P(Learned) is near 0.5, the system is maximally uncertain. A question about that concept is highly informative.
• - Hub concepts are extra valuable: A concept with many dependents or encompassing edges propagates more information through the graph when answered, making it a high-value target.
• - Already-certain concepts are skipped: If P(Learned) is near 0 or 1, asking about that concept won't change much. MEPE naturally avoids wasting questions on them.

# Diagnostic session state (simplified)
#
# After 5 questions, the engine has:
diagnostic_session:
  questions_answered: 5
  concept_states:
    - concept: voltage
      p_learned: 0.92        # high confidence — mastered
      entropy: 0.12
    - concept: current
      p_learned: 0.88        # high confidence — mastered
      entropy: 0.18
    - concept: ohms-law
      p_learned: 0.51        # maximum uncertainty!
      entropy: 0.69
    - concept: circuit-analysis
      p_learned: 0.35        # uncertain
      entropy: 0.63
    - concept: kirchhoffs-laws
      p_learned: 0.40        # uncertain
      entropy: 0.67

# MEPE picks ohms-law next because:
# 1. It has the highest entropy (0.69)
# 2. It has encompassing edges to voltage and current
#    (propagates evidence downward)
# 3. circuit-analysis depends on it
#    (the answer will shift circuit-analysis too)

Evidence propagation

When a student answers a question, the information doesn't just update one concept — it ripples through the knowledge graph. This is what makes the diagnostic efficient: one question can update many concepts simultaneously.

Downward propagation (correct answers)

If a student answers a question about Circuit Analysis correctly, that is evidence they also know its prerequisites. The system propagates a positive update to Ohm's Law, Voltage, and Current — weighted by the encompassing edges.

• - A correct answer on Circuit Analysis with an encompassing weight of 0.8 to Ohm's Law applies 80% of the normal positive update to Ohm's Law
• - If no encompassing edge exists but a prerequisite edge does, a smaller default credit (0.3) is applied
• - Propagation is recursive through the graph, with diminishing weight at each hop

Upward propagation (incorrect answers)

If a student gets a question wrong, that is evidence they may also struggle with dependent concepts. The system propagates a negative signal upward to concepts that list the failed concept as a prerequisite.

• - If a student fails an Ohm's Law question, Circuit Analysis (which requires Ohm's Law) receives a negative update
• - The reasoning: if you don't know the prerequisite, you probably don't know the dependent concept either
• - The negative update is smaller than the direct update — it's evidence, not proof

Stopping criteria

The diagnostic doesn't always ask the same number of questions. It stops when one of these conditions is met:

• - Hard cap (60 questions): The diagnostic never exceeds 60 questions regardless of remaining uncertainty. This prevents fatigue.
• - Diminishing returns (after 20 questions): After the 20th question, the engine checks whether fewer than 5 concepts remain uncertain (entropy > 0.5). If so, it stops — the remaining uncertainty isn't worth more questions.
• - Full convergence: If every concept has entropy below 0.3 (high confidence in either mastered or unmastered), the diagnostic stops immediately.

In practice, most diagnostics converge in 25-40 questions for a course with 80 concepts. Students who know everything finish faster (correct answers propagate broadly). Students who know nothing also finish faster (a few wrong answers quickly establish the baseline). Students with patchy knowledge take the longest because the graph boundaries are harder to map.

Speed bootstrapping with IRT

The diagnostic also estimates how fast each student learns, not just what they already know. It uses Item Response Theory (IRT) parameters — specifically response time and accuracy patterns — to initialize a per-concept learning velocity.

A student who answers quickly and correctly on foundational concepts gets a higher initial P(T) (learning rate) for related advanced concepts. This means the learning engine will require fewer practice problems before declaring mastery — the system adapts its pacing to the student's speed.

# What the diagnostic produces for the learning engine:
diagnostic_result:
  student_id: "stu_abc123"
  course_id: "electrical-fundamentals"
  questions_asked: 32
  duration_seconds: 840

  concept_states:
    - concept: voltage
      p_learned: 0.95
      mastery_state: mastered
      learning_velocity: 1.2     # learns 20% faster than average

    - concept: current
      p_learned: 0.91
      mastery_state: mastered
      learning_velocity: 1.1

    - concept: ohms-law
      p_learned: 0.78
      mastery_state: in_progress  # close but not mastered
      learning_velocity: 1.0

    - concept: circuit-analysis
      p_learned: 0.15
      mastery_state: unstarted
      learning_velocity: 0.9

    - concept: kirchhoffs-laws
      p_learned: 0.08
      mastery_state: unstarted
      learning_velocity: 0.85    # harder topic, slower expected pace

  # The student's frontier after diagnostic:
  # [ohms-law] — prerequisites mastered, not yet mastered itself
  # voltage and current are already mastered
  # circuit-analysis and kirchhoffs-laws are blocked by ohms-law

Diagnostic configuration

Diagnostics are enabled by default for all courses. You can customize the behavior in the course YAML:

course:
  id: electrical-fundamentals
  name: Electrical Fundamentals
  estimatedHours: 12
  version: "2026.1"
  diagnostic:
    enabled: true           # default: true
    minQuestions: 15         # default: 20
    maxQuestions: 45         # default: 60
    convergenceThreshold: 5 # stop after min if < N uncertain concepts
    skipForBeginners: false  # if true, skip diagnostic for brand-new topics

For courses targeting complete beginners (e.g., "Introduction to Programming"), consider setting skipForBeginners: true to skip the diagnostic entirely and start everyone at the root concepts.