Skip to contentRyan Katayi
    All writing

    Notes from the work

    What counts as progress in Seuil?

    The attempt history, delayed retests and explicit rules behind Seuil's French practice planner.

    Seuil · TypeScript · Learning systems

    In a French practice session, a learner can answer correctly because the tutor has just supplied most of the sentence. That is a useful step in learning. It tells me something different from answering independently in another conversation.

    Seuil needs to choose what the learner should practice next. If both answers become the same “success” flag, the planner loses the distinction it needs to make that choice.

    I built the progress model around individual attempts, including the help the learner received.

    Store the attempt before deriving progress

    An attempt records the ability being practiced, the learner's utterance, the outcome, the support level, the scenario, the speaker and the time. Outcomes include success, repair needed, unclear and abandoned. Support can be full, light, or none.

    Those records live separately from the current readiness state for an ability. The state is a summary that the app can use quickly; the attempts preserve what happened.

    That separation matters if I change a rule. I can inspect the evidence behind a readiness decision instead of being left with a percentage whose meaning has been lost. It also makes a wrong assessment easier to investigate: which attempt changed the state and what did the learner actually say?

    The database identifies a turn by user, practice session and turn index. Recording the same turn again returns the existing attempt rather than adding another piece of evidence. Repeating a request should not make the learner appear more practiced.

    A concrete rule for readiness

    For an ability to become ready, the current attempt must succeed without support. There must also be a previous unassisted success at least twelve hours earlier, in a different scenario.

    The readiness branch in the database function makes those conditions explicit:

    if p_outcome = 'success'
      and p_support_level = 'none'
      and v_previous_success_at is not null
      and p_attempted_at - v_previous_success_at >= interval '12 hours'
      and p_scenario_key is distinct from v_previous_success_scenario then
      v_next_status := 'ready';

    Here are examples of how that rule treats successful answers:

    Attempt Result
    Correct answer after a hint Schedule a retest
    First independent success Schedule a retest
    Independent success in the same scenario Schedule a retest
    Independent success in another scenario, twenty minutes later Schedule a retest
    Independent success in another scenario, at least twelve hours after the previous one Mark ready

    The twelve-hour interval is a product rule. I don't have evidence that this threshold predicts a TEF or TCF result. Its purpose is narrower: don't promote an ability on the strength of immediate repetition alone.

    “Ready” is also a state the app can revise. It isn't a permanent certificate of mastery. A later unsuccessful attempt can move the ability back into learning and schedule another review.

    The next exercise comes from application code

    I keep exercise selection in a TypeScript planner. It ranks abilities using placement, due retests, recent errors, support levels and response times. The conversational model receives that choice as context for the turn.

    Overdue retests get priority. Immediate repair has a six-hour cooldown so a learner returning shortly after a difficult attempt doesn't automatically get pulled into the same exercise again. The planner also returns a mode and a reason, which lets me inspect why it chose an ability.

    I test the ranking with a fixed clock. That makes a statement like “this retest is overdue” an input I can control, rather than a test that changes depending on when it runs. The tests cover retest priority, repair cooldown, placement and advancement.

    The assessment is still a weak point

    Explicit progression rules can't rescue a bad assessment. Before recording feedback, the normalizer checks that it names the active ability, uses a recognized outcome and includes evidence. Construction identifiers must belong to the target set supplied for the exercise.

    Those checks can reject feedback about the wrong target. They cannot establish that a French sentence was judged correctly. An inaccurate success assessment can still affect the next session.

    Keeping the utterance and support level lets me check the original assessment when a learner gets an unexpected exercise. I can then look at how the progression rule and planner used that result.

    See Seuil's practice screens and engineering decisions.