Competency Mastery Concurrency ADR

docs/openedx_learning/decisions/0002-competency-criteria-model.rst

@@ -240,11 +240,14 @@ Decision
    3. ``oel_tagging_objecttag(object_id)``
    4. ``CompetencyCriteria(oel_tagging_objecttag_id)``
    5. ``CompetencyCriteria(competency_criteria_group_id)``
-   6. ``StudentCompetencyCriteriaStatus(user_id, competency_criteria_id)``
-   7. ``StudentCompetencyCriteriaGroupStatus(user_id, competency_criteria_group_id)``
-   8. ``StudentCompetencyStatus(user_id, oel_tagging_tag_id)``
-   9. ``CompetencyRuleProfile(scope_code)`` (unique -- at most one profile per distinct scope value; a plain unique constraint on the three raw nullable scope columns would not enforce this, since SQL never treats two ``NULL`` values as equal and this project's MySQL backend does not support the conditional/partial unique indexes that would otherwise route around that; see the ``scope_code`` column in Decision 3)
-   10. ``CompetencyMasteryStatuses(status)`` (unique)
+   6. ``StudentCompetencyCriteriaStatus(user_id, competency_criteria_id)`` (unique)
+   7.  ``StudentCompetencyCriteriaStatusHistory(user_id, competency_criteria_id)``
+   8. ``StudentCompetencyCriteriaGroupStatus(user_id, competency_criteria_group_id)`` (unique)
+   9.  ``StudentCompetencyCriteriaGroupStatusHistory(user_id, competency_criteria_group_id)``
+   10. ``StudentCompetencyStatus(user_id, oel_tagging_tag_id)`` (unique)
+   11. ``StudentCompetencyStatusHistory(user_id, oel_tagging_tag_id)``
+   12. ``CompetencyRuleProfile(scope_code)`` (unique -- at most one profile per distinct scope value; a plain unique constraint on the three raw nullable scope columns would not enforce this, since SQL never treats two ``NULL`` values as equal and this project's MySQL backend does not support the conditional/partial unique indexes that would otherwise route around that; see the ``scope_code`` column in Decision 3)
+   13. ``CompetencyMasteryStatuses(status)`` (unique)
 
 6. Learner progress status concepts (``StudentCompetency*Status`` database tables)
 
@@ -257,6 +260,13 @@ Decision
    - ``StudentCompetencyStatus`` tracks top-level competency demonstration state.
    - All learner status rows use a shared lookup table (``CompetencyMasteryStatuses``) so status semantics live in one place and student status tables stay structurally consistent.
 
+   Append-only history tables:
+
+   - ``StudentCompetencyCriteriaStatusHistory``
+   - ``StudentCompetencyCriteriaGroupStatusHistory``
+   - ``StudentCompetencyStatusHistory``
+
+
    Intended update flow (bottom-up materialization):
 
    - A learner event updates one ``StudentCompetencyCriteriaStatus`` row.
@@ -275,25 +285,25 @@ Decision
 
    2. Add new database table for ``StudentCompetencyCriteriaStatus`` with these columns:
 
-      1. ``id``: unique primary key
+      1. ``id``: unique 64-bit primary key (``BigAutoField``); see :ref:`openedx-learning-adr-0005`.
       2. ``competency_criteria_id``: Foreign key to ``CompetencyCriterion.id``
-      3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
+      3. ``user_id``: Foreign key with ``db_constraint=False`` pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``
       5. ``created``: The timestamp at which the student's criterion status was set.
 
    3. Add a new database table for ``StudentCompetencyCriteriaGroupStatus`` with these columns:
 
       1. ``id``: unique primary key
       2. ``competency_criteria_group_id``: Foreign key to ``CompetencyCriteriaGroup.id``
-      3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
+      3. ``user_id``: Foreign key with ``db_constraint=False`` pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``
       5. ``created``: The timestamp at which the student's criteria-group status was set.
 
    4. Add a new database table for ``StudentCompetencyStatus`` with these columns:
 
       1. ``id``: unique primary key
       2. ``oel_tagging_tag_id``: Foreign key pointing to Tag id
-      3. ``user_id``: Foreign key pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
+      3. ``user_id``: Foreign key with ``db_constraint=False`` pointing to user_id (presumably the learner's id, although it appears that it is possible for staff to get grades as well) in ``auth_user`` table
       4. ``status_id``: Foreign key to ``CompetencyMasteryStatuses.id``. This table should have a constraint to only allow status values of “Demonstrated” and “PartiallyAttempted” since it represents overall competency demonstration state, not in-progress states.
       5. ``created``: The timestamp at which the student's competency status was set.
 

docs/openedx_learning/decisions/0003-competency-criteria-versioning.rst

@@ -44,12 +44,21 @@ For the initial implementation, versioning and traceability of competency achiev
    - A ``CompetencyRuleProfile`` is "in use" if any ``CompetencyCriterion`` assigned to it (``competency_rule_profile_id``) has an associated ``StudentCompetencyCriteriaStatus`` row. Editing an in-use profile's ``rule_type``/``rule_payload`` requires the same warning and confirmation.
    - The same warning applies when creating a more specific profile causes existing criteria to be reassigned to it, and when an authoring action switches a criterion between a profile assignment and per-criterion overrides (ADR 0002 Decision 4).
 
-5. Learner status models/tables are append-only history and do not use ``django-simple-history``:
+5. Learner status models/tables are updated in-place:
 
-   - For ``StudentCompetencyCriteriaStatus``, ``StudentCompetencyCriteriaGroupStatus``, and ``StudentCompetencyStatus``, each status change is stored as a new row with ``created`` as the write timestamp.
-   - Existing learner status rows are not updated in place.
-   - Current status is determined by the most recent row for a given learner + target entity (ordered by ``created``, with ``id`` as a tie-breaker).
-   - Older rows represent the learner status history and remain available for audit/tracing.
+   - For ``StudentCompetencyCriteriaStatus``, ``StudentCompetencyCriteriaGroupStatus``, and ``StudentCompetencyStatus``,
+     each status change updates the responsible row.
+   - Statuses only increase monotonically as described by :ref:`openedx-learning-adr-0005`;
+     downward status adjustments (for example ``Demonstrated`` to ``PartiallyAttempted``) are prohibited.
+
+6. Learner status models/tables as in 5. above each get a separate append-only history table not using ``django-simple-history``:
+
+   - For ``StudentCompetencyCriteriaStatusHistory``, ``StudentCompetencyCriteriaGroupStatusHistory``, and ``StudentCompetencyStatusHistory``,
+     each status acvance is stored as a new row with ``created`` as the write timestamp.
+   - Existing learner status rows are not updated in place in the history tables.
+   - Statuses only increase monotonically as described by :ref:`openedx-learning-adr-0005`;
+     if a change would mean a downward adjustment (for example ``Demonstrated`` to ``PartiallyAttempted``)
+     or no adjustment, this does not get stored in the history tables.
 
 
 Rejected Alternatives

docs/openedx_learning/decisions/0004-competency-mastery-concurrency.rst

@@ -0,0 +1,92 @@
+.. _openedx-learning-adr-0004:
+
+4. How should learner competency mastery be recorded concurrently and at scale?
+================================================================================
+
+Status
+------
+Proposed.
+
+Context
+-------
+When a learner is graded on a subsection (or any other learning instrument associated to a competency
+with a competency criteria, like a course or rubric criterion), the platform must evaluate whether that grade
+demonstrates any attached competencies and record the learner's mastery. Mastery is recorded at
+three levels: the criterion (leaf), the criteria group, and the competency. Per
+:ref:`openedx-learning-adr-0002` and :ref:`openedx-learning-adr-0005`, all three levels are
+*materialized* (stored), not recomputed on read, so that dashboards and other read surfaces stay
+fast. A single grade change therefore writes the changed leaf's status and then re-evaluates and
+re-writes the derived rows from that leaf up to the competency root. The re-evaluation
+is needed for multiple reasons, including notifications, and badge and certificate issuing. Per
+:ref:`openedx-learning-adr-0005`, each level is stored as an ACTIVE row updated in place, holding
+the current status for a learner and node, plus an append-only HISTORY row per genuine status
+advance.
+
+**Monotonicity: competency statuses only ever move forward.** Per
+:ref:`openedx-learning-adr-0005`, every node, at every level, advances through a small status
+lattice (``AttemptedNotDemonstrated`` to ``PartiallyAttempted`` to ``Demonstrated``) and is never
+lowered later. This holds for leaf nodes, group nodes, and top-level competency masteries.
+
+Two forces shape how recording should happen:
+
+- **Same-learner correctness.** A grade change writes the changed leaf and then re-derives the
+  group and competency rows above it. Leaf rows are always correct, since each leaf is a pure
+  function of its own grade. The derived rows are the hazard: We want to avoid a case where two evaluations for the same learner
+  that overlap can each read a stale snapshot of the sibling leaf statuses and each write a derived
+  roll-up computed from an incomplete picture (a *write-skew*).
+
+- **Throughput.** Grading is bursty and spans a very large number of learners, so the recording
+  path must keep up under peak load.
+
+Decision
+--------
+
+**1. Every write is a monotone merge, never a blind overwrite.** A node's status is written as
+``status := max(stored status, newly computed status)`` (a single ``GREATEST``-style ``UPDATE``,
+atomic at the row for the duration of that one statement, with no application-level lock). Because
+the merge takes the higher of the two values, it is commutative, idempotent, and insensitive to
+order. This is why out-of-order delivery and re-delivery are harmless without sequence tracking.
+
+**2. When a child advances, its parent is recomputed in the same transaction, under a brief row lock on that parent.**
+The merge in mechanism 1 makes a single-row write safe, but a *conjunctive*
+parent (for example "demonstrated only when all children are demonstrated") is computed by reading
+several child rows first, so two overlapping evaluations for one learner could each read a stale
+sibling and compute a parent that is too low. To prevent that, recomputing a parent takes a
+row-level lock on the parent row (a ``SELECT ... FOR UPDATE``) before reading its children: two
+updates that touch the same parent for the same learner take turns, and the second reads the first's
+committed children and computes from the complete picture. Locks are taken child-before-parent up
+the path to the root, a consistent order, so concurrent updates cannot deadlock. This is an ordinary
+single-row lock. Every read the recorder makes, here and in the mass-recompute path
+(:ref:`openedx-learning-adr-0005`), runs against the primary database and never a read replica:
+these reads feed the roll-up write and take the row locks above, so a replica's lag would compute a
+roll-up from stale siblings. The read-replica offload in :ref:`openedx-learning-adr-0005` is
+reserved for the read-only dashboard and reporting paths.
+
+**3. Entry point: edx-platform subsection grade change.** edx-platform
+computes subsection grades in an async celery task (`recalculate_subsection_grade_v3`) triggered by a score-change signal, not on the
+request thread. After that task writes the subsection grade, it calls a public openedx-core function
+within the same transaction; this function does the monotone merge and the upward roll-up. This should be generalized as needed to other places that trigger a competency status update.
+
+**4. ACTIVE writes and roll-ups commit atomically with the grade, or the whole task rolls back and retries.**
+
+**5. The leaf HISTORY append runs outside the transaction, best effort.** Because the leaf HISTORY table
+(``StudentCompetencyCriteriaStatusHistory``) may be routed to a
+separate physical database (:ref:`openedx-learning-adr-0005`), its append runs after that transaction commits, as a best-effort,
+non-blocking write.
+
+
+Rejected Alternatives
+---------------------
+
+1. Prevent concurrent writes with a deployment-wide lock.
+
+    Once every write is a monotone merge and each
+    parent recompute is serialized only against concurrent writers of that same node by a brief row lock, correctness holds without needing a larger lock.
+
+4. Recompute derived levels on read instead of materializing them.
+
+    Settled in :ref:`openedx-learning-adr-0002`.
+
+8. Send an event to openedx-core and update competency statuses in a separate celery task.
+
+    We want to avoid data drift. Wrapping the competency status update in the grade calculation transaction on edx-platform ensures atomicity.