Methodologies / Advanced Labs Methodology 24
Algorithmic Evaluation Governance
The Versioned Method is the organizing concept. The methodology is used in PMO & Compliance Governance.
KATHLEEN REYNOLDSEXECUTIVE GOVERNANCE | INCIDENT COMMANDWritten By Kathleen Reynolds Advisory
Read This Methodology As Text
The method that grades people or systems is itself a governed versioned object.
The Stages
- Define. Specify the decision, affected parties, time horizon, material downside and falsifiable claims. Here, an assessment program grades learners and issues credentials. The method owner, content owner, approving authority, assessment execution and credential issuance are identified separately.
- Authenticate. Separate observed evidence, unverified inputs, estimates and validated results; retain source and cutoff. The method version is approved, its assessment rules are frozen, and credentials are issued under it.
- Challenge. Construct the strongest rival hypothesis and identify discriminating evidence. The scoring methodology changes. The rival: retroactive recalculation may unfairly change a credential.
- Compare. Evaluate reversible, conditional, deferred and full interventions against a documented baseline. Rescoring past results, applying the new method forward only and keeping the old method are compared.
- Authorize. Assign a human or governing body with actual decision rights; record conditions and exception triggers. The approving authority approves the new method for future use, not for past records.
- Observe. Measure outcomes, error and unintended consequences without changing the original forecast vintage. Outcomes under the new version are recorded with the version that produced them.
- Reconcile. Preserve amendments, contradictory events and subsequent decisions as linked but distinct records. An appeal and correction path is recorded as a new event, and historical versions are preserved.
What It Does Not Claim
- Retroactive scoring changes can violate reliance expectations or erase auditability.
- Buyer-specific governance may differ from internal enterprise authority.