Methodologies / Advanced Labs Methodology 20

Adversarial Experimentation

The Discriminating Test is the organizing concept. The methodology is used in Advisory Practices.

KATHLEEN REYNOLDSEXECUTIVE GOVERNANCE | INCIDENT COMMANDWritten By Kathleen Reynolds Advisory
Read This Methodology As Text

A strong strategic thesis should survive its most credible alternative explanation.

The Stages

  1. Define. Specify the decision, affected parties, time horizon, material downside and falsifiable claims. Here, a growth strategy promises additional demand while delivery capacity, working capital, regulatory conditions and staffing readiness remain constrained. The thesis is stated so it could be shown wrong.
  2. Authenticate. Separate observed evidence, unverified inputs, estimates and validated results; retain source and cutoff. The sources behind the thesis keep their date and owner before the challenge begins.
  3. Challenge. Construct the strongest rival hypothesis and identify discriminating evidence. A capable opposing reviewer builds the rival: a marketing forecast may be correct while cash conversion worsens. The decisive evidence is the result the two explanations predict differently.
  4. Compare. Evaluate reversible, conditional, deferred and full interventions against a documented baseline. Immediate launch is compared with a reversible pilot and a delayed launch.
  5. Authorize. Assign a human or governing body with actual decision rights; record conditions and exception triggers. The decision owner sets the stop trigger, and dissent is recorded.
  6. Observe. Measure outcomes, error and unintended consequences without changing the original forecast vintage. The pilot runs, and the observed result is read against both the thesis and the rival.
  7. Reconcile. Preserve amendments, contradictory events and subsequent decisions as linked but distinct records. Stopping and reopening conditions are kept with the record.

What It Does Not Claim

  • Red teaming becomes theater when the opponent is a straw man or cannot change the decision.
  • A useful challenge must be falsifiable, materially relevant, and capable of changing a decision.