Methodologies / Advanced Labs Methodology 20
Adversarial Experimentation
The Discriminating Test is the organizing concept. The methodology is used in Advisory Practices.
KATHLEEN REYNOLDSEXECUTIVE GOVERNANCE | INCIDENT COMMANDWritten By Kathleen Reynolds Advisory
Read This Methodology As Text
A strong strategic thesis should survive its most credible alternative explanation.
The Stages
- Define. Specify the decision, affected parties, time horizon, material downside and falsifiable claims. Here, a growth strategy promises additional demand while delivery capacity, working capital, regulatory conditions and staffing readiness remain constrained. The thesis is stated so it could be shown wrong.
- Authenticate. Separate observed evidence, unverified inputs, estimates and validated results; retain source and cutoff. The sources behind the thesis keep their date and owner before the challenge begins.
- Challenge. Construct the strongest rival hypothesis and identify discriminating evidence. A capable opposing reviewer builds the rival: a marketing forecast may be correct while cash conversion worsens. The decisive evidence is the result the two explanations predict differently.
- Compare. Evaluate reversible, conditional, deferred and full interventions against a documented baseline. Immediate launch is compared with a reversible pilot and a delayed launch.
- Authorize. Assign a human or governing body with actual decision rights; record conditions and exception triggers. The decision owner sets the stop trigger, and dissent is recorded.
- Observe. Measure outcomes, error and unintended consequences without changing the original forecast vintage. The pilot runs, and the observed result is read against both the thesis and the rival.
- Reconcile. Preserve amendments, contradictory events and subsequent decisions as linked but distinct records. Stopping and reopening conditions are kept with the record.
What It Does Not Claim
- Red teaming becomes theater when the opponent is a straw man or cannot change the decision.
- A useful challenge must be falsifiable, materially relevant, and capable of changing a decision.