Red Teaming

simple concept: Think-Write-Share.

Ref. D223-A

This technique involves asking the following questions of any argument that is used to justify a particular course of action, or that is offered as an explanation for a problem:

Ref. 3B88-B

There is nothing inherently wrong with assumptions. Every strategy, every plan, every decision an organization makes is based on them. Assumptions have to be made as part of any planning process. The challenge is making the right assumptions, because the better your assumptions are, the stronger your plan will be—and the more likely it will be to succeed. The danger lies in making the wrong assumptions, or worse still, failing to recognize them as assumptions at all. Most plans fail because they rely on unstated or unexamined assumptions. To make sure that does not happen to your plan, it is vital that you identify and examine all the assumptions it is based on before approving it.

Ref. 4D57-C

Key Assumptions Check

Ref. 8852-D

Once any unnecessary assumptions have been removed from the list, the next step is to challenge each of the assumptions that remain by posing the following questions:

Ref. BAF4-E

Any assumptions that prove weak or that threaten the outcome of the plan need to be flagged. If possible, the red team should come up with recommendations for strengthening those assumptions or even propose contingency plans in case they prove false. If neither of these is possible, then the entire plan may need to be reevaluated.

Ref. D9DE-F

The red team should focus its efforts on those assumptions that have a high potential for failure and would have major impact on the outcome plan if they prove false. This is particularly true when the red team has limited time to complete its work. These same assumptions should help inform the metrics the organization uses to monitor its progress against the plan as it is being executed. That way, you can be sure you are making real progress, not just getting lucky.

Ref. FF5D-G

String of Pearls Analysis

Ref. A1F3-I

Step 1: Analyze the planning document and identify all the major tasks, both stated and implied, that must be completed for the plan to reach fruition. Number these tasks sequentially to create your “string of pearls.” It may be helpful to color-code these to differentiate among different phases of the plan (e.g., construction, preproduction, production, etc.) or areas of responsibility (e.g., production, marketing, distribution, sales, etc.):

Ref. CBAF-J

It is easiest to do this on a whiteboard, but you can also use presentation software such as PowerPoint or Pages. Look at each task independently and identify all the stated and unstated assumptions behind it, just as you would in a Key Assumptions Check. For example, if you were red teaming a plan to expand automobile production in China, one of those tasks might be winning approval from the Chinese government. That single task is predicated on several assumptions. It assumes the Chinese government is willing to allow an expansion of local automobile production by a foreign manufacturer, that it is willing to work with your company on such a project, and that your company is willing to abide by whatever conditions Beijing might impose. All of these assumptions should be identified on the chart.

Ref. 8758-K

Once you have completed a spiderweb chart for each task, create a spreadsheet listing each task, along with its assumptions, dependencies, and effects:

Ref. 6142-L

Analysis of Competing Hypotheses (ACH)

Analysis of Competing Hypotheses (ACH) is a method used to make decisions about complicated problems. It was developed by the CIA in the 1970s. To use this method, you first need to brainstorm and think of all possible explanations for the problem. Then you make a list of evidence that supports or refutes each explanation. After that, you make a chart where you list your hypotheses across the top and each piece of evidence down the side. Then you look at the evidence and put a check mark under each explanation that the evidence is consistent with, and an X under each explanation it is not consistent with. You should also look for any evidence that neither supports nor refutes any of the explanations and eliminate that. Finally, you look at the chart and draw a conclusion about which explanation is most likely to be correct. You should be careful to avoid confirmation bias and be sure to consider all the evidence objectively.

Ref. EF85-M

Step 7: Report your conclusions. Make sure you include an analysis of the relative likelihood of all the other explanations that could not be categorically rejected. Explain why you feel your choice is the most likely to be true. This is important, because decision makers need to know that alternative explanations exist. It may be helpful to assign a percentage value to each hypothesis to reflect the red team’s assessment of how likely or unlikely each explanation is.

Ref. F25E-N

What are the issues and the conclusions?

Ref. 4CD0-O

list all the stated and unstated assumptions that must be true for the plan to work.

Ref. E954-P

all the stated and unstated assumptions that must be true for the plan to work.

Ref. 5415-Q

Ref. 68AC-R

Ref. 1B1C-S

As you create this spreadsheet, consider how critical each of these items is to the plan’s success. If the failure of an assumption would put the plan at risk, mark it with an R. If it would cause the plan to fail, mark it with an F. Challenge each assumption, and flag those that are weak or unlikely to prove true.

Red Teaming involves creating a spreadsheet to identify any assumptions, dependencies, and effects that could put the plan at risk or cause it to fail. After creating the spreadsheet, the next step is to assign a different color to each of the items and look for any risky dependencies or cascading effects. Stress-testing any assumptions that occur multiple times across the plan is important, as the success or failure of the plan could depend on them. Finally, a graphic representation should be created to display the results and identify any gaps or weaknesses in the plan.

Ref. F752-T

Analytical conclusions should always be regarded as tentative,” writes former CIA analyst Richards Heuer, the chief architect of ACH. “The situation may change, or it may remain unchanged while you receive new information that alters your appraisal. It is always helpful to specify in advance things one should look for or be alert to that, if observed, would suggest a significant change in the probabilities….Specifying in advance what would cause you to change your mind will also make it more difficult for you to rationalize such developments, if they occur.”

Ref. 9D4C-U

Assumption Sensitivity Analysis.

Ref. C287-V

Some of the most critical thinking comes from Amazon’s secret in-house benchmarking team, which is tasked with studying the company’s different business units and comparing their performance to internal and external competitors. These are sensitive reports that often include sharp criticisms and difficult recommendations, so the benchmarking team goes to great pains to ensure they have considered every possible objection before submitting them to senior management.

Ref. 3C15-W

Asking the Right Questions: A Guide to Critical Thinking

Ref. 7210-X

*2 This applies only to arguments, not to information. Knowing the source of information can be crucial to assessing its value and accuracy. See appeal to questionable authority.

Ref. A0FC-Y