How Complex Systems Fail

by Richard I. Cook · 5 ideas

  1. Degraded Mode

    Complex systems always run with some flaws and broken parts. They keep working only because many layers of defense catch problems before they spread. So instead of planning for a flawless system, invest in the layers that catch failures.

  2. No Root Cause

    Big failures need several small failures to line up at once. Each alone was harmless. Looking for a single root cause misses how the system really broke and leads to the wrong fixes.

  3. Hindsight Bias

    After an accident, the path to failure looks obvious. It wasn't obvious to the people inside it, who had partial information and many demands. Judge decisions by what people knew at the time, not by the outcome.

  4. People Create Safety

    Operators constantly adjust and improvise to keep a system safe. Safety isn't built into the parts, it's done every day by people. Support and learn from their skill instead of blaming them when something breaks.

  5. Change Brings New Failures

    New technology often removes common failures but creates new ones that are rarer and worse. These new failures are hard to foresee. Expect new risks with every change and watch closely after it.

More on systems