Mathjax javascript librry setup

He's Making a List and Checking It Twice

The Role of Checklists in Software Development

When you think about it, we use checklists everywhere in software development. You may have to squint to see them, but they are there.

Some examples include:

  • Most large companies use some kind of phase-gate process where each gate is evaluated with a set of criteria in the form of a checklist.
  • Agile’s Definition of Done consists of checklists used to determine whether a User Story or a Sprint is “done”
  • Many teams have a checklist used by developers when they check in code
  • And some teams use a checklist to guide code reviews

While we use checklists (a good thing), we tend not to think a lot about what makes a good checklist. But we do question their value, and we certainly have bad experiences with checklists:

  • S/W development is a complex activity. It is a design activity with an infinite variety of detailed activities, conducted in a team environment. Can a checklist that’s simple enough to be usable really be valuable?
  • When we do create checklists, we often make them long-winded… lots of points and lots of detail to each point. They become unwieldy and hard to apply. We miss the forest for the trees.
  • And how do we know our checklists work? We typically don’t measure their effectiveness… we don’t look at metrics pre and post the introduction of a checklist (or any practice for that matter). When we do try to evaluate the impact of a practice change, its often done using ad-hoc gut-feel evaluations, not based on hard data.

It turns out that there are industries that have extensively studied and implemented checklists, and we can learn a lot from them.

For learning about checklists - how to develop, deploy, and measure them, lets turn to Atul Gawande. As noted on Wikipedia, Gawande is a surgeon, journalist, and political activist who is known in the public arena as an expert on reducing error, improving safety, and increasing efficiency in surgery. He started writing for Slate and the New Yorker when he was a medical resident, and is the author of 3 books. He has a simple, humble, story-telling writing style that is quite engaging.

And if you want to develop any checklist Gawande’s 3rd book The Checklist Manifesto is required reading.

Introduction - Why do we Make Mistakes?

Gwande references a paper by the philosophers Samual Gorovitz and Alasdair MacIntyre titled Toward a theory of medical fallibility, published in 1975. Their paper is an inquiry into the nature of human fallibility - that is why do we fail at what we set out to do in the world. One reason is “necessary fallibility” - some things are simply beyond our capacity. We not all-powerful, and much of the world and universe is outside our understanding and control.

There are, of course, lots of realms which are in our control. We do build lots of large software systems successfully, we build skyscrapers that don’t fall down, commercial aviation lets us fly between continents at low cost with remarkable safety, and we can save people from cancer, heat attacks, and organ failures. In such realms Gorovitz and MacIntyre point to two reasons why we may still fail:

Ignorance

We may err because science and technology have given us only a partial understanding of the world and how it works. There are buildings we don’t know how to build, cancers we can’t cure and organs we can’t replace. For an excellent engineering book on the subject I’d recommend “To Engineer is Human: The Role of Failure in Successful Design” by Henry Petroski.

Ineptitude

In these instances the knowledge exists, in most cases the activity may have been successfully completed many times in the past. Yet while we have the knowledge - “we know what to do” - we fail to apply that knowledge correctly.

Gawande makes the observation that in 21st century medicine, that balance has greatly shifted from ignorance to ineptitude. In the last few decades we have developed infinitely better knowledge and techniques in medicine. Gawande points out that in the 1950s we had very little knowledge of heart attacks, how to prevent them and how to treat them. Now we have at least a dozen effective ways to reduce your likelihood of having a heart attack, and a whole panel of effective therapies for treating them. But Gawande feels that our medical practices have not kept pace and in medicine the balance has shifted from failure due to ignorance to failure due to ineptitude.

I would argue that the same holds true in software engineering. In 1974, when I started my undergraduate degree, there was only a rudimentary knowledge of software engineering and how to manage software projects. Heck, using a source code control system was a novel idea! Object-Oriented languages were a research topic. Tools were primitive. Communications tools were almost non-existent - email barely existed. And there was little available in the way of frameworks, applications servers, databases, and libraries. What did exist generally came as expensive commercial products. There were lots of reasons of “ignorance” why software development was hard, unpredictable, and prone to failure.

But today, we have no such excuses. After 50 years of software product development we have a rich set of processes, practices, and tools that are widely available. Information and training is readily available, often for free. We have many standard design patterns to apply to problems. Yet we still fail far more often than we should.

Now the problem we face in medicine and software development is ineptitude - making sure we apply the knowledge we have consistently and correctly. The statistics Gwande quotes for medicine are staggering, and incredibly familiar-sounding to software development managers. They are failures of execution and discipline. Gwande says:

I have been trying for some time to understand the source of our greatest difficulties and stresses in medicine. It is not money or government or the threat of malpractice lawsuits or insurance company hassles … It is the complexity science has dropped on us and the enormous strains we are encountering in making good on its promise. I have seen it everywhere - in Europe, in Asia, in countries rich and poor. Moreover I have found to my surprise that the challenge is not limited to medicine.

Know-how and sophistication have increased remarkably across almost all of our realms of endeavor, and as a result so has our struggle to deliver on them. […] You see it in the 36% increase between 2004 and 2007 in lawsuits against attorneys for legal mistakes - the most common being simple administrative errors. […] You see it in flawed software design, in foreign intelligence failures, in our tottering banks - in fact, in almost any endeavor requiring mastery of complexity and large amounts of knowledge.

Such failures carry an emotional valence that seems to cloud how we think about them. Failures of ignorance we can forgive. […] But if the knowledge exists and is not applied correctly, it is difficult not to be infuriated. What do you mean half of heart attack patients don’t get treatment on time? What do you mean that two-thirds of the death penalty cases are overturned because of errors? It is not for nothing that the philosophers gave these failures so unmerciful a name ineptitude.

Here, then, is our situation. We have accumulated stupendous know-how. We have put it in the hands of some of the most highly trained, highly skilled, and hardworking people in our society. And with it, they have indeed accomplished extraordinary things. Nonetheless, that know-how is often unmanageable. Avoidable failures are common and persistent, not to mention demoralizing and frustrating, across many fields. And the reason is increasingly evident: the volume and complexity of what we know has exceeded our individual ability to deliver its benefits correctly, safely, or reliably.

That means we need a different strategy for overcoming failure, one that builds on experience and takes advantage of the knowledge people have but somehow also makes up for our inevitable human inadequacies. And there is such a strategy, although it will seem almost ridiculous in its simplicity, maybe even crazy to those of us who have spent years carefully developing ever more advanced skills and technologies.

It is a checklist.

On the Road to Checklists

Gwande’s journey to checklists began in 2006, when he was asked to lead a project by the World Health Organization (WHO) to develop a global program to reduce avoidable deaths and harm from surgery. The number of major surgical procedures was exploding worldwide. By 2004 surgeons were performing 230 million major operations annually - one for every 25 people on the planet. With serious complications running at between 3%-17% of surgeries performed, millions were dying or suffering every year from surgical complications.

His team brainstormed all kinds of ideas, and researched what kinds of interventions were effective and which were not. They identified the following common characteristics in successful interventions:

  • Simple - a vaccine, removing pump handles from contaminated wells, providing soap and instructions for hand-washing.
  • The effects were carefully measured.
  • The interventions had widely transmissible benefits, what business would term a large ROI.
  • The process for introducing the intervention, its training, monitoring its adoption were carefully planned and thought out.
  • Cultural issues that could either interfere with or promote adoption were carefully accounted for.
  • Many interventions are dependent of behavior changes … the “behavior change delivery vehicle” needs to be considered.

A number of team members reported on their experiences with checklists. For example, the Columbus Children’s hospital had developed a checklist to reduce surgical infections. Infection is one of the most common complications in surgery, and the most effective way to prevent it (apart from using proper sterile surgical techniques) was to make sure you give an appropriate antibiotic in a one hour window before the surgery starts. Administering the antibiotic in that one hour window is key. Too soon and it wears off before the surgery starts. Too late, and the incision has already been made. Getting the antibiotic in at the right time reduces the surgical infection risk by half.

This sounds like a pretty simple thing to do, but in 2005 more than one-third of the appendectomy patients did not get the right antibiotic at the right time!

The hospital’s director of surgical administration, who happened to be a pilot, decided to take the aviation approach. He designed a pre-incision “Cleared for Takeoff” checklist. A nurse on the surgical team had to verbally confirm with the team that they:

  • Had the correct patient
  • Were operating on the correct side of the body
  • That antibiotics were given, or had been deemed to be unnecessary.

Now here’s the cool part. Getting the getting the teams to stop and use the checklist - to make it a habit - was tricky. The surgical director did the normal sort of training and lectures. But he also had little metal tents made that were designed to cover the scalpels, and were engraved with the words “Cleared for Takeoff”. This did two things:

  • It was a reminder to go through the checklist.
  • It gave power to the nurses … the surgeon could not start the operation until the nurse gave the OK and removed the tent.

After 3 months, 89% of the patients got the right antibiotic at the right time. After 10 months 100% did. The checklist had become a habit. We’ll come back to the “aviation checklist model” and also what I call “the habit theory of practices” a bit later in this article.

The first attempt at the “safe surgery checklist” was assembled by Gawande’s group from all the different checklists that had been tried in the past. It gave detailed steps for everything from equipment inspection to antibiotic administration to the discussions the surgical team should have. And it was a total failure. The checklist was:

  • Too long
  • Unclear
  • Ambiguous
  • Became a distraction the patient on the table and the task at hand

Anyone who’s tried to develop an effective software development checklist or practice is familiar with these symptoms. Its a trap one just naturally falls into … trying to specify too much on one hand, while also not thinking though what’s needed to be crystal clear on a point to someone who’s trying to follow the checklist or instruction “in the field”.

Learning From the Experts

Sometimes when you need to jump a high fence, you have to back up to take a run at it!

The Aviation Checklist

To learn how to write an effective checklist, Gawande turned to the obvious industry… aviation. You might think that checklists have always been part of aviation, but Gawande traced the invention of the aviation checklist back to 1935. The U.S. Army Air Corps held a flight competition to select a manufacturer to build the military’s next-generation long-range bomber. Boeing’s aluminium-alloy Model 299 was expected to be a shoe-in. It could carry five times as many bombs as the army had requested, it could fly faster and twice as far as previous bombers. The flight “competition” was a mere formality. The “flying fortress”, as it had already been named, was the sure winner.

But it was not to be… The plane took off smoothly for its flight competition, climbed to three hundred feet, stalled, and spiraled in to a fiery crash. Five of the flight crew, including the pilot, died.

An investigation revealed nothing had gone wrong. It was a case of “pilot error”. The pilot had forgot to release the locking mechanism on the elevator and rudder controls. Now, I’ve flown gliders, and I can tell you this is a real “rookie” mistake. Checking for control mobility is the most basic of checks. But the new plane was substantially more complex than previous aircraft - It required the pilot to attend to four engines rather than two, each with its own oil-fuel mix, the retractable landing gear, the wing flaps, the electric trim tabs, the variable-pitch propellers. The “flying fortress” was “too much airplane for one man to fly”. Douglas’s smaller design was declared the winner and Boeing nearly went bankrupt.

A few people remained convinced that the flying fortress was a great plane, and the army bought a few as test planes. To address the issue that the aircraft was too complicated for a pilot to memorize everything, group of test pilots came up with the idea of a pilot’s checklist. They devised a set of simple, brief checklists, each of which could fit on an index card. There were checklists for taxiing, takeoff, flight, and landing.

And with the checklists the Model 299 became flyable. It became the B17 Flying Fortress of World War II fame. The army eventually ordered over 13,000 of the bombers, and it helped the Allies wage an effective bombing campaign against Nazi Germany.

Of course aircraft have become far more complex since WW II, and aviation checklists have become far more sophisticated. To study how checklists are used in the modern aviation industry, Gawande turned to Dan Boorman (see also this pdf), a Flight Training Technical Fellow at Boeing. Boorman is a veteran Pilot who has spent over two decades developing checklists and flight deck controls for Boeing. Part of Boorman’s job is studying thousands of aviation “incidents” (crashes and near crashes) and devising was to address and avert them. He has made a science of averting human error.

Many people think pilot’s checklists are pre-checks done for a few normal activities like takeoff and landing. But that’s not the case at all. Commercial aviation checklists cover every conceivable normal and non-normal situation that may occur in an aircraft. They have traditionally been bound into large binders, with tabs to identify each checklist. Increasingly, they are made available electronically, both on the screens as part of the flight deck, and on tablets carried by the pilots. There can be hundreds of them. But each one is brief, simple, and clear. When an emergency occurs one pilot flys the aircraft, the other looks up the check list that covers the situation.

So what makes a good and a bad checklist? Gawande writes:

… Boorman explained. Bad checklists are vague and imprecise. They are too long; they are hard to use; they are impractical. They are made by desk jockeys with no awareness of the situations in which they are to be deployed. They treat the people using the tools as dumb and try to spell out every single step. They turn people’s brains off rather than turn them on.

Good checklists, on the other hand, are precise. They are efficient, to the point, and easy to use even in the most difficult situations. They do not try to spell out everything - a checklist cannot fly a plane. Instead they provide reminders of only the most critical and important steps - the ones that even the highly skilled professionals using them could miss. Good checklists are, above all, practical.

The power of checklists is limited, Boorman emphasized. They can help experts remember how to manage a complex process or configure a complex machine. They can make priorities clearer and prompt people to function better as a team. By themselves, however, checklists cannot make anyone follow them.

And finally…

Boorman was adamant about one further point: no matter how much thought we might put in, a checklist has to be tested in the real world, which is inevitably more complicated than expected. First drafts always fall apart, he said, and one needs to to study how, make changes, and keep testing until the checklist works consistently.

Checklists in Construction

“The End of the Master Builder - Atul Gawande”

… A lesson is emerging: checklists seem to be able to defend anyone, even the experienced, against failure in many more tasks than we realized. They provide a kind of cognitive net. They catch mental flaws inherent in all of us - flaws of memory and attention and thoroughness. And because they do, they raise wide, unexpected possibilities.

But they presumably have limits as well. So a key step is to identify which kinds of situations checklists can help with, an which ones they can’t.

The other industry Gawande turned to for advice was construction. Gawande wanted to know whether checklists broke down when used for very large or complex systems. Brenda Zimmerman of York University and Sholom Glouberman of the University of Toronto, who study the science of complexity, proposed three different kinds of problems in the world:

The Simple

A simple problem is like baking a cake form a mix. There is a recipe. They may be a few basic techniques to learn, but once mastered following the recipe is repeatable - there is a high likelihood of success.

The Complicated

Complicated problems can sometimes be broken down into a series of simple problems. But there is no recipe. Success requires creating the recipes and learning how to solve the problem for the first time. Success frequently requires multiple people, or multiple teams, and specialized expertise. Unanticipated problems appear often. Timing and coordination are serious concerns. But once you learn how solve a complicated problem you can generally repeat the process again and perfect it. Building a spacecraft to send into orbit is a complicated problem.

The Complex

Complex problems are unique. Solving the problem in one situation is no guarantee of being able to solve it again in another. Raising a child is a complex problem - every child is unique. Raising one child provides experience but no assurance of success with the next. Although expertise can contribute to the process in valuable ways, it provides neither necessary nor sufficient conditions to assure success. Another feature of complex problems is that the outcome is highly uncertain. Yet, the complexity of the process and the lack of certainty do not lead us to the conclusion that it is impossible to raise a child.

For more information see:

Gawande notes that checklists address essentially simple problems - to focus attention on checking rudder and elevator controls before takeoff, or maintaining sterility in a medical procedure, or remembering all the critical avenues of defense in a tax fraud case. We are besieged by simple problems, and checklists can provide protection against such elementary errors.

However much of the critical work we do is not so simple. Even a small software development project may have a half-dozen people executing hundreds of tasks over a period of months to build an application against uncertain and changing requirements. In medicine, no two patients are alike and each patient may be cared for by a dozen specialists. Both software development projects and medical cases will take unpredictable turns, and will require expertise and judgment to create a successful outcome. Software development, like medicine and many other human endevours, involves simple, complex, and complicated problems. Can checklists help avert failure when the problems combine everything from the simple to the complex?

Gawande found his answer in the construction industry. How do we manage to successfully construct large buildings, involving complex plans, dozens of trades, hundreds of workers, and hundreds of thousands of tasks? Gawande writes:

So as I looked up at this whole building that had to stand up straight even in an earthquake, puzzling over how the workers could be sure they were constructing it properly, I realized the question had two components:

  • First, how could they be sure they had the right knowledge in hand?
  • Second, how could they be sure they were applying the knowledge correctly?

Both aspects are tricky. In designing a building, experts must take into account a disconcertingly vast range of factors … Then to turn the paper plans into reality, they face equally byzantine difficulties making sure that all the different tradesmen and machinery do their job the right way, in the right sequence, while also maintaining the flexibility to adjust for unexpected difficulties and changes.

Yet builders clearly succeed. They safely put up millions of buildings all over the globe. And they do so despite the fact that construction work has grown infinitely more complex over the decades. Moreover, they do it with a frontline workforce that regards each job particular job much the way doctors, teachers, and other professionals regard their jobs: as specialized domains in which others should not interfere.

The Checklist for Checklists (to incorporate)

He has several examples of checklists on his site. The Checklist for Checklists is a “must study” for those of interested in Software Development Practices in general, and Agile/Lean practices in particular. To quote the site:

This checklist is designed to aid the checklist creation process and ensure that your checklist helps instead of hurts. It was created by Dr. Atul Gawande, the Brigham and Women’s Hospital Center for Surgery and Public Health Dissemination Team, and Dan Boorman of Boeing.

Lean Law - Inspiring continuous improvement for legal professionals, knowledge workers, and others

The Bottom Line - Results

Why are Checklists so Hard?

Bill’s “habit theory of practices”

Comments