Pilot error or panel error?

This is the first of two posts about ergonomics, one of those fundamentals we learn early in design school and then use without remembering where it came from. I decided to start with the part I think is most underrated: what we do when someone makes a mistake.

An American military plane has just landed, and someone in the cockpit reaches over to raise the flaps (the panels on the back edge of the wing that extend so the plane can land at a slower speed). Except they pull the landing gear control. The wheels retract with the plane still on the ground.

During World War II this happened often in the P-47, the B-17 and the B-25, at a time when "pilot error" was showing up more and more in accident reports. In 1943, the person who questioned that conclusion was Lieutenant Alphonse Chapanis, a psychologist and, until the end of the war, the only one in his field at Wright Field, in Ohio.

He noticed that the landing gear and flap controls sat side by side and, in most of those planes, were identical toggle switches or nearly identical levers. The story is told by Stanley Roscoe, in a 1997 monograph from the Human Factors and Ergonomics Society, written largely from memory and from years of correspondence with colleagues from that era.

The plane that didn't make the mistake

And there was one plane where this error didn't happen: the C-47, the military version of the DC-3 airliner, used to carry troops, cargo and paratroopers. In it, the two controls were apart and worked in different ways, and Roscoe says C-47 copilots never raised the wheels after landing.

Since the C-47 had a different mission, the comparison has limits. And the "never" comes from the memory of Roscoe, who flew transport planes during the war. Still, it's a clue that the problem lived in the panel, and that was the clue Chapanis bet on.

For Chapanis, these were cockpit design errors. The wartime fix was almost handmade: a small rubber-tired wheel on the end of the landing gear control and a wedge-shaped piece on the end of the flap control. It worked. According to Roscoe, pilots of the modified planes stopped raising their wheels after landing.

After the war, he adds, this shape coding became a worldwide standard. The American certification rule for transport airplanes, 14 CFR 25.781, sets the required knob shape for six cockpit controls. The landing gear knob is a wheel, like Chapanis's, more than eighty years later.

The pilot closest to the failure

In 1947, at the aeromedical laboratory of that same Wright Field, Paul Fitts (the same Fitts who in 1954 would publish Fitts's law, about the time it takes to hit a target depending on its distance and size) and Richard Jones analyzed 460 accounts of "pilot error" in operating aircraft controls.

What they showed, as researchers David Woods and Richard Cook summarize it, is that the swapped controls could be traced to features of the panel's design, especially in high-pressure moments. The errors had a pattern, and they were predictable.

Woods and Cook also explain why the blame landed on the pilot so easily. Investigators saw that the plane was flyable, that other pilots flew it without trouble and that the right information was on the panel. That left the pilot, the person closest to the failure, and from there to calling them the cause was a step that came far too easily.

That's ergonomics. Since 2000, the International Ergonomics Association has defined the discipline as the study of interactions between people and the other elements of a system, and the profession that uses that knowledge to design.

Notice that the person enters the definition as one of the elements. When the interaction fails, the question an ergonomist asks is which element needs to change, and the answer is rarely to swap out the people.

The Hawaii script

Saturday, January 13, 2018, 8:07 a.m. Phones and TVs across Hawaii receive the message "BALLISTIC MISSILE THREAT INBOUND TO HAWAII. SEEK IMMEDIATE SHELTER. THIS IS NOT A DRILL." There was no missile. The agency denied it on Twitter at 8:20, but the correction through the same alert system only went out at 8:45, 38 minutes later.

In the first days, the explanation going around was that an employee picked the wrong option from a menu. The investigation by the FCC, the US communications regulator, told a different story, in two reports. He thought the attack was real. The FCC never interviewed the employee, but it found the account relayed by Hawaii's emergency agency sufficient.

The drill was launched without notice, during a shift change, with a recording that said "exercise, exercise, exercise" at the beginning and end but carried the phrase "this is not a drill" in the middle, against procedure. He says he didn't hear the "exercise". According to the statements, the other officers in the room heard the same recording and understood it was a drill.

That's Hawaii's C-47, and it points at the person. According to the internal investigation cited by the FCC, staff had been concerned about this employee's performance for over ten years, and he had already confused a drill with a real event at least twice. I'm not going to pretend that doesn't matter.

There's a difference here. In the B-17, the intention was right and the hand went to the wrong control, which Don Norman calls a slip. In Hawaii, according to the FCC's conclusion, the action went exactly as he intended and the error was in the belief, which Norman calls a mistake.

But design shows up again, in a less obvious place. The agency's standard drill script already used the same language as the real alert, and the FCC wrote that this could confuse anyone receiving the call without prior notice.

That morning, the supervisor who ran the drill also slipped "not a drill" into the script, one more human error on top of a procedure that already made test and attack sound too much alike.

And once the mistake was made, very little stood between his hand and the button. A single person could send the alert to the whole state, with no second confirmation, and the last barrier was an "Are you sure that you want to send this Alert?" window that looked the same for a test and a real alert.

The FCC's conclusion is that neither the false alert nor the 38-minute delay would have happened if Hawaii had reasonable safeguards in place before that day. To me, the employee's history makes the argument stronger.

If staff already had doubts about his performance, the system should have been designed precisely for that person, with someone next to him at the button and a test script that didn't sound like an attack. The mistake was his. The open road to everyone's phone was the system's.

And in digital design?

From here on, this is my own reading of these two cases. That one-size-fits-all "Are you sure?" dialog is the digital version of the identical controls. When it shows up with the same text and the same button to delete a draft and to delete the whole account, my bet is that people learn to click without reading.

And Hawaii brings up a close relative: the test environment that looks exactly like production. In a design system this weighs more, because the generic component, reused across a hundred screens, spreads the problem. Chapanis's solution fits our vocabulary well: what can't be undone calls for a different shape, a different weight and, sometimes, a second person.

Here's a tip for the next time someone says "the user made a mistake", or the thought crosses your own mind. Before closing the diagnosis, ask whether the right and wrong options looked the same, whether the test looked like the real thing, whether the confirmation changed with the size of the damage and whether a single person could do something irreversible.

And if you can, look for your own C-47, and face what it shows you, even when it points at the person. Human error is where the investigation starts. In the next post I'll open up the whole map of ergonomics, from the origin of the word to the areas it covers. A kiss, and see you then!

---

For anyone who wants the source:

• Roscoe, S. N. (1997). The Adolescence of Engineering Psychology. Human Factors History Monograph Series, vol. 1. Human Factors and Ergonomics Society (link). The Chapanis story from 1943: the planes affected, the identical side-by-side controls, the C-47, the wheel and the wedge, and the postwar standardization. It's an account from memory, written more than fifty years later.
• Fitts, P. M., & Jones, R. E. (1947). Analysis of factors contributing to 460 "pilot-error" experiences in operating aircraft controls. Memorandum Report TSEAA-694-12, Aero Medical Laboratory, Wright Field. The study of the 460 accounts (I didn't read the original, which isn't openly available; the reference and summary come from Woods and Cook).
• Curatola, J. (2024). Douglas C-47 Skytrain. The National WWII Museum (link). The C-47 as the military version of the DC-3 and its missions.
• FAA. Pilot's Handbook of Aeronautical Knowledge, ch. 6, Flight Controls (link). What flaps are and what they do on landing.
• Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381–391 (link). The Fitts's law paper.
• Woods, D. D., & Cook, R. I. (1999). Perspectives on Human Error: Hindsight Biases and Local Rationality. In Durso, F. T. (Ed.), Handbook of Applied Cognition. Wiley (link, chapter version hosted by a third party). The summary of what Fitts and Jones found and the explanation of why blame fell on the person closest to the failure.
• United States. 14 CFR § 25.781, Cockpit control knob shape (link). The current rule with the required knob shapes for six controls, including the landing gear wheel.
• International Ergonomics Association. What Is Ergonomics? (link). The 2000 definition.
• Norman, D. (2013). The Design of Everyday Things: Revised and Expanded Edition. Basic Books. The distinction between slips and mistakes, in the chapter "Human Error? No, Bad Design".
• FCC, Public Safety and Homeland Security Bureau (2018). Report and Recommendations: Hawaii Emergency Management Agency January 13, 2018 False Alert (link). The alert text, the timeline, the drill recording, the other officers who understood it was a drill, the decision not to interview the employee, the 8:20 Twitter denial, the employee's history, the script using the same language as the real alert, the supervisor's deviation that morning, the confirmation window, the lack of a second confirmation and the conclusion on safeguards.
• Wamsley, L. (2018). False Hawaii Missile Alert Sent After Drill Recording Said "This Is Not A Drill". NPR, republished by WAMC (link). The January preliminary report refuting the wrong-click-in-the-menu version.