Improving Knowledge Tests to Save Lives
Article appeared in Flying Magazine May, 2016 by John King –
“John, Dr. Williams is dead. I thought you’d want to know.” The news hit me like a thunderbolt. The caller was an FAA inspector. Just two weeks earlier, I had asked him to talk with Dr. Williams. Dr. Williams was a towering figure. He was a physician, a radiologist and an Episcopalian priest. He was a pillar in his community. But as a pilot, he had worried me.
Martha and I were teaching two-day ground schools, and Dr. Williams had been in my class. He just didn’t follow the normal conventions of classroom behavior. He was impatient and in a hurry. He returned late from breaks and blurted out comments in class. I was worried he might behave impatiently in his flying. In fact, I was so concerned that when the FAA inspector came to administer the knowledge test, I asked him to speak to Dr. Williams.
“John,” my FAA friend said, “I can’t just pick someone out of your classroom and lecture him because you told me I should. He’ll call his congressman. You talk to him.”
“He won’t listen to me,” I said. “I’m just a traveling ground instructor.” So neither one of us talked with him.
He died on a solo cross-country. On the first leg, he got lost and wound up in the mountains. Asking for help on the radio, he said, “There are clouds around me with trees in them.” He landed safely at his destination.
Greatly relieved to see him, the folks at the Flight Service Station literally begged him to come in to talk. He didn’t have time, he said. He was scheduled to make a speech after he returned to his home airport. Without shutting off his engine, he took off on the return leg. He died in the same mountains on his way back.
I was devastated. I felt terribly guilty. I had foreseen that this might happen, yet I hadn’t spoken to him. I considered quitting teaching flying. I felt that I didn’t want to continue teaching people to do something that could kill them. Martha and I were traveling on a circuit of cities, teaching more than 2,000 pilots a year. All too often we returned to a city to learn that a pilot we knew had died. It was getting to me, and Dr. Williams was the tipping point.
Moreover, I was deeply discouraged that many of the questions we needed to prepare pilots for on the knowledge test were obscure, trivial and even tricky. As a result, we were being forced to teach obscurity while pilots were coming to grief because they did not know how to identify and mitigate the risks of flying. Being part of that failed and dysfunctional system was depressing.
I loved flying and loved teaching it. It would be, I decided, my job to teach pilots how to identify and mitigate the risks of flight. I made one more resolution: I would never again fail to speak out when doing so offered any chance of saving someone’s life.
The unanswered question is: What would I have said to Dr. Williams that would have gotten a positive result? I still don’t know. That’s probably the real reason neither my FAA friend nor I spoke with him. An even more pertinent question is: What, if anything, could have been done to head off that catastrophe? I believe the answer would have been to teach him aviation risk management, but it would have needed to start with his first flight lesson.
Physicians are not unaccustomed to the idea of managing risk. Dr. Williams had been a radiologist. There are all sorts of risks associated with radiology. Dr. Williams and other goal-oriented people can develop the habit of identifying and mitigating risks in aviation as they do in the rest of their lives if that habit is cultivated from the very first flight lesson.
As time progressed, it became apparent that Martha and I were far from the only people who had these concerns. In the March 2001 issue, Flying published an interview of me by Lane Wallace titled “Battling the ‘Big Lie.’” This gave me the venue to speak out to the aviation community. I wanted pilots to know that we should recognize that the activity of flying is risky, and we should manage the risks. It was a provocative interview, and it took courage for Flying editor Mac McClellan to run it.
In response to the story, Jim Lauerman, head of Avemco Insurance, wrote a letter proposing that we work together to help pilots manage risks. The folks at Avemco had been mourning the all-too-often loss of pilots and customers as Martha and I had. In response, we developed a series of practical risk-management courses, and Avemco grants a premium credit to pilots who take them.
At about the same time, Bob Wright, who was head of the FAA’s General Aviation and Commercial Division, spearheaded the implementation of the FAA-Industry Training Standards (FITS), with a goal of incorporating scenario-based training and risk management into flight training programs.
Still, the accident rate remained high, and the FAA knowledge tests continued to be profoundly irrelevant. In May 2011, the Society of Aviation and Flight Educators (SAFE) organized a landmark conference in Atlanta, which brought together hundreds of concerned pilots and instructors. Among them was Van Kerns, head of the FAA’s Regulatory Support Division, which is responsible for airman testing standards. Van listened while person after person railed about the poor quality and irrelevance of the knowledge tests.
That evening Van bumped into Susan Parson, who is special assistant to the Director of the FAA’s Flight Standards Service. Susan, always deeply disturbed by the irrelevance of the exams, was by then steaming as result of the ATP written exam she had recently taken. Van and Susan agreed that this needed to change.
Not one to ignore opportunity, Susan seized on the momentum provided by this perfect storm of events. She organized the Airman Testing Standards and Training Aviation Rulemaking Committee (ARC) to reform the way airmen are tested and evaluated.
Susan recruited folks from nearly every segment of the aviation training community, including even me, and the right people from the FAA, and set us all to work. The result of the five-year effort from the ARC and the two ACS Working Groups that followed was the development and implementation of the Airman Certification Standards (ACS). The ACS incorporates everything a pilot is required to know or be able to do for a specific certificate or rating into a single document.
I am delighted to report that beginning in June, the ACS will replace the Practical Test Standards for the Private Pilot-Airplane and Instrument Rating-Airplane practical tests.
The first thing folks will notice when they prepare for a check ride is that now, for the first time, there are standards for the knowledge a pilot is expected to demonstrate on both the knowledge test and practical test. The knowledge required of pilots will be relevant to a pilot’s ability to get utility from the aircraft, and to identify and mitigate the risks of flight. No longer will the pilot be tested on the trivial and obscure.
Plus, a pilot will be evaluated on the ability to actually apply the knowledge they have learned to identify and mitigate the risks of flight. Pilots with this habit will be situationally aware and far less likely to be caught by surprise by events that, with risk-management skills, they would have seen coming.
I now have hope that extremely goal-oriented pilots like Dr. Williams will get the help they need to ward off tragedy. If the ACS had been in place when Dr. Williams was learning to fly, perhaps he would have learned to identify and mitigate the risks associated with scheduling a speech right after a solo cross-country.