The Story Behind the Development of Airman Certification Standards
Article appeared in Flying Magazine December 2020 by John King
It was a wonderful era in our lives. We were following our dream of flying ourselves to meet with pilots to provide weekend ground schools. When Martha and I first were married we pledged to be equal partners in everything we would do. This committed us to be in some kind of small business. Our failed first attempt was a fleet service business that we didn’t enjoy. This time we had picked something we knew would be fun.
In this business we rented meeting rooms in hotels and taught complete ground schools in one weekend. Until then, most ground schools had taken two or three nights a week for six or seven weeks. In our weekend ground schools I would teach private, commercial and flight instructor students in one room and Martha would teach instrument and instrument flight instructor students in another room
What made it such a wonderful era was that we flew our own airplane to a circuit of cities around the Midwest and West to teach the courses. We returned to most cities every other month, where we would have the pleasure of meeting up again with dozens of pilots and spending an aviation weekend with them. Our arrangement was that once someone took one of our classes they could come back for free as many times as they wanted. It was good for business because they frequently brought friends with them. Our classes grew rapidly—and so did the parties in the evenings. We had wonderful times and developed great friendships.
You can imagine our deep sense of grief when occasionally we’d return to a city and someone would say, “Did you hear about Bill?” “No,” we’d say, “what happened to Bill?” They would then explain the aircraft crash that had killed him. It happened all too often. We began to realize the things that were killing these learning pilots were not the things the FAA was asking on the knowledge tests.
The problem was the FAA wanted a bell-shaped distribution of test scores on the test results. It was difficult for them to get a bell-shaped curve because our highly motivated students would come back to our classes (and the parties) and discuss with us what they had been asked on their knowledge tests. We would immediately change our courses to do a better job of covering anything they had missed. Consequently, our next students would not miss many questions—frustrating the test-givers. To get applicants to miss questions, the FAA had to make the questions trickier and more difficult—and less and less relevant to the risk management issues that pilots face in real life.
For instance, performance questions required double interpolations when better risk management would suggest just choosing more conservative numbers.
ADF questions would test for deeper understanding by asking reverse questions—even on the private test. Questions would provide magnetic bearing to the station and relative bearing and ask for the magnetic heading—an exercise that is never needed while flying. A pilot in flight finds the magnetic heading by looking at the heading indicator.
Martha and I began to realize that we were spending our weekends covering tricky questions and trivia that was irrelevant to actual flight, while the things actually causing fatalities were not being asked and therefore not being taught.
In their efforts to ensure that an adequate number of test questions would be missed, the FAA testing folks moved to make the test questions secret from the aviation training community—the people who were actually teaching pilots. They saw us as adversaries. I brought this adversarial relationship up with Nick Sabatini, the Associate Administrator for Aviation Safety, at a “Meet the FAA” session at AirVenture. Nick was concerned, and mandated that the airman testing folks meet with the aviation community annually. The result was an adversarial meeting once a year.
After several years of this, the Society of Aviation and Flight Educators (SAFE) held a national meeting in Atlanta. During the meeting, multiple speakers publicly lamented the poor quality of the FAA test questions and the adversarial relationship with airman testing. This greatly impressed the FAA managers who were there. Consequently, in 2011 an Aviation Rule-Making Committee (ARC) on the subject was formed. An ARC consists of both FAA and aviation community members and meets periodically to provide advice and information to the FAA.
There were about 25 or so members from the aviation community and 15 or so from the FAA. The atmosphere between the two groups was hostile. But the purpose of the meetings was for us to talk. And talk we did. The ARC lasted two years and made recommendations to the FAA about how to reform testing. Then it evolved into a working group which has lasted another seven years. The result has been nothing short of a miracle. Most significantly, the hostility slowly went away. As we communicated, we began to develop mutual respect and understanding. The FAA began to refer to members of the “aviation community” rather than to “industry.” Members of the community began to appreciate the competence and goodwill of the FAA.
We gradually made significant progress on fixing important problems. We became aware that while there were existing standards for the skills that could be required of an applicant, at the time there were no standards for the knowledge that could be asked of an applicant—on either the knowledge test or the practical test. The knowledge tests could and did ask an applicant anything no matter how trivial. As the discussions continued, we agreed it was a good idea to have standards for the knowledge so applicants would know to be studying meaningful and relevant concepts.
But then we started discussing the idea that although applicants would have to demonstrate skills and knowledge, applicants were not required to demonstrate the ability to identify and mitigate risks. Yet it was not lack of skills that was the biggest cause of fatalities. It was poor risk management. Remembering my friends who had come to grief, I became a champion for the idea that pilots should be required to demonstrate the ability to identify and mitigate risks. And standards for risk management became part of pilot check rides.
The biggest regret I have is that the FAA never did accept the recommendation of the ARC that the test questions be returned to the public domain—depriving the test writers of valuable input from the aviation community, and robbing applicants, their instructors and examiners of the specific details on any knowledge test questions they miss, plus creating a business for companies who are covertly purchasing the questions and selling them to willing buyers.
But all in all, I have to say that these nine years have been the most inspiring example of developing respect and collaboration I have ever seen. And the result is, rather than merely having practical test standards we now have complete standards for the certification of pilots. And pilots are learning the practice of identifying the risks in a flight and developing the habit of mitigating them. I am hopeful that this will save thousands of lives.