This article is published in Aviation Week & Space Technology and is free to read until Jun 07, 2024. If you want to read more articles from this publication, please click the link to subscribe.
Military aircraft competitions are decided by more than just design, of course. As the U.S. Air Force Next-Generation Air Dominance program contract award nears, we look at the lessons industry learned from the service’s McDonnell Douglas F-15 replacement decision.
- Showmanship and understanding the customer can be pivotal in competitions
- A contractor’s prior performance could be key
In 1981, when the Air Force identified the need to replace its F-15s, it launched the Advanced Tactical Fighter (ATF) program. The ensuing competition is often remembered as among the hardest fought and most technically demanding, culminating in arguably the single greatest capability leap between fighter generations to date. However, the competing Lockheed-Boeing-General Dynamics F-22 and Northrop-McDonnell Douglas F-23 represented more than dueling design philosophies. They reflected each company’s key individuals—and to an extent, corporate cultures—in gauging what the customer really wanted.
Making the Cut
The Air Force spent the first four months of 1991 closely examining all four airframe-engine combinations for the engineering and manufacturing development (EMD) phase. The ATF source selection was a three-stage process involving the Source Selection Evaluation Board (SSEB), Source Selection Advisory Council (SSAC) and Source Selection Authority (SSA). The SSEB evaluated contractor proposals relative to the request for proposals (RFP).
The RFP placed the greatest importance on equally rated contractor technical performance and program management, followed by cost. Bill Sweetman, who later became an AW&ST senior defense editor, reported in aerospace magazine Interavia in May 1991 that evaluators used a series of “stoplight charts” to grade contractor proposals, with blue indicating superior performance relative to the requirement, green as meeting the requirement, yellow as being below required performance but fixable and red as seriously deficient in performance. Under these conditions, the Air Force appeared to have weighted deficient performance more heavily than excess performance.
Once the SSEB concluded its evaluation, the findings went to the SSAC, which typically compares the merits of each proposal and briefs the SSA. The ATF SSAC did not make specific recommendations to the SSA; then-Air Force Secretary Donald B. Rice made the final decision. On April 23, 1991, Rice announced the selection of the F-22/Pratt & Whitney F119 combination as the winner of the ATF competition.
Two distinct but interrelated narratives emerged from government and industry following the source selection announcement concerning why the choice was made. One focused on prime contractorship and the other on showmanship and the teams’ understanding of the customer.
“Prime Contractorship”
At the April 23 press conference, Rice explained that all of the proposals met Air Force requirements, and each of the four combinations was awardable. However, the Lockheed-Pratt & Whitney combination “clearly offered better capability at lower cost, thereby providing the Air Force with true best value.” Government sources maintain that the Lockheed team won on the merits of its program and project management plans rather than on technical performance.
Just after the contract award, Aviation Week published a special report on the ATF selection in which an Air Force official familiar with the source selection process was paraphrased as saying: “Lockheed and Pratt were considered more likely to accomplish what they proposed and to manage the development program successfully. Northrop and GE were considered more likely to have problems” (AW&ST April 29, 1991). The Lockheed-Pratt & Whitney aircraft’s slightly lower cost also was significant, since “a few percentage points in as high-cost a program as the advanced tactical fighter involve large amounts of money,” the official was paraphrased as saying. “It was very clean,” the official said of the decision. “There were not a lot of gut-wrenching judgment calls to be made.”
Rice explained that the Air Force cross-referenced contractor ATF proposals with records of their past programs to judge the veracity of their proposals and their ability to manage risk and control costs. He declined to specify how much prior performance on external programs was weighed on ATF, but circumstantial evidence suggests it was a major contributing factor.
The Soviet Union was collapsing, and defense budgets were correspondingly slated to decline. For the ATF to survive, the Air Force was highly incentivized to choose the least risky and most affordable airframe-engine combination led by the prime with the strongest program management history. At the time, both Northrop and McDonnell Douglas were facing mounting pressure to deliver on their flagship programs.
Northrop’s top pair of low-observable (LO), meaning radar stealth, programs—the AGM-137 Tri-Service Standoff Attack Missile (TSSAM) and B-2A—were experiencing substantial cost and schedule delays. TSSAM was canceled in 1994 with a projected unit cost of $2 million ($3.8 million in fiscal 2025 dollars). More alarmingly for Northrop, the B-2 was facing mounting congressional scrutiny. By 1988, the program was running 18-24 months behind schedule as a result of an Air Force-mandated redesign to improve low-altitude penetration characteristics and emerging signature performance issues. Rice frequently had to defend the program from both the media and Congress during this period.
Meanwhile, Northrop’s principal F-23 partner, McDonnell Douglas, was facing existential financial and legal liabilities along with General Dynamics following the termination for default of the Navy’s A-12 program in January 1991. The A-12, a stealthy carrier-based bomber, was developed under a $4.8 billion ($10.8 billion in fiscal 2025 dollars), fixed-price EMD contract. By March 1990, the A-12 was likely over budget by $1 billion ($2.3 billion in fiscal 2025 dollars) and at least a year behind schedule.
Then-Defense Secretary Dick Cheney told Congress on April 26, 1990, that the A-12 program “appears to be reasonably well-handled at this point.” As additional cost and schedule overruns became public, he canceled the program and alleged that he had been misled by the Navy and A-12 industry team. Subsequent analysis by James P. Stephenson and others concluded the Navy shared responsibility for a significant portion of the A-12’s failures and sought to deflect blame to the General Dynamics-McDonnell Douglas team. However, at the time of the ATF source selection, the A-12 tarnished both the reputations and financial solvency of General Dynamics and McDonnell Douglas.
“The A-12 helped us because nobody’s buying on promises anymore,” Sweetman quoted a Lockheed executive as saying. In contrast to the B-2 and A-12, Lockheed’s F-117A Nighthawk performed beyond expectations between January and February 1991 during the Gulf War.
Lockheed’s F-22A proposal also may have appeared lower-risk because it entailed fewer design changes from its YF-22. Lockheed’s Navy Advanced Tactical Fighter (NATF) was more similar to its ATF offering as well, whereas Northrop’s NATF featured an entirely different canard diamond wing planform.
The Navy canceled the NATF in its fiscal 1991 budget request, but Lockheed ATF Program Manager Sherman Mullin nevertheless credited the Navy’s support as a contributing factor toward its ATF victory. “The Navy still got a vote in the ATF competition, and, as we found out later for certain, it casted for our F-22 team,” he said. Mullin also cited Gen. (ret.) Alton D. Slay as having markedly improved Lockheed’s technical proposals through a series of red team reviews. “We would have never won the tough ATF competition without the contributions of Slay and his team,” Mullin said.
Measure of Merit
Industry teams perceived different nuances within the ATF concept of operations and took divergent paths with their designs. In reflecting on his nearly six-decade aerospace career, Lockheed Martin Skunk Works Engineering Technical Fellow Leland Nicolai captured this concept as the measure of merit: the “qualitative aircraft characteristic that describes what is really important to the customer.” The measure of merit is often an emotional or subjective customer preference that is otherwise unstated in the standard list of requirements. Both teams’ designs had to navigate the Air Force’s conflicting requirements in LO, maneuverability and supercruise performance.
In a 2021 interview with C.W. Lemoine, former ATF Technical Director Eric Abell recalled telling his Air Force colleagues the following regarding each company’s briefings prior to the demonstration/validation (dem/val) phase: “Northrop is an engineering company which has hired some managers, and Lockheed is a management company which has hired some engineers.” Both teams’ designs and marketing approaches thoroughly reflected this prescient observation.
Northrop’s measure-of-merit approach focusing on supercruise and LO matched comments in late 1985 from then-ATF Program Manager Albert Piccirillo comparing the ATF concept to World War I submarine warfare, with the aim being “to kill without being seen, disengage and disappear.” Chief Scientist Yu-Ping Lui recounted Northrop’s sentiment upon hearing Lockheed’s source selection: “I was crushed because we knew the numbers. We knew the RCS [radar cross-section] numbers they accomplished compared to ours. And there’s no reason that we would lose. So I would not accept the fact.”
Lockheed’s strategy gauged maneuverability as the measure of merit, focusing on the preferences of its user base and evaluation authority instead of what was explicitly stated in the RFP. “I think we gave the Air Force the airplane they secretly wanted, when Northrop built the airplane that was exactly what the Air Force asked for,” YF-22 Test Pilot Tom Morgenfeld recalled.
Northrop YF-23 Test Pilot Paul Metz would later describe showmanship as a key ingredient in Lockheed’s victory relative to the engineering logos of Northrop. Given the compressed schedule of the dem/val phase, Northrop concluded that the 90-day preparation time for high angle-of-attack (AoA) testing was not justified. The YF-23 PAV-1 demonstrated 25-deg. AoA, but Northrop provided engineering data showing its aircraft was controllable to 60 deg. Lockheed made high AoA testing a focal point of its dem/val flight-test program.
Similarly, Lockheed demonstrated live test firings of both the AIM-9 and AIM-120 air-to-air missiles, while Northrop declined to conduct live-fire launches and instead collected weapon bay data. Metz believes Lockheed’s displays left a lasting impression on its customer and described its approach as emulating General Dynamics’ iconic YF-16 versus F-4 turn radius demonstration, which was widely used to sell the program.
Lessons Learned
As the Air Force looks to award the NGAD contract this year, what will be the service’s measure of merit? A key factor will be how Air Force Secretary Frank Kendall’s extensive acquisition experience influences the source selection criteria. Kendall served as the undersecretary of defense for acquisition, technology and logistics from May 2012 to January 2017. In October 2012, he instructed DARPA to study next-generation air dominance concepts and subsequently launched the Aerospace Innovation Initiative in fiscal 2015. It could be fitting if Kendall serves as the NGAD SSA.
In 2019, according to then-Air Force Chief of Staff Gen. David Goldfein, the NGAD family of systems consists of five core elements: the platform itself, collaborative combat aircraft, adaptive-cycle engines and two classified components. The Air Force has spent at least $7 billion to date in adjusted fiscal 2025 dollars on pre-EMD efforts for NGAD, including technology maturation and risk reduction—closely matching the approximately $6.4 billion government contribution to the ATF pre-EMD. Industry provided at least an additional $4 billion for the ATF. For that sum of approximately $10.4 billion, the service demonstrated and prototyped the following programs to support the ATF:
- Ultra Reliable Radar program to mature active, electronically scanned array technologies.
- Pave Pillar integrated avionics architecture.
- Integrated Electronic Warfare Systems.
- F-15 Short-Takeoff-and-Landing/Maneuver Technology Demonstrator for thrust-reversing and supermaneuverability.
- Four prototype air vehicles (YF-22 and YF-23) with flightworthy YF119 and YF120 engines.
- Ground-tested XF119 and XF120 engines.
- Full-scale RCS pole models to validate LO performance.
- Two flying avionics laboratories to validate sensor fusion.
The Aerospace Innovation Initiative supporting NGAD similarly produced X-plane demonstrators, with at least one flying by September 2020. Far fewer details about these demonstrators have been disclosed relative to the ATF, but following historical trends, these demonstrators likely tested individual pieces of NGAD.
Kendall may take lessons from the Long-Range Strike Bomber competition, including funding both teams through the preliminary design review to reduce risk and mandating that early EMD prototypes be fitted with all the core avionics systems needed for initial operating capability. Both teams also are expected to submit digital designs utilizing model-based simulations. Notably, Boeing is believed to have gained an initial edge in this field by leveraging its commercial Black Diamond processes. As with the ATF and Joint Strike Fighter programs, NGAD demonstrators will likely play a pivotal role in deciding source selection.
The ATF experience also highlights how winning proposals are more than the promise of next-generation aircraft designs. Teams of highly experienced engineers, program managers and acquisition community officials are required to translate designs into operational aircraft, which is a particularly troublesome point when Lockheed and Boeing arguably both suffer from poor program and project management.