Primary finding
Probable cause
The pilot's failure to use checklist procedures when switching operational control from PPO-1 to PPO-2, which resulted in the fuel valve inadvertently being shut off and the subsequent total loss of engine power, and lack of a flight instructor in the GCS, as required by the CBP's approval to allow the pilot to fly the Predator B. Factors associated with the accident were repeated and unresolved console lockups, inadequate maintenance procedures performed by the manufacturer, and the operator's inadequate surveillance of the UAS program.
Investigator assessment
Analysis narrative
The unmanned aircraft (UA), a Predator B, collided with the terrain following a loss of engine power while patrolling the southern U.S. border on a Customs and Border Protection (CPB) mission. The UA's takeoff was delayed due to the inability to establish a communication link between the UA and Pilot Payload Operator (PPO)-1 console during initial power-up. After troubleshooting the problem, an avionics technician switched the main processor cards between PPO-1 and PPO-2. Personnel who were maintaining the unmanned aircraft system (UAS) stated there were very few spare parts purchased with the UAS, which is why they switched the main processor cards instead of replacing the card in PPO-1. The link was subsequently established, and the flight was initiated. The flight was being flown from a ground control station (GCS), which contained two nearly identical control consoles: PPO-1 and PPO-2. Normally, a certified pilot controls the UA from PPO-1, and the camera payload operator (typically a U.S. Border Patrol agent) controls the camera, which is mounted on the UA, from PPO-2. Although the aircraft control levers (flaps, condition lever, throttle, and speed lever) on PPO-1 and PPO-2 appear identical, they may have different functions depending on which console controls the UA. When PPO-1 controls the UA, movement the condition lever to the forward position opens the fuel valve to the engine; movement to the middle position closes the fuel valve to the engine, which shuts down the engine; and movement to the aft position causes the propeller to feather. When the UA is controlled by PPO-1, the condition lever at the PPO-2 console controls the camera's iris setting. Moving the lever forward increases the iris opening, moving the lever to the middle position locks the camera's iris setting, and moving the lever aft decreases the opening. Typically, the lever is set in the middle position. Console lockup checklist procedures indicate that, before switching operational control between the two consoles, the pilot must match the control positions on PPO?2 to those on PPO-1 by moving the PPO-2 condition lever from the middle position to the forward position, which keeps the engine operating. The pilot stated in a postaccident interview that, during the flight, PPO-1 locked up, so he switched control of the UA to PPO-2. In doing so, he did not use the checklist and failed to match the position of the controls on PPO-2 to how they were set on PPO-1. This resulted in the condition lever being in the fuel cutoff position when the switch to PPO-2 was made, and the fuel supply to the engine was shut off. With no engine power, the UA began to descend. The pilot realized that the UA was not maintaining altitude but did not immediately identify that the condition lever was in the fuel cutoff position. The pilot and avionics technician decided to shut down the entire system and send the UA into its lost-link profile, which is a predetermined autonomous flightpath, until they could figure out what the problem was. After the system was shut down, the UA descended below line of sight (LOS), and communications could not be reestablished. The UA began to fly its lost-link profile as it descended to impact with the terrain. When the UA lost engine power, it began to operate on battery power. On battery power, the UA began to shed electrical equipment to conserve electrical power. In doing so, electrical power to the transponder was shut down. This resulted in air traffic control not being able to detect a Mode C transponder return for the UA as it descended below the bottom of the temporary flight restricted airspace. The primary radar return was also lost when the UA descended below the LOS in the mountainous area. The investigation revealed a series of computer lockups had occurred since the CBP UAS began operating. Nine lockups occurred in a 3-month period before the accident, including 2 on the day of the accident before takeoff and another on April 19, 2006, 6 days before the accident. Troubleshooting before and after the accident did not determine the cause of the lockups. Neither the CBP nor its contractors had a documented maintenance program that ensured that maintenance tasks were performed correctly and that comprehensive root-cause analyses and corrective action procedures were required when failures, such as console lockups, occurred repeatedly. Review of the CBP's training records showed that the accident pilot had recently transitioned from flying the Predator A to flying the Predator B and had only 27 hours of Predator B flight time. According to the CBP, the pilot was given verbal approval to fly its Predator B with the caveat that the pilot's instructor would be present in the GCS when the pilot was flying. This verbal approval was not standard practice for the CBP. The instructor pilot was in another building on the airport and did not enter the GCS until after it was shut down and the UA entered the lost-link procedure. The investigation also revealed that the CBP was providing a minimal amount of operational oversight for the UAS program at the time of the accident.
Source record
Factual narrative
History of FlightOn April 25, 2006, about 0350 mountain standard time, a MQ-9 (Predator B) aircraft, serial number BP-101, call sign OMAHA 10, collided with the terrain approximately 10 nautical miles northwest of the Nogales International Airport (OLS), Nogales, Arizona. The unmanned aircraft system (UAS) was owned by U.S. Customs and Border Protection (CBP) and operated as a public-use aircraft. The flight was operating in night visual meteorological conditions (VMC). An instrument flight rules (IFR) flight plan had been filed and activated for the flight. The unmanned aircraft (UA) sustained substantial damage. There were no injuries to persons on the ground. The flight originated from the Libby Army Airfield (FHU), Sierra Vista, Arizona, at 1851, on April 24, 2006. The wreckage was located at 0630. The flight was being flown from a ground control station (GCS) located at FHU. The GCS contains two nearly identical pilot payload operator (PPO) consoles, PPO-1 and PPO-2. Normally, a certified pilot controls the UA from PPO-1, and the camera payload operator (typically a U.S. Border Patrol agent) controls the camera, which is mounted on the UA, from PPO-2. Although the aircraft control levers (flaps, condition lever, throttle, and speed lever) on PPO-1 and PPO-2 appear identical, they may have different functions depending on which console controls the UA. When PPO-1 controls the UA, movement the condition lever to the forward position opens the fuel valve to the engine; movement to the middle position closes the fuel valve to the engine, which shuts down the engine; and movement to the aft position causes the propeller to feather. When the UA is controlled by PPO-1, the condition lever at the PPO-2 console controls the camera's iris setting. Moving the lever forward increases the iris opening, moving the lever to the middle position locks the camera's iris setting, and moving the lever aft decreases the opening. Typically, the lever is set in the middle position. In addition to the pilot and payload operator, other personnel present in the GCS were an avionics technician and a sensor operator, both of whom are General Atomics Aeronautical Systems, Inc. (GA-ASI) employees. GA-ASI manufactures the Predator B and was contracted by CBP to fly and maintain BP-101. The flight was originally scheduled to take off at 1700 but was delayed because of the inability to establish a communication link between the UA and PPO-1 during initial power up. The avionics technician stated he powered down the UA and downloaded the system status. He then recycled the power on PPO-1 and PPO-2, but again he was not able to establish an uplink on PPO-1. The technician did not attempt to gain an uplink on PPO-2 during either of these power-ups. The technician reported that he again captured the system status data on his laptop and called his supervisor at the manufacturer's facility in California for assistance. He reported that his supervisor and the technical support personnel with whom he spoke had not seen this type of problem before. They recommended that he switch the main processor cards between PPO-1 and PPO-2. The technician stated that he did this, powered up the system, and was able to establish an uplink on both PPO-1 and PPO-2. He stated that everything operated normally at this point, and he went off duty at 2000. Because the UA typically stays airborne for extended periods of time, more than one pilot is scheduled to fly during each mission. The pilots rotate flying duties every couple of hours throughout the duration of the flight. The pilot who was flying the initial part of the accident flight, including the takeoff, was not the accident pilot. The accident pilot reported that he was scheduled to work from 1900 on April 24, 2006, until 0500 on the day of the accident. The accident pilot reported that he took control of the flight at 1900 when BP-101 was already airborne and operating in the temporary flight restriction (TFR) airspace. He reported that he flew from 1900 until 2100. At 2100, another pilot resumed control of the flight. The accident pilot took control of the flight again at 0300 and was scheduled to fly until 0500. He stated that the change-over briefing at 0300 was normal and that nothing had changed with the flight. He reported that, shortly after he resumed control of the flight, the lower monitor screen went blank on PPO-1. The screen then reappeared, but the telemetry (transmitted data) was locked up, so he decided to switch control of the UA to PPO-2. The pilot stated that he informed the Border Patrol agent who was at PPO-2 that he needed to switch positions. The Border Patrol agent stated that he moved away from PPO-2 and left the GCS. The pilot stated that he verified the ignition was "hot" on PPO-2 and that the stability augmentation system was on. He reported that, at some point, he used his cell phone to call another pilot (who had been his instructor) to discuss what was going on. At the time, the instructor was in a hangar building across the ramp. Checklist procedures state that there should be pilots in both the PPO-1 and PPO-2 seats before switching control of the UA from one PPO to the other. CBP stated that its procedures call for the avionics technician to assume the duties of a co-pilot for the purpose of assisting with the checklist items before switching control from one PPO to the other. This did not occur during the accident sequence. The pilot stated that he did not use the checklist when making the switch. Checklist procedures state that before switching operational control between the two consoles, the pilot must match the control positions on the new console to those on the console that had been controlling the UA. The pilot stated in an interview that he was in a "hurry" and that he failed to do this. The condition lever on PPO-2 was in the fuel cutoff position when the switch from PPO-1 to PPO-2 occurred. As a result, the fuel was cut off to the UA engine when control was transferred to PPO-2. The pilot stated that, after the switch to the PPO-2 console, he noticed that the UA was not maintaining altitude, but he did not know why. He decided to shut down the ground data terminal (GDT) so that the UA would begin its lost-link procedure. This procedure called for the UA to autonomously climb to 15,000 feet above mean sea level (msl) and fly a predetermined course until contact could be reestablished. With no engine power, the UA continued to descend below line-of-sight (LOS) communications, and further attempts to reestablish contact with the UA were not successful. The pilot reported that the instructor pilot entered the GCS shortly after the avionics technician turned off the GDT. He informed the instructor of what occurred, and the instructor looked at the controls and stated that the controls were not positioned correctly. The instructor tried to reestablish contact with BP-101 in both the GCS and the mobile GCS (MGCS); however, BP-101 had already descended below LOS, and contact could not be reestablished. The avionics technician who was positioned at the multifunction workstation (MFW) in the rear of the GCS recalled the events, as follows. He stated that he heard the pilot say that PPO-1 had locked up. He then noticed that the chart display on his monitor had locked up. The technician stated that he walked up to the front of the GCS and looked at the status-warning screen on PPO-2, which indicated that PPO-1 was locked up. He advised the pilot that they needed to switch control to PPO-2. He then went back to the MFW to open up another program, which showed him what processes were running on PPO-1 so that he could record this information. The technician then returned to the front of the GCS, at which time the pilot was using his cell phone to call for support. He advised the pilot again that they needed to switch control from PPO-1 to PPO-2. The technician stated that the p