EVALUATION OF A NEW DESCENT PROCEDURE FOR AIRLINE PILOTS

Barry Crane^1, Everett Palmer² & Nancy Smith¹

¹San Jose State Foundation

²NASA Ames Research Center

A new air traffic control tool was developed to assist controllers in sequencing airport arrival traffic. This tool depends on aircraft flying idle power descents to meet a series of assigned speed and altitude targets. The evaluation of flight crew performance of a new air traffic control (ATC) procedure that coordinates an FMS-equipped aircraft's descent path with this desired path is the subject of this study.

The Center-TRACON Automation System (CTAS) and the Descent Advisor. NASA has developed a set of new computer-based air traffic control tools called the "Center-TRACON Automation System," or CTAS. CTAS uses predicted flight trajectories for individual aircraft to generate advisories that assist controllers in the management of aircraft arriving at an airport. These trajectory predictions are based on aircraft-specific performance models and use current wind measurements along with the aircraft's current airspeed, altitude, heading, and flight plan (Erzberger, 1994). One part of CTAS, the Descent Advisor (DA), is designed to predict an aircraft's position along a known descent trajectory. The DA uses this predicted descent trajectory to identify potential conflicts between aircraft and to estimate when each aircraft will arrive at a "metering waypoint" on the descent path. The DA then presents controllers with a top of descent point and cruise and descent airspeed advisories designed to improve the sequencing and spacing of these aircraft at this metering waypoint in preparation for landing (Williams & Green, 1991).

CTAS/DA Cockpit Descent Procedures. Special ATC clearances enable the controller to assign descent parameters to an aircraft that will match its descent path to the path the Descent Advisor used to make its arrival time estimate. These clearances include a descent airspeed, an assigned descent point, and a speed and altitude crossing restriction at the metering waypoint. The clearances and the constraints they impose on the veritcal flight path constitute the CTAS Descent Procedure.

Descent Advisor Field Evaluations. NASA researchers, working with the Federal Aviation Administration (FAA) and United Airlines, conducted two field evaluations of the CTAS/DA in the Denver area in 1994 and 1995 (Green & Vivona, 1996; Cashion, et al., 1995; Palmer, et al., 1997). The second evaluation tested the accuracy of the DA's arrival time predictions for commercial flights arriving at Denver International Airport. 185 aircraft participated in this evaluation, including nine different aircraft types.

Flight crews who participated in the 1995 Field Evaluation received a standard Jeppesen chart describing the CTAS/DA clearances as part of a "CTAS Descent Procedure" to be flown during the evaluation. Aircraft specific flight manual bulletins that described how to comply with the procedure were distributed by the airlines to pilots in each participating fleet. NASA researchers developed these procedures and documents with design assistance from airline pilots, airline training staff, and air traffic controllers. Qualified line pilots participated in simulator testing of the procedure, clearances and briefing documents as they were developed.

Despite the care taken during procedure design, problems were observed during the Field Evaluation that indicated that the clearances were too long, and contained too much information (Palmer, et al., 1997). Pilots often needed to have clearances repeated or clarified, and frequent readback errors occurred. Controllers were concerned about the length and number of communications needed for each aircraft. Because of an altitude sector boundary at 27,000 feet (flight level 270, or FL270), two descent clearances had to be issued: the first cleared the aircraft from cruise altitude to FL270, and the second, following the handoff to the low altitude controller, from FL270 to the metering fix. The need for two descent clearances was explained in the procedure documents, but still resulted crew misunderstandings about the procedure's intent.

CTAS Clearances used in 1995 Field Evaluation

Initial CTAS Notification (Sector 3 controller):

"Company 123, expect CTAS descent, expect to cross TOMSN at FL190 and 250 knots, maintain FL___."

Descent Clearance (Sector 14):

"Company 123, maintain FL___ until __miles E/W of __, descend and maintain FL270, maintain __Mach/ __knots in the descent."

Further Descent Clearance (Sector 13):

"Company 123, continue descent at __knots, cross TOMSN at and maintain FL190 and 250 knots."

Pilot and controller participants from the 1995 Field Evaluation met with NASA researchers in February 1996 to review results from the evaluation and to discuss possible improvements to the procedure. Suggestions included 1) reducing clearance length, by transferring some of the needed procedure information to a published chart; 2) simplifying clearance handling at the altitude sector boundary in the descent path; and 3) changing the procedure's name. Their suggestions resulted in the "Precision Descent" procedure flown by pilots in the current simulator study. Unlike the CTAS Descent Procedure, the Precision Descent uses terse, non-standard clearances that rely on a procedure chart for their correct interpretation.

Precision Descent Clearances

Initial Notification (Sector 3):

"Company 123, expect Precision Descent."

Descent Clearance (Sector 14):

"Company 123, cleared for Precision Descent, __miles E/W of __, __Mach/__ knots."

A Charted Procedure. Clearance phraseology used in the 1995 Field Evaluation was designed to be as standard as possible, and to stand alone. The use of a published, charted procedure to communicate compliance requirements for the Precision Descent allows reduction of clearance length and the amount of information that needs to be provided to flight crews by air traffic controllers. The procedure's Chart presents non-varying compliance targets (the bottom-of-descent crossing restrictions), leaving only the descent information that varies with each flight (descent airspeed and assigned descent point) to be communicated in special clearances. The Chart explains the non-standard, abbreviated terminology used in these two clearances.

"Precision Descent" Name. The CTAS Descent Procedure was renamed the Precision Descent to avoid confusion of "C-TAS" with "T-CAS" (an acronym for Traffic Collision Avoidance System). It was also intended to remind pilots of the "Profile Descent," an older descent procedure that also includes a sequence of altitude and speed restrictions.

Eliminate Sector Boundary Clearance. The Precision Descent eliminates the need for a second descent clearance after crossing the high/low altitude sector boundary by having the high altitude controller clear the aircraft from cruise altitude to the metering waypoint in the low altitude sector. Special clearances can be used when needed to stop the aircraft at the sector boundary (or any other mid-descent altitude).

Precision Descent Training. During the 1995 Field Evaluation, briefing was designed to ensure that aircraft flew descent trajectories that would allow evaluation of the Descent Advisor's predictive accuracy. Pilots received briefing material describing the Descent Advisor, the CTAS Descent Procedure, and aircraft-specific technique recommendations. The procedure's clearances used conventional terminology, and NASA observers flew in the cockpit on many participating flights, answering pilots' questions about the procedure's purpose and its compliance requirements. The minimum training needed for correct performance of the CTAS Descent Procedure was not established. One goal of the present study was to evalute two alternative methods for operational introduction of the Precision Descent.

The evaluation was designed to test use of a charted procedure to reduce clearance length for CTAS trajectory clearances. Scenario variables were manipulated to determine how challenging or unusual conditions would affect performance of the procedure. A second purpose of this study was to compare two methods for introduction of the Precision Descent to skilled pilots. Both methods rely exclusively on paper documents and ATC clearances to support procedure performance. Results from this comparison are reported by Smith & Palmer ( 1997).

Methods

Eight type-rated flight crews who fly for major air carriers were recruited to participate in the study. Each flight crew flew eight Precision Descents in a Boeing 747-400 full-motion simulator, for a total of 64 descents. Pilots were seated in the simulator and given 10 minutes to review procedure briefing material before the first flight. No additional training or performance feedback was provided during the study.

Scenario Design

Each run began in the late cruise phase of flight roughly 160 miles northwest of Denver International Airport. Scenarios were designed to be as realistic as possible, with an experienced air traffic controller handling communications with the flight crew, and live background chatter on the air traffic radio frequency. Minor distractor tasks were scripted into each flight (flight attendent queries, traffic and weather advisories, minor equipment failures) to manipulate workload during descent preparation. All but two of the runs were flown to landing.

Scenario Variables. Eight different scenarios were created by manipulating eleven different scenario variables. Four of these altered the context for the descent (Table 1A). These included wind speed and direction; the offset of the Precision Descent's assigned descent point (ADP) from the correctly-computed VNAV top-of-descent; how close to the ADP the descent clearance was issued; and the excess energy the aircraft had in descent.

The remaining seven scenario variables combined different possible operational variations of the procedure and its clearances (Table 1B). These included changing the assigned descent speed in mid-descent; issuing mid-descent altitude hold or speed change clearances; cancelling the speed restriction at the metering waypoint; lateral route changes; cruise altitude changes; and the use of intermediate altitude descent clearances. The same scenario order was used for each crew.

Scenario Clearances. Clearance phraseology and timing was scripted for each run. Variations on the Precision Descent clearance that could be issued for traffic, or to prevent an aircraft from crossing a sector boundary were used in runs 3, 4 and 5. Two different versions of this "Intermediate Altitude Clearance" were compared between crews, with one version using a single clearance, and the other a two-part alternative:

Intermediate Altitude Descent Clearance, Version A:

"Company 123, cleared for Precision Descent, __miles E/W of __, __Mach/__ knots, except maintain FL270."

Intermediate Altitude Descent Clearance, Version B:

"Company 123, cleared for Precision Descent, __miles E/W of __, __Mach/__ knots." (followed by:)

"Company 123, after established in the descent maintain FL270 for traffic."

Procedure Compliance

The procedure Chart described six different performance requirements for the Precision Descent. Flight crew compliance on every descent was obtained for each of these six measures. Table 2 lists the frequency of deviations from these charted requirements for each run.

Top of Descent. Three procedure requirements related to the top of descent: 1) notify ATC if the VNAV top of descent is more than 5 nm from the assigned descent point. 2) Maintain cruise airspeed until reaching the assigned descent point. 3) Begin descent within 5 nm of the assigned descent point.

Descent Speed. Descent speed tolerance was +/- 10 knots.

Bottom of Descent. The metering waypoint, TOMSN, had a charted crossing restriction of FL190 and 250 knots.

ATC Communications

ATC communications were analyzed for six of the eight runs; Table 3 presents the results from this analysis. These measures include: 1) elapsed time for Descent Clearance transactions, from initial ATC contact until the end of the last crew reply; 2) number of crew readback errors (incorrect parameter values); 3) number of incomplete readbacks (clearance parameters missing from readback); 4) requests from the crews for ATC to clarify clearance or Precision Descent requirements.

Descent Clearance Types. Scenarios 3 and 4 used the Intermediate Altitude Descent Clearances shown above. Single clearance format "A" was used for the first four crews; '2-part' format "B" was used for the last four.

Pilot Questionnaires

Pilot responses to specific questions that addressed the acceptability of the procedure, the difficulty of performing the procedure or any of its tasks, and the clarity (or lack of clarity) of the procedure and its clearances are summarized by question type.

Procedure Acceptability. Five questions asked pilots to rate the acceptability of the Precision Descent clearances, the ADS, the ADP, and the Precision Descent overall. The average response to each of these questions was positive, from 0.3 to 1.9 on a Likert scale that ranged from -3 (completely unacceptable) to +3 (completely acceptable).

Perceived Difficulty. Responses to five of the six questions related to procedure difficulty indicate that pilots did not find the procedure difficult to perform. The one exception was: "Did you feel rushed during any part of the Precision Descent?" with 14 of 16 pilots answering "yes."

Confusion Reports. Most pilots said they did not find the procedure's clearances "unclear or possibly confusing." However, 13 of 16 pilots did report contacting ATC to clarify one or more of the clearances issued during the simulator study.

Several of the scenario manipulations appeared to affect crew performance and understanding of the descent procedure. The four most important factors are described below.

Mach/IAS descent speed assignments. Many crews were unaware that a "Mach/IAS" combination was a valid entry on the CDU's Descent page. These crews sought other ways to enter the two values into the FMS, and often entered the assigned descent Mach as cruise target in the MCP or the CDU's Cruise page and the IAS on the Descent page. Insufficient crew knowledge about an infrequently used CDU page resulted in deviations from cruise Mach on many Precision Descents with Mach/IAS speed assignments that were flown in the simulator study. This error will also occur in actual operations unless pilots are made aware of this feature of the CDU's Descent page.

Non-standard clearances. Most of the clearance handling problems occurred on scenarios that used an intermediate altitude descent clearance. Time to issue descent clearances for runs 3 and 4 were 10 to 20 seconds longer than for other runs, and incomplete readbacks and clarification requests were most frequently observed in run 3. These problems were reduced, but not eliminated, by the two-part clearance "B". Timing the second (intermediate altitude) clearance to avoid interfering with crew handling of the Precision Descent clearance was difficult; and some crews confused the flight level value in the second clearance with the indicated airspeed in the first (e.g., FL270 and 280 kts.). Problems caused by other non-standard clearances included crew confusion about the crossing speed restriction at the metering waypoint. This confusion occurred in two different contexts: 1) when the Precision Descent clearance was issued after the crossing speed restriction was lifted, and 2) when cleared airspeed was changed during descent. Roughly 80% of crew clarification requests occurred after mid-descent clearances that changed the Assigned Descent Speed, and were related to the descent speed and crossing restriction.

Excess energy. Excess energy during descent appeared to reduce the crews ability to meet the charted crossing restriction at the metering waypoint. Six of the twelve TOMSN altitude deviations and eight of the sixteen TOMSN speed deviations occurred during runs 4 and 7, the runs with the highest excess energy. Most of these crews did notify ATC that they would be high or fast at the metering waypoint, indicating that they understood the procedure's requirements, but were unable to comply.

Clearance length. The normal Precision Descent Clearance does not appear to cause problems; incidence of readback errors and incomplete readbacks was low. Communication problems increased, however, when any additions were made to the basic Precision Descent clearance (such as adding a Mach value to the descent speed).

The next four scenario variables did not appear to affect crew performance of the procedure.

Procedure novelty. With two notable exceptions, performance of the Precision Descent on the first run was excellent. All clearance readbacks were correct and complete, and compliance with charted procedure requirements was almost complete. The two exceptions both related to speed compliance during descent. Nearly half of the descent speed errors and almost one third of the crossing restriction airspeed errors that were observed in this study occurred on the first descent. From recorded crew comments, pilots understood the procedure's compliance requirements, but wanted to assess the ability of the computed VNAV descent to meet the assigned speed targets. On subsequent flights, crews used speed brakes or throttle adjustments, or changed autoflight modes in order to meet airspeed targets. It's not clear if this crew "experimentation" on their first Precision Descent would have occurred in actual flight, or if this was a simulator-related artifact.

Separation of ADP and VNAV T/D. Descent outside of the 5 mile ADP tolerance was the least frequently observed compliance error in the study. In all but five of the 64 runs, descent was begun within the charted 5 mile tolerance of the assigned descent point. Even when the descent clearance was issued late (runs 2-8) and the VNAV T/D was outside the 5 mile range (runs 3, 4, 7 and 8), crews apparently had no problem initiating descent within the assigned descent window.

Clearance timing. During the first descent, flight crews were given 40 nm (~5 minutes) to prepare for descent from the time the Descent Clearance was issued. Clearances in all subsequent flights were issued between 10 and 20 nm (~2-3 minutes) from the assigned descent point.

Comparison with 1995 Field Evaluation. The average transaction time for ATC communications was reduced from 32 seconds reported for the 1995 Field Evaluation (Palmer, et al., 1997) to 22.5 seconds in the Precision Descent study. Clearance readback errors or incomplete readbacks occurred on 22% of the transactions in 1995, compared to 25% in the simulator study. Direct comparison of flight crew performance measures is difficult because of differences in the kind of data that was obtained, in the context for performance, in crew briefing protocols, and in scenario difficulty.

The Precision Descent succeeded in reducing clearance transaction times by 30% by using non-standard clearances to support a charted procedure, and there is evidence that it reduced other problems associated with clearance transactions. Crew compliance with the procedure under "normal" conditions was excellent; especially notable was the first-time performance observed in the study. Some deviations from "normal" descent conditions that were tested in the scenario manipulations created problems for flight crews, however; these included any changes to clearance phraseology, especially the addition of an intermediate altitude clearance; interruptions to the descent or mid-descent speed changes; Mach/IAS descent speed clearances; and descent trajectories that cause the aircraft to have excess energy. Some recommendations for dealing with these problems are: 1) If mid-descent clearances during the Precision Descent (interruptions to the descent profile, or changes to the Assigned Descent Speed), restate the metering waypoint crossing restrictions. 2) Alert controllers to the possibility of crew confusion with modified clearances. 3) Alert flight crews of FMS-equipped aircraft that Mach/IAS descent speed assignments can be entered on the CDU DES page. 4) Avoid issuing clearances that may result in high energy descents.

Cashion, P., Feary, M., Goka, T., Graham, H., Palmer, E., & Smith, N. (1995) Development and initial field evaluation of flight desck procedures for flying CTAS descent clearances. Presented at the The Eighth International Symposium on Aviation Psychology, Columbus, Ohio.

Erzberger, H. (1994). Concerning the Center-TRACON Automation System (CTAS). Presented to the Federal Aviation Administration Research and Development Advisory Committe, July 12, 1994, Washington, D.C.

Green, S., Vivona, R. (1996) Field evaluation of descent advisor trajectory prediction accuracy. Presented at the AIAA Guidance Navigation and Control Conference, July 29-31, 1996, San Diego, CA

Palmer, E., Crane, B., Johnson, N., Smith, N., Feary, M., Cashion, P., Goka, T., Green, S., & Sanford, B. (1997) Field evaluation of flight deck procedures for flying CTAS descents. Presented at the The Ninth International Symposium on Aviation Psychology, Columbus, Ohio.

Smith, N. & Palmer, E. (1997) An analysis of the relationship between procedure support documentation and task performance. Presented at the The Ninth International Symposium on Aviation Psychology, Columbus, Ohio.

Williams, D. H. & Green, S. M. (1991). Airborne four-dimensional flight management in a time-based air traffic control environment, (NASA Technical Memorandum #4249).