Human Systems Integration Division @ NASA Ames

The effort described in this report was a project to support the Federal Aviation Administration (FAA) in evaluating and comparing modeling approaches to predict pilot performance in NextGen operations. This research effort was intended to assess the current state of the art regarding modeling of pilot performance on the flight deck and to provide guidance regarding research needs in modeling and validation. One hundred and eighty-seven references were identified that examined computational models of pilot performance. We identified 12 different features of each of these modeling efforts to facilitate comparisons. A subset of these features focused on the quality and extent of validating model predictions against pilot-in-the-loop simulation data. In addition, we identified 12 different aspects of pilot performance that were the focus of the model in question. We report a deep dive analysis of six of the modeled aspects of pilot performance: pilot error, workload, multi-tasking, situation awareness, pilot-automation interaction, and roles & responsibility, focusing attention on the nature and findings of the several models within each section, including the extent and quality of validation and verification. We then describe in detail, several model architectures, that appeared in more than one of the modeling efforts in our deep dives. We then present overall conclusions and recommendations in a final section. We emphasize the importance of continuing to validate the models, in particular those that accommodate more than one aspect of pilot performance within their architecture.