Human simulation has been used successfully to guide design. However, past approaches have simulated relatively simple tasks in which performance variability arises from people's physical rather than cognitive limitations. In contrast, our approach focuses on limitations of human memory and attention that lead to error.
We are developing a computer program intended to partially automate the process of predicting excessive operator memory requirements in new procedure and equipment designs. The system includes components for simulating a physical operating environment, a human operator, and an experimental observer.
The simulation of the physical environment is specialized for the domain of TRACON air traffic control. This system component includes models of a radar scope and other equipment that a user may be interested in evaluating.
The human operator model describes domain-independent sources of performance variability including perceptual limitations (e.g. acuity, field-of-view, discriminability) and motor limitations (Fitt's law, limited hand-independence). The centerpiece of the model describes plan execution mechanisms which enable the simulated agent to carry out complex tasks with an approximately human proneness to habit captures and other systematic forms of error.
This element of the model is intended to be significant in two ways. First, other simulations of human operators are incapable of operating in domains as complex as air traffic control. These models are therefore unable to determine whether designs violate even straightforward constraints --- e.g. insuring that an operator doesn't have to look at two widely separated objects at the same time --- because there is no way of determining whether the agent would ever get into the proscribed situation. Our operator model implements the capabilities of a "sketchy planner" [Firby 1989] and is thus capable of interleaving task execution, gradually specifying plans based on intentionally acquired information, and other relatively complex activities.
Second, most other systems neglect to consider the role of cognitive factors in performance, especially factors associated with executing tasks in complex, dynamic environments. This unnecessarily limits the usefulness of such systems for predicting design-facilitated operator error since many of the most important kinds of operator error can be predicted (in a statistical sense) if cognitive factors are taken into account.
In addition to components for simulating a physical environment and human operator, the system simulates an observer charged with noticing anomalies in the operator's performance in order to provide guidance to a designer. This part of the system consists of a component for monitoring deviations from expected performance and a component for explaining and classifying the expectation failure using model-based-reasoning techniques. The need to automate this task follows from the vast amount of data generated over the course of a simulation, although human observers of human subjects are faced with a similar, immensely time-consuming task.
In air traffic control, and in many other domains, design decisions are constrained not only by human factors and physics, but by materials costs, operator retraining costs, compatibility with past equipment and procedures, and any number of political factors. Reports on prototype simulation studies usually describe observed problems with a design, but do not attempt to prescribe specific solutions. Similarly, our system outputs problems experienced by a simulated operator while using specified equipment and procedures, leaving the decision about how to cope with these problems in the hands of the user.