Interface Requirements and Experimental Hypotheses
Primary: Quick and Accurate Collection of Real-Time Intermodal Data
The primary purpose of the system being tested was the quick and accurate collection of real-time information arriving via vocal messages into a structured database. It was hypothesized that even the minor visual differences between the three interface variants would have an impact on the timing and accuracy of the intermodal process. The impact of the voice-to-text transcription simulation on the third interface was a point of particular interest, as it could have been used as a verification of the message that was heard, hypothesized to result in a slower response time, but better accuracy.
Secondary: Development and Maintenance of Situational Awareness
The secondary purpose of the interfaces being tested was the establishment and maintenance of situational awareness by the user. Several research programs have addressed aspects of situational awareness applied to decision making. Boyd’s Observe-Orient-Decide-Act (OODA) loop framework (Boyd 1996) is one of the best- known and often cited. The stimulus-hypothesis-option-response (SHOR) model (Wohl 1981) is very similar, but there is no indication that Boyd and Wohl were aware of one another’s work (Grant and Kooter 2005).
The definition of situational awareness provided by Endsley was adopted for this work. Endsley’s model of situational awareness provides for a decomposition into three well-defined levels, which support a well-tested objective evaluation methodology: The situation awareness global assessment technique (SAGAT) (Endsley 1988, Endsley et al. 1998). SAGAT calls for a simulation to be paused at a random point, have the informational displays blanked, and then query the subject’s recollection and understanding of the frozen situation across the three levels of situational awareness: (1) perception of elements in the current situation, (2) comprehension of the current situation, and (3) projection of the future state of the situation.
The simulation system driving the experiment was configured to deliver a partial SAGAT assessment at a random time once the fast message delivery period had been running long enough to establish a moderate level of situational complexity.* Only a partial SAGAT assessment was possible as the timing and sequencing of events was controlled by the simulation to be sufficiently randomized that participants could not predict even the very near-term future, which could have impacted the speed and accuracy of their data entry. Therefore, no assessment of level-3 situational awareness (projection of the future state) was made. '
Within the three interface designs being tested in this experiment, indicators were placed to show the number of incidents per neighborhood, number of committed resources, number of current incident types, and so on. These indicators would be updated by the simulation, and so always contain accurate information, even if it had not been properly recorded by the subject. On the first interface type, which was the nongraphical variant, the situational information would be displayed as simple numbers. On the other two interface variants, the information would be displayed as numbers supplemented by small bar graphs. The experimental hypothesis was that the bar graphs would support better situational awareness, but that the text transcriptions of the audio messages, which were visible only on, interface #3 might counteract this effect.
-  * A quantitative proxy for the concept of “situational complexity” was established for this experiment.The contributing factors were the number of concurrent incidents within the simulation, as well as ashort-term average of the message arrival rate.
-  An additional factor that led to the exclusion of an assessment of level-3 situational awareness wasthe time constraints on each simulation session. Had the simulations been extended in time, allowingindividual resources to work through several cycles of activity, a subject might legitimately be able tomake a determination of how much longer an incident was likely to last or perhaps which would be thenext resource to clear their assignment and become available for another.