SYSTEMS-FOCUSSED ANALYSIS METHODS

Data collected using the methods described in Sections 2.2 and 2.3 can be analysed using a variety of techniques, including simple descriptive analyses and inferential statistics to compare behaviour and performance between conditions. Although these methods are useful to provide an initial understanding and overview of the raw data, a key advantage of many of these data collection methods is the ability to analyse the data in greater depth using additional human factors analysis techniques that offer specific insights about system functioning.

Network Analysis

Network analysis can be used to understand the relationships between tasks, social actors (humans and technology) and knowledge concepts. In this research programme, we have generally used network analysis in relation to the latter, to focus on the situation awareness of users at rail level crossings. Here, the data input is the verbal protocol transcripts.

Situation awareness networks illustrate the concepts verbalised by participants and the relationships between them, to provide a detailed picture of what participants’ situation awareness comprised at key points. Within situation awareness networks, the nodes represent pieces of information or concepts relevant to situation awareness (e.g. speed, lights).

Situation awareness networks can be constructed manually or via software tools such as Leximancer, which uses text representations of natural language to interrogate verbal transcripts and identify themes, concepts and the relationships between them. The software has previously been used for situation awareness network construction and analysis in various on-road studies (Salmon et al. 2013a, 2013d, 2014a, 2014b, Walker et al. 2011). An important strength of this approach is that it provides a reliable, repeatable process for constructing situation awareness networks.

A further strength of using networks to describe participant situation awareness is the ability to analyse them in various ways using network analysis metrics. Various metrics are used to analyse situation awareness networks. Metrics that may have relevance include the following:

  • Network density: This metric represents the level of interconnectivity of the network in terms of relationships between nodes. Density is expressed as a value between 0 and 1, with 0 representing a network with no connections between nodes, and 1 representing a network in which every node is connected to every other concept (Kakimoto et al. 2006).
  • Sociometric status: This metric provides a measure of how ‘busy’ a node is relative to the total number of nodes within the network under analysis (Houghton et al. 2006). Nodes with sociometric status values greater than the mean sociometric status value plus one standard deviation may be designated as ‘key’ (i.e. most connected) nodes in the social and situation awareness networks.
  • Centrality: This metric measures the standing of a node within a network in terms of its distance from other nodes in the network (Houghton et al. 2006). A ‘central’ node is relatively close to all other nodes in the network in terms of connections. That is, an interaction with other nodes in the network is achieved through the lowest number of connections.

The use of these metrics enables conclusions to be made regarding the structure of situation awareness. In an on-road study context, for example, this allows conclusions to be made regarding:

  • • The most important pieces of information being used when negotiating road environments (sociometric status, centrality).
  • • Differences in the connectedness of information when drivers negotiate different types of road environments, such as intersections versus rail level crossings (e.g. network density: does situation awareness become harder to attain and is the driver required to use more information?).
  • • Instances where important pieces of information (e.g. speed reductions) are not well integrated in drivers’ understanding of the situation (sociometric status, centrality).

Analysis of data via network analysis metrics is normally supported through a network analysis software tool such as Agna. This involves importing the network data into Agna in the form of a matrix of the concepts (e.g. car, traffic lights, pedestrian, speed) and the relationships between them (e.g. ‘car’ was mentioned with ‘speed’ seven times; ‘car’ was mentioned with ‘pedestrian’ once). The network metrics are then calculated automatically by the software tool by selecting the appropriate metrics.

BOX 2.8 USING NETWORK ANALYSIS TO UNDERSTAND SITUATION AWARENESS AT RAIL LEVEL CROSSINGS

In this research programme, we applied network analysis to the transcripts of verbal protocols obtained during on-road studies to understand driver situation awareness at rail level crossings (see Chapter 4).

 
Source
< Prev   CONTENTS   Source   Next >