Cognitive Task Analysis Interviews
Cognitive task analysis techniques are used to describe the unobservable cognitive aspects of task performance. These approaches were developed to identify and describe the cognitive processes and strategies underlying decision-making, judgements and goal generation (Militello and Hutton 1998). Data collection typically relies upon retrospective interviews with experts, with structured questions that probe the key aspects of the decision-making process. A key aim of cognitive task analysis approaches is to uncover the tacit knowledge of experts, which is difficult to verbalise or to capture through traditional interview techniques (Crandall et al. 2006).
There are several methods for cognitive task analysis, with a leading method emerging being the Critical Decision Method (CDM; Klein and Armstrong 2005,
Klein et al. 1989). CDM is a semi-structured interview technique that uses structured prompts to aid recall of past events and explore factors that shape decision-making. It can be used to identify training requirements, generate training materials, provide input for the development of decision support systems and evaluate how new decision support systems impact task performance (Klein et al. 1989). It evolved from the Critical Incident Technique (Flanagan 1954) and was developed to study naturalistic decision-making strategies of experienced personnel. It has been applied within complex and dynamic domains such as firefighting, military and emergency medicine (Klein et al. 1989). It has also been used in previous transport research in aviation (Plant and Stanton 2013), maritime operations (0vergard et al. 2015) and rail (Tichon 2007).
Probes are used within the semi-structured interview format to elicit the following types of information (see O’Hare et al. 2000 for full question set):
- • Goal specification: What were your specific goals at the various decision points?
- • Cue identification: What features were you looking at when you formulated your decision?
- • Expectancy: Were you expecting to make this type of decision?
- • Conceptual model: Are there any situations in which your decision would have turned out differently?
- • Situation assessment: Did you use all the information available to you when formulating the decision?
- • Decision blocking - stress: Was there any stage during the decision-making process in which you found it difficult to process and integrate the information available?
- • Analogy/generalisation: Were you at any time reminded of previous experiences in which a similar decision was made?
Interviews conducted with CDM should be transcribed verbatim before analysis. There are several options for analysing the interview transcripts. For instance, responses can be coded using automated thematic analysis software programmes such as Leximancer™ or can be coded manually using a structured a priori framework, or using an exploratory emergent theme approach (Wong 2004).
BOX 2.3 USING THE CRITICAL DECISION METHOD WITH DRIVERS
As part of this research programme, we used CDM to better understand driver behaviour at rail level crossings. The outcomes of this are described in Chapter 4, along with information about how we used the method with novice as well as experienced drivers, to understand the differences in the decisionmaking processes of these two groups.