Cognitive Task Analysis Approach
A specific research approach for measuring individual and team cognition within CSE is a CTA. This approach employs many of the previously identified KE methods in a systematic manner to understand the cognition necessary to perform a specific task. Schraagen et al. (2000, p. 3) define a CTA as “the extension of traditional task analysis techniques to yield information about the knowledge, thought processes, and goal structures that underlie observable task performance.” Although a CTA is directly related to KE methods, it is a separate research approach. There is often confusion regarding KE methods and CTAs, with many publications using these terms interchangeably. From our perspective, while KE methods and CTAs share commonalties, a CTA is a goal-directed approach to understanding cognition of a specific task, whereas KE is a meta-family of methods. The relationship between the two is that a CTA’s success is dependent on utilizing many KE specific methods.
In CTA, cognitive refers to the methods underlying reasoning and knowledge, such as: perceiving and paying attention, having the knowledge, skills, and strategies to adapt in dynamic situations, and knowing the purposes, goals, and motivations for cognitive work. Task, refers to the outcome people are working toward; not the activities they undertake on the way to achieving that outcome. Finally, analysis, refers to decomposing an item into its component parts in order to better understand those parts and how they relate to each other (i.e., the process from bottom-up to top-down) (Crandall et al. 2006). Overall, CTA is a process that researchers can use to determine the key drivers of cognition (i.e., determining accurate and complete descriptions of cognitive processes and decisions) in a variety of applications (Crandall et al. 2006; Clark et al. 2007). However, there are many methods that can be used during a CTA, and choosing the right one for the right task can be challenging for a researcher. Therefore, it is important to know which CTA methods will be successful in different situations. Fortunately, there are reviews outlining how to choose methods and techniques for conducting a CTA (Crandall et al. 2006; Clark et al. 2007). Crandall et al. (2006) outline what they delineate as three main aspects of a successful CTA: (1) KE, (2) data analysis, and (3) knowledge representations. Each of these aspects consists of their own specific considerations. For instance, KE methods are distinguished using two categories: (1) data collection and (2) method focus. Data collection simply refers to how the data were collected, which might be through interviews, self-reports (such as surveys, questionnaires, diaries, and logs), observations, or automated capture by computers. The second category, method focus, refers to aspects of time, realism, difficulty, or generality. These are explained in the following paragraph.
If the method focuses on time, then the researcher will want to know: when did the events happen? For instance, retrospective data allows researchers to find incidents based on particular or specific events, and also allows them to focus on certain kinds of events and aspects of cognitive performance. These types of data can be obtained via interview or self-report; guided questionnaires can be particularly effective because they can help people remember their experiences in more detail, ideally allowing them to provide a description of how they made their judgments and decisions. KE methods may also be focused on realism, that is, was the data gathered in real-world settings? In this case, researchers often use simulations, for instance: flight simulators can recreate real-world events, like accidents, and are used in individual and team trainings and to help develop new technologies. Difficulty is another possible focus—how challenging is the case and how tough is it to collect the data? Fields like healthcare, aviation, or nuclear power provide examples of difficult situations, such as accidents, where incident-based methods (observing and interviewing some specific accidents) can be used to understand how people make sense of and respond to these situations; hopefully providing insight into the sorts of errors people might make and how safety can be improved. Finally, the last focus is generality—how specific is the task? In this situation, researchers seek to elicit declarative knowledge in a given domain. Techniques such as concept mapping, in which core concepts and their relationships are depicted, can be used to survey general knowledge, specifically the goals people have when completing a task and how those goals are prioritized and related to each other (Crandall et al. 2006).
Stepping back from KE and moving to Crandall et al. (2006) second aspect of CTA, data analysis comprises: data preparation, data structure, discovering meaning, and representing key findings. Data preparation refers to assessing how complete and accurate the whole data set is (i.e., all data recorded should have a clear label and complete identifying information). The next piece of data analysis and data structure requires dissecting the data in order to rearrange it and find patterns inside it; this can be done, for instance, through coding, lists, and descriptive statistics. Discovering the meaning entails extracting key questions, issues, and emergent threads of meaning from the data (some procedures: categories, ratings, and statistical test). Finally, representing the key findings involves converting the data’s discovered meaning into a visual form, bringing its story to the forefront (some techniques: charts, graphs, storyboards, and concept maps) (Crandall et al. 2006).
The third and final aspect of CTA is knowledge representation, which focuses on the critical tasks of data visualization, presenting findings, and communicating meaning. This aspect is also inherently linked to data analysis, and the combination of these two aspects provides different representations of the products from the analysis process than the representations produced from KE alone. In this case, there are four types of analytic products that are typically yielded: (1) textual description; (2) tables graphs and illustrations; (3) qualitative models (e.g., flowcharts); and (4) simulation, numerical, and symbolic models (e.g., computer models) (Crandall et al. 2006).