The Dynamical Systems Approach to Team Cognition: Theory, Models, and Metrics
Terri A. Dunbar Jamie C. Gorman David A. Grimm and Adam Werner2
Georgia Institute of Technology1 United States Military Academy2
Dynamical Systems Viewpoint of Team Cognition.................................................55
Variability and Stability.......................................................................................56
Attractors and Repellers......................................................................................57
Team Dynamics across Domains.............................................................................58
Command and Control........................................................................................59
Research in Command and Control................................................................59
Methods in Command and Control................................................................60
Research in Collaborative Problem Solving...................................................61
Methods in Collaborative Problem Solving...................................................61
Research in Healthcare...................................................................................63
Methods in Healthcare....................................................................................64
Research in Human-Autonomy Teaming......................................................65
Methods in Human-Autonomy Teaming.......................................................65
Current Issues in Team Dynamics...........................................................................66
Description vs. Explanation................................................................................66
Limitations and Future Directions...........................................................................68
The complexity of the modern workplace has resulted in the more frequent use of teams or teams of teams to fulfill the goals and mission of the organization. Subsequently, it is important for team researchers to understand how they can best support these teams through the careful measurement and assessment of team processes and team effectiveness. To this end, in this chapter we focus on a dynamical systems approach (both theory and methods) to team processes, namely team cognition. Our goal in highlighting this approach is to provide readers with an understanding of how they can apply concepts from dynamical systems to team cognition, identify regularities in system dynamics across different work domains, and incorporate methods from dynamical systems approaches to their own research.
A dynamical system is a system (i.e., a set of interacting components; Bunge, 1979) that changes over time whose behavior is tracked to predict future states of the system (Abraham & Shaw, 1992). Informal definitions of dynamics in team cognition investigate how the construct of interest (e.g., team composition; Mathieu, Tannenbaum, Donsbach, & Alliger, 2014) changes over time but may neglect to consider multiple levels of the system, the constraints within which the team operates, or the type of explanation or mechanism posited. The type of explanation in dynamical systems theories differs from traditional cognitive theories and neuroscience theories in that rather than focusing on static explanations (e.g., knowledge structures), linear explanations (e.g., input-process-output models), or material explanations (e.g., neurons) of cognitive processes, dynamical systems theory focuses on temporal, linear and nonlinear, and nonmaterial explanations, such as changes in processes over time through forces that repel, attract, or stabilize cognitive processes (Juarrero, 1999). These types of explanations are what novices often struggle with when attempting to adopt dynamical systems theories and methods.
Systems approaches to human behavior historically originate in functionalism, an early framework from the late 19th century that sought to understand mental activities in terms of the utility of the behaviors they served (e.g., James, 1890). Functionalism stood in stark contrast to the structuralism approaches at the time, which used introspective techniques to uncover the components of the mind, often ignoring the context within which mental activities were occurring. Many 20th- century psychologists appeared inspired by functionalist approaches, including Vygotsky with activity theory (Vygotsky, 1978) and Gibson with the ecological theory of perception (Gibson, 1979). Rather than focusing on individual elements that constitute mental activities or the static representations of these activities, these functionalism-inspired theories used the person-object/environment system as the primary unit of analysis, a major theoretical transition that is still a part of dynamical systems theories today.
Although functionalism-inspired frameworks lend themselves well towards systems theorizing, historically, the methodologies behind these theories often lacked the sophisticated mathematical techniques that we associate with dynamical systems methods today. The roots of modern dynamical systems methods include mathematical techniques such as those designed to measure the degree of self-organization (i.e., where systems-level order arises from component-level interactions; Ashby, 1947) of chaotic systems (i.e., systems that produce different behaviors depending on the initial conditions of the system; Lorenz, 1963) using chaos theory-inspired methods, such as fractal analysis. Many current proponents of ecological theory also use dynamical systems methods to answer their research questions about perception and action (e.g., Coey, Kallen, Chemero, & Richardson, 2018) because these methods allow one to model the relationships between different parts of a system.
Why study team cognition as a dynamical system? A dynamical systems approach, as we will show in this chapter, offers researchers much in the form of predicting future system states and novel forms of assessment and training. Although this approach differs from what is typically studied in the behavioral sciences, we hope to illustrate that the concepts and techniques from a dynamical systems approach can be accessible to a wide range of researchers.
To this end, in this chapter we review concepts from dynamical systems theory, dynamical systems methods used in different work domains, and current/future directions for this approach. First, we provide a background by defining team cognition as a dynamical system and applicable concepts from dynamical systems theory. Second, we highlight four different team domains, focusing on the characteristics of the system constraints in these domains, applicable research findings, and the types of methods applied in this domain. Third, we discuss the most current work in the dynamical systems approach to team cognition: sources of variation underlying team effectiveness and real-time dynamics. Finally, at the end of the chapter, we discuss the limitations and future directions of the approach.
DYNAMICAL SYSTEMS VIEWPOINT OF TEAM COGNITION
What does team cognition mean from a dynamical systems viewpoint? From this perspective, teams are dynamical systems. Teams perform within a set of constraints (i.e., conditions that alter the opportunities for thoughts and actions; Juarrero, 1999), such as the goal the team is working towards, the tasks individual team members are performing, and the environmental context the team is performing within (Gorman, 2014). Within this set of constraints, team members interact with one another to work towards their goal. These goals and subgoals may change over time, depending on the constraints of the system and how the team interacts over time. The interactions between team members are important from this perspective, both in the history of interactions and the interactions as they are currently ongoing, because these interactions are what helps the team maintain and monitor their progress towards their overall goal.
Team cognition, then, from a dynamical systems viewpoint, focuses on how the interactions between team members over time are embedded within the constraints of the team’s goal and task as well as the individual properties of the team members (e.g., their capabilities, limitations, and roles). This suggests that the areas of measurement for team cognition could include task and goal constraints, team interactions (verbal and nonverbal), team member knowledge and experience, and team member actions (motor behavior) or physiology (autonomic and central). We will see examples of some of these areas of measurement in the upcoming section, “Team Dynamics across Domains.”
Although the methods used to measure team dynamics are similar to those from various areas of psychology or human factors, the concepts in dynamical systems differ from what is often encountered in the behavioral sciences. Traditional psychology concepts are often rooted in individual thoughts (e.g., mental models) and behaviors (e.g., individual operator’s actions) rather than in processes that change over time. Investigating team cognition from a dynamical systems viewpoint requires shifting from an individualistic perspective to a systems perspective, where concepts are now focused on relationships across levels of measurement (e.g., local variability/ global stability), between people (e.g., interactions between components), changes over time (e.g., phase transitions), or in response to outside disturbances (e.g., skilled behavior). To this end, we discuss some major concepts in dynamical systems theory, both the basic terminology and how the concept is applied to teams. The following concepts are important for understanding the rest of the book chapter: variability, stability, attractors, phase transitions, and skill.
Variability and Stability
Variability refers to fluctuations in patterns of behavior over time, whereas stability is its opposite, resistance to change over time. Variability and stability need not be considered a dichotomy; rather, it should be considered a continuum. When the goal of the research is to identify stable differences in behavior or cognition between experimental conditions, variability represents measurement error and stability is the more desired characteristic. However, when the goal of the research is to track cognitive processes that change over time, variability and stability both reveal important information about the characteristics of the system, such as whether the system is undergoing a change in the system state or has settled on a particular system state.
This distinction between variability and stability also depends on the timescale of measurement. For example, we can measure team cognition at different points in time, such as the team’s communication during one work session or across the entire duration of time the team has worked together. At a small scale, the variations in the team’s communication may appear to be highly variable. However, on a longer timescale, the variations appear less random and more stable. This overall stability occurs because the team members’ communication is in service to the overall team goal, keeping the team on track to reach their goal. This phenomenon is called local variability with global stability, where behavior at the local level appears highly variable or random, but the behavior is stable and less variable at a global level (Gorman, Dunbar, Grimm, & Gipson, 2017).
Metastability refers to a stable state of the system other than the system’s state of least energy (Treffner & Kelso, 1999). In a metastable state, stability is maintained through constant actions taken against the energy being pushed into the system. As a result, the output of the system in a metastable state is unpredictable. A small amount of energy might not change the system state, but a large amount of energy could push the system into a new state. For example, imagine a ball resting at the top of a hill with a slope down to either side of it. This location represents a metastable state. If the ball is strongly pushed towards either direction (i.e., energy is supplied to the system), then the ball will leave the metastable state towards a state of least energy, the bottom of the hill. If the ball is slightly nudged, then the ball may stay in the metastable state or move towards the state of least energy depending on the strength of the nudge, the steepness of the slopes, and the size of the ledge that the ball is resting on. The team’s overall dynamics can be thought of as a metastable state, where communication serves to push the team into different system states (Gorman, Amazeen, & Cooke, 2010a).
Attractors and Repellers
An attractor is a system state that the system will evolve to regardless of the initial state of the system (Abraham & Shaw, 1992). A repeller, on the other hand, is the opposite—a system state that the system will not approach. Attractors and repellers are either inherently stable (outside disturbances, or perturbations, have little effect), unstable (perturbations have a large effect), or metastable (stability is being actively maintained). The influence of an attractor or repeller is evident from the time course of the system. From the initial system state, the system undergoes a transition period where the system’s behavior fluctuates erratically until it either settles on a behavioral state (i.e., attractor), or settles away from a behavioral state (i.e., repeller). When predicting the future state of the system, a dynamical system will continue to gravitate towards attractors and away from repellers. In teams, attractors have been studied in motor coordination, where the primary interest was attractor formation under certain task constraints (e.g., Gorman & Crites, 2015).
Phase transitions are shifts in the system from one system state to another (Haken, 1983). Phase transitions represent qualitative changes in the system state, such as the transition from a liquid to a gas. Phase transitions can be identified in the system trajectory through increased variability in the overall system’s behavior. Changing the control parameter, a variable that when changed leads to a change in overall system behavior (Thelen, Ulrich, & Wolff, 1991), can induce phase transitions in the system. For example, phase transitions between different forms of infants’ stepping behavior can be induced by changing the task context, such as stepping under water or on a treadmill, and detected through changes in the variability of the infants’ stepping behavior itself (Thelen et ah, 1991).
High-performance skill is often equated to the amount of training hours the individual or team has completed, the difficulty of acquiring expertise, and the qualitative differences between expert and novice performance (Schneider, 1985). A dynamical systems view of skill focuses less on the overall performance or expertise of the individual or team as the locus of skill and more on the pattern of behaviors the individual or team engages in and the actions they take towards outside disturbances
(i.e., perturbations). The three dynamical systems characteristics of highly skilled behavior are flexible, adaptive, and resilient (Thelen & Smith, 1994). To be flexible, a team must have a repertoire of behaviors that considers the varying task, team, and environmental constraints. To be adaptive, a team must rapidly modify their behaviors in response to perturbations. To be resilient, a team must recover quickly from perturbations. In teams, skilled behavior has been examined in response to perturbations (e.g., Gorman, Cooke, & Amazeen, 2010b).
TEAM DYNAMICS ACROSS DOMAINS
Teams operate differently depending on their domain. Some teams may operate within very specific task constraints yet work flexibly across different environments. Other teams are constrained to work towards a specific goal but can perform their task in whatever way helps them reach their goal. Despite these differences, the dynamical principles underlying team cognition are similar across different work domains. In this section of the chapter, we provide an overview of the characteristics of and constraints within four work domains, highlighting the research findings and methods used to assess team dynamics in that domain. The four domains we highlight are command and control, collaborative problem solving, healthcare, and human-autonomy teaming. Table 3.1 summarizes the main constraints, findings, and methods for each domain.
Summary of Team Dynamics across Domains
Military rules & regulations Ranks
Experience affects novel task performance.
Team members need to know who knows what information. Mixing up team composition can result in more skilled behavior.
Expertise in problem type
Macrocognition theory fits CPS team behavior.
Distinct qualitative problem solving phases can be captured as phase transitions.
Lag sequential analysis Entropy
Changing team member roles Changes in workload Emergent tasks
Events occurring during simulation are observable in the neurodynamic and communication patterns of the team.
Discrete recurrence analysis
Capabilities of autonomy Autonomy’s interaction patterns
All-human teams and HATs perform equally well but differ in how they coordinate.
Command and Control
Command and control (C2) teams are directed by a commander who executes specific orders to the team to fulfill an overall mission goal (Department of the Army, 2012). Though prevalent in other domains, C2 teams are largely associated with the military, paramilitary, and emergency response teams. C2 teams operate within very specific constraints (e.g., military rules and regulations, ranks of the commander and team members, or mission goals) with highly trained individuals under highly regimented interaction patterns. Each aspect of the system is critical to the success of the C2 team.
C2 teams can be centralized to one environment or decentralized and distributed throughout an environment or operating area. Decentralized command and control (DC2) teams are becoming more prevalent given advances in technology and the military’s desire for increased mobility in combat (Heininger, 2016). Rather than being centrally located, DC2 teams flexibly and heterogeneously distribute resources across multiple environments that are tactically relevant to the overall mission (Gorman, Cooke, & Winner, 2006b). DC2 teams typically involve multiple dozens of people but could be hundreds, if not thousands, of individuals scattered throughout an operating environment. No one individual or operator is necessarily responsible for being aware of the entire DC2 environment, but instead all are responsible for their own local environment (Gorman et al., 2006b). Compared to the C2 system constraints, DC2 system constraints incorporate an additional layer of interaction complexity due to the number of people working in the system; however, the DC2 system has much greater flexibility in their environmental constraints by comparison.
Research in Command and Control
One important aspect of the C2 system is that the experience of the individual team members is vitally important to novel tasks. C2 teams who are experienced working together perform better on a novel task than teams with no experience working together (Cooke, Gorman, Duran, & Taylor, 2007). From a dynamical systems perspective, this novel task performance benefit is due to the experience team members get by constantly coordinating with one another. New interaction patterns emerge over time through repetitive coordination in the C2 problem space (Cooke, Gorman, Myers, & Duran, 2013).
C2 and DC2 systems do not need to be aware of what each team member is doing during task performance; rather, they should coordinate flexibly as needed to maintain a dynamic and collaborative situation awareness. Distributed situation awareness models (e.g., Stanton, Stewart, Harris, et al., 2006) suggest that teams should not share awareness because each team member may have a different purpose and, because of these differences, over-sharing could potentially be confusing or overwhelming to the individual team member. This implication is particularly important for DC2 teams, which can involve thousands of individuals scattered across the globe, where shared awareness is simply not feasible. Stanton and colleagues (2006) suggest instead that individuals should be aware of who has what knowledge and how that knowledge is useful to them rather than knowing everything about the situation.
Another unique finding relevant to C2 systems is that changes in team composition and retention interval can lead to improvements in the C2 system’s performance (Gorman, Cooke, Pederson, et al., 2006a). Gorman et al. (2006a) examined the effects of different team compositions (intact vs. mixed) across different retention intervals (short vs. long). Mixed C2 teams exhibited more skilled behavior compared to intact C2 teams when given a longer retention interval between missions. The findings also imply that team mixing in C2 systems over longer periods of time can lead to more adaptable, better performing C2 teams. Gorman et ah (2006a) believe that this occurs because the change in teammates causes a perturbation in the team’s coordination, ultimately leading to more adaptability and resilience than in intact teams.
Methods in Command and Control
Most methods for assessing C2 teams are not dynamical systems methods. However, systems methods such as event analysis of systematic teamwork (EAST) methodology (Stanton et ah, 2006; Walker, Gibson. Stanton, et ah, 2006) still approach C2 teams as systems rather than individual team members. The EAST methodology uses data from hierarchical task analyses, observations of task performance, and debriefing interviews to produce three system representations: social network (i.e., who communicates to who), task network (i.e., goals of different team members), and knowledge network (i.e., relationship between the different types of information needed for successful task performance). These networks can ultimately be used to improve the design of systems that manage these networks.
Dynamical systems methods for assessing C2 teams directly measure team member interactions in response to a changing operational environment. A process- based measure, coordinated awareness of situations by teams (CAST; Gorman et ah, 2006b), assesses how team members interact to coordinate in nonroutine situations. CAST addresses unanticipated events by using roadblocks (i.e., nonroutine situations embedded in the task) to perturb the steady state of a C2 team. Teams must coordinate their perceptions and actions by interacting to overcome the roadblock. A highly skilled C2 team will successfully coordinate around these roadblocks.
Collaborative problem solving (CPS) involves mutual coordination across people to solve a problem (Roschelle & Teasley, 1995). Although CPS is not a domain per se, CPS is often studied within particular domains where teams utilize CPS as part of their work, such as firefighting and student teams. Considering the dynamics of the CPS system, CPS teams operate within a few constraints, such as the experience of the individual team members with the problem and the problem space itself, which shape the interactions the CPS teams must take towards their overall goal of solving the problem or problem set. One theoretical approach to CPS that captures system dynamics is the macrocognition in teams theory.
Macrocognition in teams aims to determine how teams build knowledge structures while engaging in CPS and how teams accomplish their goals through coordination
(Fiore, Smith-Jentsch, Salas, Warner, & Letsky, 2010). Macrocognition is the “process of transforming internalized knowledge into externalized team knowledge through individual and team knowledge building processes” (Fiore et ah, 2010. p. 204-205). Macrocognition in teams consists of five components: Individual and team knowledge building, internalized and externalized knowledge, and team problem-solving outcomes (Fiore et ah, 2010). During the individual and team knowledge-building phases, individual team members process their problem space and disseminate this knowledge to their team, who translate the knowledge into actions that the team can take towards accomplishing their goal. Internalized knowledge refers to individual team member knowledge, whereas externalized knowledge refers to the relationships between knowledge and team-level knowledge. Team interactions influence team problem-solving outcomes and contribute to whether the problem is solved.
Research in Collaborative Problem Solving
Multiple studies provide support for the concept of macrocognition in teams during CPS. Hutchins and Kendall (2010) examined communication from experienced teams executing tasks such as from firefighters on 9/11. The researchers found that communication for firefighters fit within the five components of macrocognition in teams. Seeber, Maier, and Weber (2013) found similar results when examining distributed teams using collaborative software as a communication source. The distributed teams also generally followed the five components for macrocognition in teams.
Many theories in CPS, including macrocognition in teams, hypothesize that there are distinct qualitative phases that occur during successful problem solving. These qualitative phases are often measured through the team’s communication. Earlier work investigating the temporality of these phases focused on how the structure of the problem impacted the immediate transitions (identified through lag-sequential analysis) between different types of problem-solving communication (Kapur, 2011). More specifically, student teams working on ill-structured problems exhibited much more complexity in their temporal patterns, shifting often between different problemsolving phases, whereas teams working on well-structured problems tended to immediately transition to the same type of problem-solving phase. Recently, Wiltshire, Butner, and Fiore (2018) discovered evidence of phase transitions between problemsolving phases. In this study, the researchers integrated dynamical systems theory with existing CPS theory by tracking the variability of different types of coded communication associated with different problem-solving phases over time. The researchers found increased variability in communication codes prior to phase transitions into new problem-solving phases. Ricca, Bowers, and Jordan (2019) applied the same analytic techniques to the communication from teams of fifth-grade students engaged in a robotics engineering design project and also found evidence of phase transitions between problem-solving phases.
Methods in Collaborative Problem Solving
CPS team dynamics are primarily studied through verbal communications (e.g., Wilshire et ah, 2018), although researchers have also examined movement coordination during CPS (e.g., Wiltshire, Steffensen, & Fiore, 2019). Initial attempts investigating the temporality of CPS team dynamics used an analytic technique called lag sequential analysis (LSA), which analyzes a sequence for cross-dependencies (Kapur, 2011). More specifically, LSA compares the expected transition probabilities between observations to the actual transition probabilities, identifying the statistically significant transitions within the dataset (Bakeman & Gottman, 1997). Other techniques investigating temporal dynamics in CPS teams include identifying phase transitions in the entropy of coded team communication (Wiltshire, et ah, 2018) and movement coordination across multiple timescales (Wiltshire et ah, 2019).
In the CPS study identifying phase transitions in coded communication, the researchers applied a sliding window entropy technique to team communication (Wiltshire et ah, 2018), where entropy is measured across a window of time (e.g., 200 seconds), shifted up one unit of time (e.g., 1 second), recalculated over the new window of time, and so on across the entire communication time series to create an entropy time series. Here, entropy refers to the level of disorder in a signal, where high entropy is highly disordered and low entropy is highly predictable (Shannon & Weaver, 1949). There was high variability in the communication entropy prior to changes in the CPS phase, indicating a phase transition occurred.
The CPS research investigating movement coordination across timescales used cross-wavelet coherence (Wiltshire et ah, 2018), which measures the coherence (similar to a cross-correlation) and relative phase (i.e., the difference between the oscillations of two or more time series) between two time series across multiple frequencies and timescales (Issartel, Marin, Gaillot, Bardainne, & Cadopi, 2006). By using cross-wavelet coherence rather than only relative phase, the researchers were able to measure coordination across multiple time scales rather than a single timescale. Because this is a new area in CPS, further research is necessary to determine how well other dynamical systems concepts apply in this domain.
As in other domains, simulation has come to play a significant role in the training and practice of medical professionals (Aebersold & Tschannen, 2013). Indeed, a longitudinal study of nursing students (Hayden, Smiley, Alexander, et ah, 2014) found that student nurses receiving traditional training (no more than 10% clinical hours in simulation) and student nurses that received enhanced simulation training (50% clinical hours in simulation) had similar levels of clinical competency, nursing knowledge, and post-training clinical competency. Moreover, reductions in adverse patient outcomes in community hospitals have been linked to simulation training (Riley, Davis, Miller, et ah, 2011). Simulation training for healthcare teams often focuses on developing effective communication patterns with the ultimate goal of transferring these skills to real-world task performance. This training goal is particularly pertinent considering teamwork errors related to communication difficulties have been cited as one of the most prevalent factors contributing to medical mishaps (Sutcliffe. Lewton, & Rosenthal, 2004).
Teamwork measurement in the healthcare domain has traditionally centered on observational methods and metrics (e.g., TeamStepps). However, research on team dynamics in medical simulations has also begun to reveal how shifts in neurophysiological (Pappada, Papadimos, Lipps, et al., 2016) and communication (Gorman, Grimm, Stevens, et al., 2019) patterns can be used to identify increases in workload, emergent tasks (perturbations), and changing team member roles that can interfere with medical care. It has been argued that these dynamical systems methods for understanding shifts in teamwork in the medical domain have the potential for real-time monitoring and prediction of team interaction patterns associated with medical errors (Gorman et al., 2019).
Research in Healthcare
Stevens and colleagues (Stevens, Galloway, Halpin, & Willemsen-Dunlap, 2016b) have observed that prior to and during particularly challenging simulation training events, such as a patient seizing during surgery, the neural (EEG) dynamics of neurosurgical teams (described in the next section) enters a more disorganized state (“high entropy”) compared to nominal task conditions (“low entropy”). Furthermore, they have observed that more experienced teams demonstrate this effect to a greater degree than less experienced teams (Stevens, Galloway, Gorman, et al., 2016a), which suggests that neurodynamic patterns might be more easily tied to adverse events as teams gain experience working together. Stevens and colleagues have supported the external validity of their neurodynamic research by observing similar neurodynamic patterns in high school student teams (Stevens, Galloway, Berka, & Sprang, 2009) and submarine crews (Stevens, Gorman, Amazeen, et al., 2013). More recently, Stevens, Galloway, and Dunlap (2019) have validated their neurodynamic methods developed using simulations in a live operating room (OR).
Research has begun to tie these team neurodynamics to the behavioral and speech dynamics exhibited by medical teams during challenging simulation training events. Gorman and colleagues (2019) were able to automatically detect transitions in team communication patterns that corresponded to unexpected perturbations that interfere with patient care, including fire in the OR, changes in team composition, and handoffs, during medical training simulations. In some cases these speech dynamics were correlated with neural dynamics (Willemsen-Dunlap, Halpin, Stevens, et al., 2017). However, the more general finding is that speech and neural dynamics provide complementary real-time information streams that are diagnostic of anomalies and adverse events in the OR. For example, Gorman et al. (2019) describe a medical simulation in which the communication dynamics identified a change in team membership at the start of the surgery that neurodynamics did not, but that neurodynamics identified a patient seizing event during surgery (Stevens, Galloway, Willemsen-Dunlap, et al., 2018) that communication dynamics did not. These researchers concluded that highly skilled teams engage in a continuous stream of explicit and implicit team coordination (Entin & Serfaty, 1999) in which communication dynamics address the former and neurodynamics address the latter aspect of team coordination. Currently, these methods focus on identifying team coordination anomalies related to unexpected turns in the simulation scenario and how the team reacted to them. However, these methods may also be useful for revealing natural transitions (e.g., the transition from planning to task performance; Gorman et al., 2019) during which confusion and miscommunication are also prevalent.
Taken together, these studies have revealed objective metrics of team coordination dynamics that have the potential to provide real-time feedback for medical team simulation training and potentially in the live OR. However, further research using live OR teams as well as medical simulations is needed to elucidate how feedback should be provided to enhance team coordination dynamics and learning in medical teams in real time.
Methods in Healthcare
Much of the research we have described uses a method called team neurodynamics. Neurodynamic entropy maps millisecond level recordings of electrical activity at team members’ scalps using EEG onto information-based metrics of team synchrony (Stevens & Galloway, 2017). Essentially, high, medium, and low activation at the various EEG sensor sites are recorded over time for each team member, and these patterns of activation across team members constitute a symbolic set of team-member activation distributions (e.g., Team Member A - high activation/Team Member В - low activation is a different distribution than Team Member A - high/ Team Member В - high). Fluctuations in these team-level symbolic distributions are analyzed over time by tracking their entropy, where high entropy corresponds to changing team distributions and low entropy corresponds to stable team distributions. Other metrics based on this method allow researchers either to model neural synchrony at the team level or to examine shared and mutual information between team member’s neural patterns (Stevens et al., 2018). The resulting suite of measurements are called team neurodynamics.
The communication research cited in the healthcare domain utilized a method called discrete recurrence analysis (Gorman, Cooke, Amazeen, & Fouse, 2012a). Utilizing this method, a researcher can take any time or event series (such as a time series of speaker turn-taking events) and compute measures such as %DET (Webber & Marwan, 2015), which quantify the amount of organization in speech patterns (e.g., turn-taking patterns). The %DET measure (organization in turn-taking patterns) can be recomputed using a windowing procedure as new data come in (Gorman et ah, 2019). This results in a %DET (communication organization) time series. Anomalies in this time series correspond to unusual shifts in communication patterns that can be detected in real time. This is the method Gorman et ah (2019) used to detect significant transitions in speech patterns corresponding to nonroutine medical crises in the simulated OR.
Human-autonomy teaming (HAT) involves the coordination of human team members with autonomous, or synthetic, team members (Schulte, Donath, & Lange, 2016). Autonomy differs from automation in that autonomous systems perform entirely independently from the human operator, displaying some degree of intelligent behavior, whereas in automated systems, the system will simply perform the actions it was programmed to perform (Endsley, 2015). In HAT systems, team members are constrained not just by the individual team member capabilities, the team task, and the overall mission goal but also by the unique capabilities and interaction patterns associated with the autonomous agent.
Research in Human-Autonomy Teaming
Researchers have been investigating the dynamics of HAT in the context of remote piloted vehicle operations when there are two human team members (navigator and photographer) teamed with autonomy (pilot; Myers, Ball, Cooke, et al., 2017; Demir, Cooke, & Amazeen, 2018; Demir, McNeese, & Cooke, 2018; McNeese, Demir, Cooke, & Myers, 2018) as opposed to all-human teams. Several early experiments found that all-human and human-autonomy teams perform equally well but differ in their coordination dynamics because autonomy may not understand what it means to be a good teammate (e.g., pushing and pulling of information; back-up behaviors). In the case of HAT, the coordination dynamics rely on mutual adjustments of the human team members to facilitate human-autonomy coordination (e.g., McNeese et ah, 2018).
Similar to CPS, HATs apply to a variety of domains. For instance, many of the same principles underlying HATs discussed here are present in other applications such as self-driving autonomous vehicles (Campbell, Egerstedt, How, & Murray, 2010; Lugano, 2017), urban search and rescue (Krujiff, Janicek, Keshavdas, et ah, 2014), and other military applications beyond the remotely piloted vehicle operations, including unmanned ground vehicles and robotic teleoperations (Chen, Durlach, Sloan, & Bowens, 2008). Regardless of the domain, many of the potential problems that arise during HATs include brittleness (i.e., automation is unable to perform outside of designed parameters), lack of transparency (i.e., automation does not display what it is doing to the human), miscalibrated trust (i.e., human has an inappropriate level of trust in the automation), and lack of shared awareness (i.e., automation does not display the information it is using to perform the task; Shively, Lachter, Brandt, et ah, 2017). Considering that these common factors arise across a variety of HAT applications, we propose that a dynamical systems approach to HATs could benefit all of these applications through the methods described here.
More recently, human-autonomy team dynamics have been viewed from the perspective of human-systems integration by investigating coordination dynamics across operators (human and autonomy) and across vehicle and control parameters of the broader remote piloted vehicle systems (Grimm, Demir, Gorman, & Cooke, 2018). This research builds on the idea that to perform effectively, the system must exhibit skilled behavior by being flexible, adaptive, and resilient. The system accomplishes this through continuously reorganizing across operator, vehicle, and control layers in response to changing task demands (i.e., destabilization due to perturbations) and/or changing system configurations (i.e., a fundamental change in the coordination task; e.g., all-human vs. human-autonomy system configurations).
Methods in Human-Autonomy Teaming
Current methods in HAT investigate skilled behavior in the system by perturbing the team and determining how the team coordinates in response to the perturbation. One way to measure the system’s response to perturbations is through the length of time it takes for the system to organize, called time to system reorganization, relaxation time, or reorganization time. Time to system reorganization is calculated as the first significant peak from the time of the perturbation onset in a windowed entropy time series, described previously in the “Methods in Collaborative Problem Solving” section. Time to system reorganization can be used directly as a measure of team performance or as a predictor of system effectiveness and efficiency. For example, prior research correlated time to system reorganization with system effectiveness (i.e., a weighted composite performance score of various system parameters) and found that the configuration of the system impacted the relationship between these two variables (Gorman, Demir, Cooke, & Grimm, 2019).
CURRENT ISSUES IN TEAM DYNAMICS
Current work in the dynamical systems approach to team cognition is addressing some of the challenges from critics of this approach. For instance, dynamical systems methods have been criticized for its focus on descriptive methods such as curve fitting (Rosenbaum, 1998). Current research is now' focusing on the explanatory power of dynamical systems methods, such as identifying the sources of team effectiveness. Another area of current research is on the real-time dynamics of behaviors as the team task unfolds, as this will lend itself towards practical and useful applications for real world environments.
Description vs. Explanation
Descriptive methods for team cognition aim to describe w'hat team cognition is. In dynamical systems terms, this would be describing the dynamics of team cognition in different domains or under different constraints. Much of this research has focused on curve fitting the data to determine the underlying dynamics of the team’s coordination, such as fitting communication data to a pow'er-law' distribution or an exponential distribution (e.g., Dunbar & Gorman, 2014). However, the classical curve fitting method does not identify sources of variation (e.g., specific team member behaviors) that underlie team effectiveness, which is a focus of current work.
Current work on the dynamics of team communication focuses on identifying sources of team effectiveness by using a method of post hoc filtering to identify which team members contribute significantly to team communication during perturbations. The logic behind the filtering method is that we can identify w'hich team members contributed to the team’s communication dynamics by removing their inputs post hoc and identifying how this impacts the overall dynamics or reorganiza- tional behavior of the team. This process involves creating an initial time series consisting of a dynamical system measure such as %DET (Gorman, Hessler, Amazeen, Cooke, & Shope, 2012b) as derived from communication flow, or which speaker was speaking at a certain time throughout the time series (Grimm, Gorman, Stevens, et al., 2017). Each speaker is identified in this time series with a unique numerical identifier. It is critical that there are no speaker overlaps during this initial time series. Then a nonlinear prediction algorithm as developed by Kantz & Schreiber (1997) is used to generate another time series w'hich measures communication reorganization, where large values indicate a high degree of anomalous team behavior and a large amount of reorganization at the team communication level. The final step before applying the filtering method would be to use average mutual information (i.e., the measure of information contained in variable Y about variable X AMI; Abarbanel, 1996; Cover & Thomas, 2006) to generate another time series for each team member and help identify which team members may be driving team communication during perturbations.
After using this strategy to identify potential team members, team members are filtered by replacing their unique identifier in the original time series with a null value (0). Then the analysis is repeated with the hypothesized influential team member filtered out. If the overall activity of the team, as measured in the initial time series with the dynamical system measure, falls below the significance, then one can infer that the respective team member was a significant driving factor behind the communication reorganization. This finding may indicate leadership emergence because the significance of the team process dropped after the removal of the filtered team member. This process can be repeated for each team member during the filtering process. Consequently, this is strictly a post hoc filtering procedure currently. Future research could be carried out to the possibility of filtering such team members in true real-time settings.
Real-time dynamics allow researchers to understand how effective teams behave at the system level. Using concepts and measurements from dynamical systems theory, methods have recently been developed that identify real-time sources of variation during team coordination. For example, in medical simulation training, entropy has been used to track a team’s communication to better inform what areas of feedback instructors should target for their trainees following the training session (Grimm et al., 2017). Currently, the main benefit from the real-time method lies in team training and evaluation. However, this method could be used in other domains, for example, to identify how quickly the team responds to emergencies and how effectively the team responds to the emergency. Although many of these types of evaluations are conducted post hoc, real-time dynamics could be utilized as a method to help detect emergencies in real-life situations.
For more applied purposes, real-time analysis of team communication may allow teams to react faster to catastrophic errors by identifying anomalous team coordination patterns. For example, breakdowns in team communication are at least partially responsible for delayed responses to Hurricane Katrina (Leonard & Howitt, 2006). Real-time dynamics could help detect errors from incidents such as Hurricane Katrina much sooner than the methods that are currently available; however, software would need to be developed to facilitate real-time analysis, particularly realtime analysis of overt team member communication. In order to create this software, there are practical issues that would need to be solved first, for example, developing a method of handling several people communicating and talking at the same time. As such, software that identifies group members’ communication solely by their voices may not be enough. However, technologies could be used to record vocal patterns and vibrations, but these types of technologies would need to be unobtrusive. Another alternative to overt team communication would be a covert method, such as measuring the neurophysiology of the team. Past research has used neurophysiological measures to analyze team behaviors (e.g., Stevens et al., 2016b), which could be extended to real-time analysis.
Another useful application of real-time dynamics lies in simulation training. Simulation training has been used in both healthcare (McGaghie, Issenberg, Petrusa & Scalese, 2010) and military settings (Salas, Priest, Wilson, & Burke, 2006). Real-time methods could be applied to provide a portrait of overall team coordination across the duration of a mission or during the simulated surgery. If any crises occurred during the simulations, these measures could identify any useful or harmful team-level behaviors. This information could then be used to modify the team’s behavior during the simulation training.
LIMITATIONS AND FUTURE DIRECTIONS
One of the major limitations of the dynamical systems approach to team cognition is that dynamical systems theory is not a conventional mode of explanation in experimental psychology, human factors, and allied disciplines. Dynamics does not offer traditional modes of explanation, such as material explanations (e.g., neurons), but rather focuses on unseen forces that repel, attract, and stabilize cognitive processes. Hence, explanations in terms of attractors or phase transitions are less tangible than hypothesizing team knowledge structures and/or mental simulation as the fundamental mechanism of team cognition. It is the case, however, that these latter constructs operate under the constraints of dynamic interactions, and in the future, we need to do a better job educating psychologists and the public at large on how these intermediate representations are subject to the principles of dynamical organization if we are to fully explore the impact of dynamical systems approaches to team cognition.
A related limitation of the dynamical systems approach involves the concern of process vs. mechanism and what dynamical systems really explain about team cognition. Whereas systems are coordinated through observable/repeatable processes, underlying (hypothesized) mechanisms must cause those processes. The issue for the dynamical systems approach to team cognition is that if there is no theory reduction (i.e., no fundamental substance, e.g., explanations in terms of neural pathways) beyond the temporal dynamics, then what are the mechanisms? Put differently, dynamical systems theories in psychology have been criticized for drawing conclusions about psychological mechanisms simply because they carry a particular dynamical signature, such that no fundamental mechanism is posited that produces the signature (Rosenbaum, 1998). We believe, however, that this entails a confusion about what we mean by mechanism. If a mechanism must be something material like a neural pathway, then dynamical systems explanations will always be unsatisfying. However, if we move beyond materialistic modes of explanation toward nonmaterial dynamic forces as mechanisms (Juarrero, 1999), then several of the dynamical principles mentioned in this chapter might foot the bill as dynamical mechanisms of team cognition. For example, dynamical mechanisms include attractor formation and alteration, synchronization, and destabilization and adaptation of thought and behavior (Peng, Havlin, Hausdorff, et al., 1995). Are these mechanisms any less real than that neural pathways and mental representations? We owe it to the dynamical systems theory of team cognition to pursue this question.
Finally, we want to acknowledge that because the dynamical systems theory of team cognition is relatively new, there is limited evidence for its application to enhancing team effectiveness compared to more traditional shared cognition theories. In the past, training methods have been developed based on perturbation/adaptation dynamics, which was demonstrated to be as or more effective than traditional methods for training adaptive teams (e.g., Gorman et al., 2010b). Initial work of real-time perturbation detection through team communication analysis also appeared promising (e.g., Gorman et ah, 2012b). However, future work should attempt to extend the practical implications of the dynamical systems approach to team cognition. As mentioned previously, we are developing real-time analysis and feedback for guiding instructor feedback to trainees following medical simulation training (e.g., Grimm et ah, 2017). Ideally, this type of training tool would allow the instructor to guide trainees by incorporating feedback based on emergent team dynamics under crisis conditions. In the future, we anticipate that incorporating systems thinking into team training will benefit learners by instructing them on how team dynamics emerge and the conditions under which those dynamics can be controlled. As we have noted previously (Gorman et ah, 2017), we picture this as training metacognitive processes for team dynamics or, perhaps, systems thinking from the perspective of an element within the system.
Although dynamical systems approaches to human behavior have proliferated in fields such as ecological psychology, developmental psychology, information systems, and others, team cognition researchers have been slow to adopt this approach despite the need for understanding teamwork in complex systems. The dynamical systems theory to team cognition shifts the focus to a systems-level of analysis. This means that dynamics researchers are changing their measurement from exclusively focusing on the thoughts and behaviors of individual team members to also including the coordination that occurs between team members over time within the constraints of their goal and task environment. Accordingly, the locus of explanation in this type of research has shifted from a material level to a more intangible level, perhaps making the theory and methods in this approach seem less approachable to novices.
The current state of dynamical systems research in teams has shifted from descriptive methods, describing how different concepts and methods from nonlinear dynamics also apply to human behavior, to explanatory methods that identify the sources of team effectiveness. The goal is now to develop software that can identify these sources (e.g., through machine learning algorithms trained on related training sets of data), analyze them in real time, and produce feedback on performance in a way that is accessible to a novice to dynamical systems theory. Although many research questions remain, this approach has successfully developed new conceptualizations of team cognition and performance and new methods of assessing team coordination. The future of this approach lies in its ability to assess and predict successful team performance in real time across a wide variety of domains and data sources.
Abarbanel, H. (1996). Analysis of observed chaotic data. New York, NY: Springer
Abraham, R., & Shaw, C. D. (1992). Dynamics: The geometry of behavior. Boston, MA: Addison-Wesley.
Aebersold, M. & Tschannen, D. (2013). Simulation in nursing practice: The impact on patient care. The Online Journal of Issues in Nursing, 18, Manuscript 6.
Ashby, W. (1947). Principles of the self-organizing dynamic system. Journal of General Psychology, 37, 125-128.
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis. New York: Cambridge University Press.
Bunge, M. (1979). Ontology II: A world of systems. Netherlands: Springer.
Campbell, M., Egerstedt, M„ How, J. P, & Murray, R. M. (2010). Autonomous driving in urban environments: approaches, lessons and challenges. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 36.5(1928), 4649-4672.
Chen, J. Y., Durlach, P. J., Sloan, J. A., & Bowens, L. D. (2008). Human-robot interaction in the context of simulated route reconnaissance missions. Military Psychology, 20(3), 135-149.
Coey, C. A., Kallen, R. W., Chemero, A., & Richardson, M. J. (2018). Exploring complexity matching and asynchrony dynamics in synchronized and syncopated task performances. Human Movement Science, 62, 81-104.
Cooke, N. J., Gorman. J. C„ Duran, J. L., & Taylor, A. R. (2007). Team cognition in experienced command-and-control teams. Journal of Experimental Psychology: Applied, 13(3), 146.
Cooke, N. J., Gorman, J. C., Myers, C. W., & Duran, J. L. (2013). Interactive team cognition. Cognitive Science, 37(2), 255-285.Cover, T. M.. & Thomas, J. A. (2006). Elements of information theory (2nd ed.). Hoboken, NJ: John Wiley.
Department of the Army. (2012). Mission Command (ADP 6-0). Washington, DC: Headquarters.
Demir, M., Cooke, N. J., & Amazeen, P. G. (2018). A conceptual model of team dynamical behaviors and performance in human-autonomy teaming. Cognitive Systems Research, 52,497-507.
Demir, M., McNeese, N. J., & Cooke, N. J. (2018). The impact of perceived autonomous agents on dynamic team behaviors. IEEE Transactions on Emerging Topics in Computational Intelligence, 2(4), 258-267.
Dunbar, T. A. & Gorman, J. C. (2014). Fractal effects of task constraints in the self-organization of team communication. Talk presented to the Human Factors and Ergonomics Society 58th Annual Meeting.
Endsley, M. R. (2015). Autonomous Horizons: System Autonomy in the Air Force - A Path to the Future (Autonomous Horizons No. AF/STTR 15-01). Washington DC: Department of the Air Force Headquarters of the Air Force.
Entin, E. E., & Serfaty, D. (1999). Adaptive team coordination. Human Factors, 41(2), 312-325.
Fiore, S. M„ Smith-Jentsch, K. A.. Salas, E., Warner, N.. & Letsky. M. (2010). Towards an understanding of macrocognition in teams: developing and defining complex collaborative processes and products. Theoretical Issues in Ergonomics Science, 11(4), 250-271.
Gibson, J. (1979). The ecological approach to visual perception. New York, NY: Psychology Press.
Gorman, J. C. (2014). Team coordination and dynamics: Two central issues. Current Directions in Psychological Science, 23, 355-360.
Gorman, J. C., Amazeen, P. G., and Cooke, N. J. (2010a). Team coordination dynamics. Nonlinear Dynamics, Psychology, and Life Sciences, 14(3), 265-289.
Gorman, J. C., Cooke, N. J., & Amazeen, P. G. (2010b). Training adaptive teams. Human Factors, 52(2), 295-307.
Gorman, J. C., Cooke, N. J., Amazeen, P. G., & Fouse, S. (2012a). Measuring patterns in team interaction sequences using a discrete recurrence approach. Human Factors, 54, 503-517.
Gorman, J. C., Cooke, N. J., Pedersen, H. K., Winner, J., Andrews, D., & Amazeen, P. G. (2006a). Changes in team composition after a break: Building adaptive command-and- control teams. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 50(3), 487-491.
Gorman, J. C., Cooke, N. J., & Winner, J. L. (2006b). Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49(12-13), 1312-1325.
Gorman, J. C., & Crites, M. J. (2015). Learning to tie well with others: Bimanual versus intermanual performance of a highly practised skill. Ergonomics, 58(5), 680-697.
Gorman, J. C., Demir, M., Cooke, N. J., & Grimm, D. A. (2019). Evaluating sociotechni- cal dynamics in a simulated remotely-piloted aircraft system: A layered dynamics approach. Ergonomics, 62(5), 629-643.
Gorman, J. C., Dunbar, T. A., Grimm, D., & Gipson, C. L. (2017). Understanding and modeling teams as dynamical systems. Frontiers in Psychology, https://doi.org/10.3389/ fpsyg.2017.01053
Gorman, J. C., Grimm, D. A., Stevens, R. H., Galloway, T., Willemsen-Dunlap, A. M., & Halpin, D. J. (2019). Measuring real-time team cognition during team training. Human Factors, https://doi.org/10.1177/0018720819852791.Gorman, J. C., Hessler, E. E., Amazeen, P. G., Cooke, N. J., & Shope, S. M. (2012b). Dynamical analysis in real time: detecting perturbations to team communication. Ergonomics, 55(8), 825-839.
Grimm, D., Demir, M., Gorman, J. C., & Cooke, N. J. (2018). Systems level evaluation of resilience in human-autonomy teaming under degraded conditions. IEEE Resilience Week Conference, 124-130.
Grimm, D. A., Gorman, J. C., Stevens, R. H., Galloway, T. L., Willemsen-Dunlap, A. M., & Halpin, D. J. (2017). Demonstration of a method for real-time detection of anomalies in team communication. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 6/(1), 282-286.
Haken, H. (1983). Synergetics, an introduction: Non-equilibrium phase transitions and selforganization in physics, chemistry, and biology (3d ed.). Berlin: Springer.
Hayden, Jennifer K.; Smiley, Richard A.; Alexander, Maryann; Kardong-Edgren, Suzan; and Jeffries, Pamela R. (2014). The NCSBN national simulation study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. Journal of Nursing Regulation, 5(2), C1-S64.
Heiniger,C. (2016). Army designing next-gen command posts. Retrieved from https://www. army.mil/article/167807/army_designing_next_gen_command_posts.
Hutchins, S. G., & Kendall, T. (2010). The role of cognition in team collaboration during complex problem solving. In K. L. Mosier & U. M. Fischer (Eds.), Informed by knowledge: Expert performance in complex situations (pp. 69-89). New York, NY: Taylor & Francis.
Issartel, J., Marin, L., Gaillot, P, Bardainne, T., & Cadopi, M. (2006). A practical guide to time—frequency analysis in the study of human motor behavior: The contribution of wavelet transform. Journal of Motor Behavior, 38(2), 139-159.
James, W. (1890). The principles of psychology. London: Macmillan.
Juarrero, A. (1999). Dynamics in action: Intentional behavior as a complex system. Cambridge, MA: MIT Press.
Kantz, H., & Schreiber, T. (1997). Determinism and predictability. Nonlinear time series analysis (pp. 42-57). Cambridge, United Kingdom: Cambridge University Press.
Kruijff, G. J. M., Janicek, M., Keshavdas, S., Larochelle, B., Zender, H., Smets, N. J.,... & Sulk, M. (2014). Experience in system design for human-robot teaming in urban search and rescue. In K. Yoshida & S. Tadokoro (Eds.) Field and Service Robotics (pp. 111- 125). Berlin Heidelberg: Springer.
Kapur, M. (2011). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. Computer-Supported Collaborative Learning, 6, 39-56.
Kruijff, G. J. M., Janicek, M., Keshavdas, S., Larochelle, B., Zender, H., Smets, N. J.,... & Liu, M. (2014). Experience in system design for human-robot teaming in urban search and rescue. In K. Yoshida & S. Tadokoro (Eds.) Field and Service Robotics (pp. 111- 125). Berlin, Heidelberg: Springer.
Leonard, H. B., & Howitt, A. M. (2006). Katrina as prelude: Preparing for and responding to Katrina-class disturbances in the United States—Testimony to U.S. Senate Committee, March 8, 2006. Journal of Homeland Security and Emergency Management, 3, 1-20.
Lorenz, E. N. (1963). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20(2), 130-141.
Lugano, G. (2017). Virtual assistants and self-driving cars. In 2017 15th International Conference on ITS Telecommunications (ITST) (pp. 1-5). Warsaw, Poland: IEEE, doi: 10.1109/ITST.2017.7972192
Mathieu, J. E„ Tannebaum, S. I., Donsbach, J. S., & Alliger, G. M. (2014). A review and integration of team composition models: Moving toward a dynamic and temporal framework. Journal of Management, 40(1), 130-160.
McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010). A critical review' of simulation - based medical education research: 2003-2009. Medical Education, 44, 50-63.
McNeese, N. J., Demir, M., Cooke, N. J., & Myers, C. (2018). Teaming with a synthetic teammate: Insights into human-autonomy teaming. Human Factors, 60(2), 262-273.
Myers, C. W.. Ball, J. T.. Cooke. N. J., Freiman. M. D.. Caisse, M., Rodgers, S. M., Demir, M., & McNeese, N. J. (2017). Autonomous intelligent agents for team training: Making the case for synthetic teammates. IEEE Transactions on Intelligent Systems.
Pappada, S. M., Papadimos, T. J., Lipps, J. A., Feeney, J. J., Durkee, К. T., Galster, S. M.. Winfield. S. R., Pfeil, S. A.. Bhandary. S. P, Castellon-Larios, K.. Stoicea, N.. & Moffatt-Bruce, S. D. (2016). Establishing an instrumented training environment for simulation-based training of health care providers: An initial proof of concept. International Journal of Academic Medicine, 2, 32-40.
Peng, С. K., Havlin, S.. Hausdorff. J. M., Mietus, J. E.. Stanley, H. E.. & Goldberger. A. L. (1995). Fractal mechanisms and heart rate dynamics: long-range correlations and their breakdown with disease. Journal of Electrocardiology, 28, 59-65.
Ricca, В. P, Bow'ers, N., & Jordan, M. E. (2019). Seeking emergence through temporal analysis of collaborative-group discourse: A complex-systems approach. The Journal of Experimental Education, 1-17. https://doi.org/10.1080/00220973.2019.1628691
Riley, W., Davis. S., Miller. K., Hansen. H., Sainfort, F.. & Sweet, R. (2011). Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Joint Commission Journal on Quality and Patient Safety / Joint Commission Resources, 31, 357-364.
Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. O’Malley (Ed.) Computer Supported Collaborative Learning (pp. 69-97). Berlin, Heidelberg: Springer.
Rosenbaum, D. A. (1998). Is dynamical systems modeling just curve fitting? Motor Control, 2, 101-104.
Salas. E.. Priest, H. A.. Wilson, K. A.. & Burke, C. S. (2006). Scenario-based training: Improving military mission performance and adaptability. In A. B. Adler, C. A. Castro, & T. W. Britt (Eds.), Operational Stress. Military life: The psychology of serving in peace and combat: Operational stress (pp. 32-53). Westport, CT: Praeger Security International.
Schneider, W. (1985). Training high-performance skills: Fallacies and guidelines. Human Factors, 27(3), 285-300.
Schulte, A., Donath, D„ & Lange, D. S. (2016). Design patterns for human-cognitive agent teaming. In D. Harris (Ed.), Engineering Psychology and Cognitive Ergonomics (pp. 231-243). Cham, Switzerland: Springer International Publishing.
Seeber, I., Maier, R., & Weber, B. (2013). Macrocognition in collaboration: Analyzing processes of team knowledge building with CoPrA. Croup Decision and Negotiation, 22(5), 915-942.
Shannon, C., & Weaver, W. (1949). The mathematical theory of communication. Urbana: University of Illinois Press.
Shively, R. J.. Lachter, J.. Brandt, S. L„ Matessa, M., Battiste, V., & Johnson, W. W. (2017). Why human-autonomy teaming? In International Conference on Applied Human Factors and Ergonomics (pp. 3-11). Los Angeles, CA: Springer.
Stanton, N. A., Stewart, R., Harris, D., Houghton, R. J., Baber, C., McMaster, R., ... & Linsell, M. (2006). Distributed situation awareness in dynamic systems: theoretical development and application of an ergonomics methodology. Ergonomics, 49(12-13), 1288-1311.
Stevens, R. H. & Galloway, T. (2017). Are neurodynamic organizations a fundamental property of teamwork? Frontiers in Psychology, 8, 644.
Stevens, R. H., Galloway, T., Berka, C., & Sprang, M. (2009). Neurophysiologic collaboration patterns during team problem solving. Proceedings of the Human Factors and Ergonomics Society, 53(12). 804-808.
Stevens, R., H.. Galloway, T.. Gorman, J.. Willemsen-Dunlap, A., & Halpin, D. (2016a). Toward objective measures of team dynamics during healthcare simulation training. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, 5, 50-54.
Stevens, R., Galloway, T., Halpin, D., & Willemsen-Dunlap, A. (2016b). Healthcare teams neurodynamically reorganize when resolving uncertainty. Entropy, /
Stevens, R. H., Galloway, T., Willemsen-Dunlap, A., Gorman, J. C., Halpin, D., & Grimm, D. A. (2018). Making sense of team information. Proceedings of the Human Factors and Ergonomics Society, 62(1), 114-118.
Stevens, R. H„ Gorman. J. C., Amazeen. P. G.. Likens, A., Galloway, T. (2013). The organizational neurodynamics of teams. Nonlinear Dynamics, Psychology, & Life Sciences, 17. 67-86.
Sutcliffe, К. M., Lewton, E., & Rosenthal, M. E. (2004). Communication failures: An insidious contributor to medical mishaps. Academic Medicine, 79, 186-194.
Thelen, E., & Smith, L. B. (1994). A dynamic systems approach to the development of cognition and action. Cambridge, MA: MIT Press.
Thelen, E.. Ulrich. B. D., & Wolff, P. H. (1991). Hidden skills: A dynamic systems analysis of treadmill stepping during the first year. Monographs of the Society for Research in Child Development, 56(1), 1-103.
Treffner, P. J., & Kelso, J. A. S. (1999). Dynamic encounters: Long memory during functional stabilization. Ecological Psychology, II, 103-137.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, UK: Harvard University.
Walker, G. H.. Gibson. H.. Stanton, N. A., Baber, C., Salmon, P., & Green, D. (2006). Event analysis of systemic teamwork (EAST): a novel integration of ergonomics methods to analyse C4i activity. Ergonomics, 49(12-13). 1345-1369.
Webber, C. L. & Marwan, N. (Eds.) (2015). Recurrence quantification analysis: Theory and best practices. New York: Springer.
Willemsen-Dunlap, A., Halpin, D. Stevens. R.H., Galloway. T.L., Gorman, J. (2017) Identifying neural and speech correlates of uncertainty during healthcare simulation training. Talk presented at the March, 2017 HFES-Health Care International meeting, New' Orleans, LA.
Wiltshire, T. J„ Butner, J. E., & Fiore, S. M. (2018). Problem - solving phase transitions during team collaboration. Cognitive Science, 42(1), 129-167.
Wiltshire, T. J., Steffensen, S. V., & Fiore, S. M. (2019). Multiscale movement coordination dynamics in collaborative team problem solving. Applied Ergonomics, 79, 143-151.