Studying Team Cognition in the C3Fire Microworld

Bjorn J. E. Johansson, Rego Cranlund, and Peter Berggren

CONTENTS

Introduction..............................................................................................................29

Microworlds.............................................................................................................30

Team Research in Microworlds...........................................................................32

C3Fire......................................................................................................................32

Studying Situation Awareness and Cooperation in Teams..................................36

C3Fire as a Tool for Studying Command and Control Teams............................36

Studies of the Impact of GIS on Emergency Response Management.................38

Using C3Fire to Support Role-Playing Exercises in Emergency

Response Research.........................................................................................42

Developing the Shared Priorities Instrument of Shared Strategic

Understanding................................................................................................43

Conclusion...............................................................................................................45

References................................................................................................................46

INTRODUCTION

The aim of this chapter is to provide a description of how the C3Fire microworld has contributed to knowledge development concerning various aspects of team cognition and how the platform has been and can be utilized for research on team cognition. C3Fire has been used for more than two decades by several research institutes and universities for a wide range of studies, resulting in more than a hundred different publications. Some of the main research tracks that have used C3Fire as a research tool include distributed decision making, command and control, the effects of new technologies, how cultural differences manifest in team decision making, and the development of a new measure of shared understanding in teams. In this chapter, we focus on C3Fire studies concerning aspects of team cognition, such as how organizational arrangements affect communication and performance, how new technologies affect the outcome of teamwork, and how the microworld has been used to support the development of a new measure of sharedness in teams. This endeavor will be undertaken in a chronological fashion with some selected work performed by some of the main Swedish C3Fire user groups, beginning with early work at Linkoping University concerning different organizational structures and their impact on team decision making, followed by a presentation of some of the investigations of how geographical information systems can support decision making, then proceeding towards the most recent publications on method development conducted at the Swedish Defence Research Agency.

The chapter is organized as follows: Firstly, we will provide a background to the microworld research tradition related to team cognition. Then, we will provide a summary of the previously mentioned research tracks that have been investigated with the C3Fire microworld. After that, we continue by discussing pros and cons of the microworld approach in relation to team cognition research. Lastly, we present our conclusions and recommendations based on the from the conducted studies.

MICROWORLDS

Microworlds have been used in research on decision making since the late 1970s (Brehmer, 2004; Dorner, 1980; Dorner, Kreuzig, Reiter, & Staudel, 1983; Funke, 1993, 2001), although the etymology of the term remains hidden in history. Other terms that have been used are scaled worlds (Gray, 2002; Schiflett et al., 2004), synthetic environments (Cooke & Shope, 2004), and simulations (McNeese et ah, 2005). In this chapter, we refer to computer-based simulations that share some properties that are common to all these notions when using the term microworld. Gonzalez, Vanyukov, and Martin (2005) provide a detailed list of the most commonly used microworlds, which is informative for anyone seeking a broad perspective on the types of problems and simulations that have been developed. Microworlds were originally developed to allow researchers to investigate complex aspects of decision making (Funke, 2010). The German tradition of /complexes problemlosung (complex problem solving; Dorner & Schaub, 1992), initiated by Dietrich Dorner of the Bamberg University, used various simulations to investigate how individuals cope with highly complex decision tasks in an explorative fashion. In Sweden, Berndt Brehmer and colleagues pursued the task of understanding how people cope with the control of dynamic systems (Brehmer, 1987; Brehmer, 2004, 2005), using simulated environments in traditional experiments. Brehmer & Dorner (1993) jointly suggested that microworlds bridge the gap between (psychological) laboratory studies and the “deep blue sea” of field research. A microworld is implemented in a computer and can be seen as a computer-based simulation. This is partly true if we by simulation mean any computer program that has some similarity with a real-world task. That would however be a misuse of the term “simulation,” since a simulation often is interpreted to be a model-based implementation of a more or less exact representation of a real-world task (Griine-Yanoff & Weirich, 2010). For example, a flight simulator for professional training may be very advanced, providing an almost entirely realistic interaction (Liu, Macchiarella, & Vincenzi, 2009). This is not the purpose of a microworld.

In experiments with microworlds, subjects are required to interact with and control computer simulations of systems such as forest fires, companies, or developing countries for some period of time. Microworlds are not designed to be high fidelity simulations. Instead, they are related to the systems that they represent in the same manner as wood cuts are related to what they represent. That is, it is possible to recognise what is being represented, but there is little detail. However, microworlds always have the fundamental characteristics of decision problems of interest, here, viz., complexity and

in-transparency.

(Brehmer, 2000, pp. 7-8)

The purpose of a microworld is thus to present a recognizable problem to the subjects taking part in a study. However, the microworld must still be complex enough so that the subjects experience a dynamic situation presenting a certain degree of uncertainty. A recognizable task often used is forest fire fighting (Svenmarck & Brehmer, 1991; Granlund, 2002; Gray, 2002). When implementing something like a forest fire in a microworld, the important point is not to preserve detailed real-world characteristics, but rather to have a level of fidelity that is high enough to be acceptable by the participants in the study but yet low enough to be easily manageable and analyzable by the researcher(s). It should thus trigger essential cognitive processes that are representative of the processes taking place in a real-world task. According to Brehmer and Dorner (1993) microworlds are characterized by the fact that they are complex, dynamic, and opaque, suggesting that they will trigger cognitive processes taking place when coping with such challenges. They are complex as the subjects have to consider a number of aspects, like different courses of actions or contradicting goals. Secondly, they are dynamic in the sense that subjects, at least in the forest firefighting example, have to consider different time scales and sudden, most likely, unforeseen, effects since the relationships between different variables are difficult to understand. The opaqueness comes from the fact that parts of the simulation are invisible to the subject, who has a “black box” view of it. Participants in the microworld thus have to make hypotheses and test them in order to handle the situation (Brehmer & Dorner, 1993). These three characteristics are representative to many real-world situations, and definitely to many situations in which teams work, such as crisis management or military operations.

In this chapter we define computer-based simulations intended to present research participants with a recognizable problem, triggering cognitive processes, that reflect core aspects of a real-world phenomena while allowing for experimental control at a level of resolution possible to interpret and analyze from a scientific point of view. Thus, microworlds are research tools aligning themselves between “analogue” experimentations and high-fidelity simulation or field studies.

Therefore, it could be assumed that microworlds are suitable not only for investigating how individuals behave when they are confronted with a dynamic problem, but also for how teams handle such tasks. Brehmer and Svenmarck (1995) developed such a microworld, based on the forest firefighting task, called D3Fire. It was designed to allow for studies on “distributed decision making,” which presents problems that cannot be managed by individual decision makers, but rather must be handled by a team. D3Fire was mainly used to study how the problem of extinguishing the forest fire was affected by different configurations of the communication paths between participants, such as hierarchical structures versus networked (Brehmer & Svenmarck, 1995; Svenmarck & Brehmer, 1994). The D3Fire simulation was later used as inspiration for the C3Fire microworld developed by Rego Granlund during the late 1990s (Granlund, 2002), although the latter was designed to be platform independent and highly configurable, properties that the D3Fire microworld lacked. During the same time period, Australian researchers developed a platform called the Networked Fire Chief (Omodei, Elliott, Walshe, & Wearing, 2005) at La Trobe University in Melbourne, mainly to “investigate the underlying causes of unsafe decisions in context of wildland firefighting” (Omodei et ah, 2005, p 1). The Networked Fire Chief platform held similar characteristics as C3Fire, although with a narrower research focus, and will not be described further in this chapter. Instead, we will carry on by providing a general introduction to team research in microworlds.

Team Research in Microworlds

For the purpose of this chapter, the relation between team research, mainly team cognition, and microworlds will be discussed in this section. Even though teams have been studied in social psychology in terms of groups, the area of team research has since the 1990s called for empirical studies of teams (Swezey & Salas, 1992; Baker & Salas, 1992; Salas & Fiore, 2004; Salas, Fiore, & Letsky, 2012; Tannenbaum, Mathieu, Salas, & Cohen, 2012; Wildman, Salas, & Scott, 2014). Often these calls have asked for validated metrics, a larger empirical foundation, or to move outside of the laboratory. As is pointed out by these authors, there are several difficulties to overcome. The primary problem with team research in the wild is the possibility to collect data, especially when it comes to less frequently occurring events. Therefore, much knowledge is based on case-like descriptions of, for example, crisis response, emergency management, and similar events (Comfort, 2007; Boin & McConnell, 2007; Bodeau, Fedorowicz, Markus, & Brooks, 2009; Ouyang & Wang, 2015; Johansson, Trnka, & Berggren, 2016). In summary, some of the problems associated with team research are number of cases, level of analysis, metrics issues, experimental control, learning effects, etc. As described previously, many of these problems where identified by Brehmer and Dorner (1993), whereas they proposed the microworld approach as a sound way of overcoming these concerns.

A broad definition of team cognition is the cognitive activity that occurs within a team (Cooke, Gorman, & Rowe, 2009). How team members manage information, communicate, coordinate actions, and collaborate are central aspects of team cognition. In addition, how this can be assessed, measured, and modeled are other concerns. However, many of these aspects can be studied in microworld settings. The challenge is to find a suitable microworld platform that allows for such studies. Next, we describe the C3Fire platform, which is a microworld based on the forest firefighting task that was developed specifically to study team decision making.

C3FIRE

The C3Fire microworld was developed over several years, and it is therefore difficult to provide a just description of all functionality of the system. This section will hence only give an overview of the general properties and functions of the system. The purpose of C3Fire is, as in most microworlds (Brehmer & Dorner 1993), that the participants should experience a task environment that has some of the important complex dynamic behavior of a real system, but without the nitty-gritties of the actual task. Decisions are made in an environment where the researcher can manipulate dynamics, complexity, and the degree of opaqueness. Both the forest fire and the firefighting organization can be configured to exhibit varying degrees of complexity and dynamics. The forest fire will change both autonomously and as a consequence of actions taken by the research participants (Granlund, 1997, 2001, 2002, 2003). The decision making can be configured to be distributed over a number of persons and can be viewed as team decision making where the members have different roles, tasks, and items of information available for their decision process. The organization can be designed to mimic a hierarchy where the decision makers work on different time scales. The firefighting unit chiefs (often referred to as “ground chiefs”) are responsible for the low-level operation, such as the fire fighting, which is done in a short time frame. The staff works at a higher time level and are responsible for the co-ordination of the firefighting units and the strategic thinking (Granlund, 2002, 2003).

Computer-based monitoring is used to collect data about participants actions and communication in the C3Fire system. The monitoring is integrated in the simulation as well as all the information tools used by the participants, such as text messages and shared geographical information systems (Granlund, 2003; Johansson, Persson, Granlund, & Mattsson, 2003). During a session, the C3Fire system creates a data log. The log process receives data from the simulation about the current activities in the forest-fire simulation. All events added to the data log file are time-stamped and can be easily imported into Microsoft Excel and most of the common software for statistical analysis. The outline of the simulation and its components can be found in Figure 2.1.

Basic components of the C3Fire simulation system. Source

FIGURE 2.1 Basic components of the C3Fire simulation system. Source: Johansson. Trnka, & Granlund, 2007.

The participants interact with the simulation through an interface that can be configured in different ways. Typically, it presents a map view of the environment, in which the participants can control the units they are responsible for. Mostly, it also includes a messaging system that allows for communication with other participants. The communication patterns that are possible are defined in a configuration file. The type of information that is available to the participants, for example how large parts of the simulated area are visible from each unit, or at all, is also defined in advance. An example of the C3Fire interface can be found in Figure 2.2.

Organizational aspects of the participating team can be configured in different ways, partly by simply assigning participants to certain roles, but also by dictating communication paths and providing exclusive access to specific resources to certain roles. For example, a simple hierarchy can be constructed by assigning some participants as commanders, while other participants are assigned as ground chiefs, responsible for controlling the firefighting units. If the communication paths mimic these roles, this is enforced, as ground chiefs only can communicate “vertically” with their commanders, and not with their fellow ground chiefs, assuring that their activities are coordinated by the commanders in the simulation (see Figure 2.3).

The C3Fire interface as shown to a participant responsible for controlling three firefighting units. The messaging system can be seen in the lower left part of the figure

FIGURE 2.2 The C3Fire interface as shown to a participant responsible for controlling three firefighting units. The messaging system can be seen in the lower left part of the figure.

An example of a hierarchical arrangement of a C3Fire experiment

FIGURE 2.3 An example of a hierarchical arrangement of a C3Fire experiment.

As C3Fire can be configured to represent a number of different roles and organizational structures, it allows for the investigation of a wide range of team-related research. Participant performance in the C3Fire microworld can easily be extracted from the log files (Granlund & Johansson, 2003). For example, performance can be calculated in terms of the number of cells affected by fire, number of structures affected, or whether a cell has been fully burnt out or if the participants have stopped the fire in time. As these measures are recorded dynamically (not truly dynamically, but with a high enough sampling rate to capture any changes) the participants’ progress can easily be tracked and analyzed. This performance can be connected to individual participants, teams, or the whole system and related to, for example, communication messages sent. The generated log files also allows for playback of the scenario development, to use for training and feedback to the participants. The temporal aspects of simulation can be used as a performance measure, for example, how long did it take for team member X to respond to a message or how long did it take until all fires were controlled? Time can also be set to frame the experimental design, for example, how much fire was controlled before the scenario was stopped after 20 minutes? Our experience with scenario length is that most scenarios are run for no more than 45 minutes (even though a scenario could continue for many hours). Obviously, this is dependent on the research questions asked. The speed by which different types of terrain is consumed by the fires can also be manipulated, as well as the speed of fire fighting and the movement speed of vehicles.

Next, we will describe the most important research efforts that have been conducted using the C3Fire simulation.

Artman (1999) performed the initial studies using C3Fire with the aim of investigating how teamwork was impacted by different organizational configurations (serial, parallel, and an “optional” alternative). This was especially evaluated in terms of situational awareness (Endsley, 1995) and co-operation (by communication analysis). This work postulated a relationship between communication within a team and the degree of situational awareness that the team could reach. The specific formulation was that “communication must of course be relevant in order to support team situation awareness. This study will investigate what kind of communication is relevant” (Artman, 1999, p. 1406). It was found that most groups in the optional condition performed worse than the groups in the other conditions, indicating that the optional organization, where participants were allowed to choose how to organize themselves, has a negative impact on performance as that approach created uncertainties regarding information flows and responsibilities, eventually also hampering situational awareness. It was also found that members of teams in the parallel condition differed from members of teams in the serial condition in their situation awareness. This was attributed to the fact that the responsibility for keeping track of unit positions and fire development was distributed between different persons in the parallel organization, while it was handled by a single individual in the serial configuration. This distribution may have led to a situation were no members of the team really were able to create a picture of the situation as a whole. Further, more successful commander teams produced more planning in relation to hypothesis, as well as sending fewer messages in total between the units, than the less successful team (Artman, 1999). The Artman studies showed how C3Fire can be used to investigate several aspects of team decision making as well as how technology can affect team decision making. This inspired a set of further studies at both Linkoping University and the Swedish National Defence College (SNDC).

C3Fire as a Tool for Studying Command and Control Teams

The aftermath of the United States victory in the 1991 Gulf War against Iraq created a case for the importance of information and communication technology (ICT) on the battlefield, as it seemed to provide a way to create unsurpassed situational awareness as well as superior coordination of resources (Carlerby & Johansson, 2017). This “information superiority” promised a “revolution in military affairs” (RMA), which created a huge interest from military actors, industrial suppliers, as well as researchers and command and control concept developers. This development eventually led to the introduction of concepts such as network-centric warfare (NCW; Cebrowski & Garstka, 1998; Osinga, 2010). Most military research institutes initiated studies concerning how such concepts could be utilized, including the SNDC. Alberts, Garstka, and Stein (1999) defined NCW in the following way:

We define NCW as an information superiority-enabled concept of operations that generates increased combat power by networking sensors, decision makers, and shooters to achieve shared awareness, increased speed of command, higher tempo of operations, greater lethality, increased survivability, and a degree of self-synchronisation.

(Alberts, Garstka, & Stein, 1999, p. 2)

A series of experiments were launched at the SNDC w'ith the aim of testing some of these suggestions, in particular the promises of increased shared awareness and increased performance. This was done in the context of the development of a new concept for command and control environments, the Joint Mobile Command and Control Post of the Future (Sundin & Friman, 1998, 2000). The purpose of the Joint Mobile Command and Control Post project w'as to create a command environment (a staff room) w here teams of decision makers could work jointly around a large table-like screen, promoting a shared view as well as an environment that enabled discussion and creativity to a larger extent than a traditional staff environment, thus increasing the quality of decisions made (Sundin & Friman, 1998). A prototypical environment was built at the SNDC, where such a table-like screen, and additional computers and screens, were available for experimentation. It was soon concluded that such conceptual propositions would be difficult to study through full-scale exercises or high-fidelity simulation as it would require extensive implementation of technologies that were not yet available. Instead, this was approached by utilizing the C3Fire microworld as the basic research platform. Several studies were conducted, where the effects of rapid information flows through geographical information systems could be utilized to increase shared awareness. For example, Parush and Ma (2012) showed how a team display supported team performance during communication breakdowns in a command and control task using the C3Fire environment. Another example is how' the effect of updating maps directly from sensors in the field contrasted with the traditional approach of having a staff member process and deliver information to decision makers (Johansson, Granlund, & Waern, 2000; Granlund et al., 2001; Persson & Johansson, 2001; Johansson, Persson, Granlund, & Mattsson, 2003). These studies showed that although the errors in information clearly were reduced in when maps were updated directly from the field rather than by a staff member, overall team performance did not improve. This was partly attributed, as in the case of the Artman (1999) study, that the work division in the staff team was less clear in the condition where information was presented directly in the shared map than when a specific staff member added data to the shared map. Video analysis suggested that the military commanders in the manual updating condition recognized the similarities in working w'ith traditional paper-based maps. In the direct update, the command team rather gathered around the shared map, waiting for something to happen. A hypothesis is that this hampered performance in the direct-updating condition since the tool eliminated the need to constantly monitor the emails coming from the ground chiefs, but it also forced them to look at the shared map (Johansson, 2005). In manual update the commander and the assistant completed the picture based on verbal reports from the two communications officers. These verbal reports were spoken out in the room (by the communication officers), meaning that anyone in the room also could hear w'hat w'as going on. This can possibly have minimized the need to observe the shared screen for assessing the situation. There is also a risk that the direct updating condition, with its higher rate of data input on the screen, creates a situation where the commanders are “chasing” the situation rather than handling it.

These studies initiated further research concerning the effects of new technologies on team decision making, as will be described next.

Studies of the Impact of GIS on Emergency Response Management

These are studies conducted at Linkoping University funded by KBM (Swedish acronym for Krisberedskapsmyndigheten, the crisis preparedness authority which later merged into the Swedish Civil Contingencies Agency; MSB) during the period of 2005 to 2011.

During the time from 2005 to 2011, three research projects were performed at Linkoping University with the goal to investigate the impact of a decision support system that presents real-time information through a global positioning system (GPS) to decision makers in crisis management organizations. The goal was to compare the performance between teams that had access to unit positions and sensor information in the command post with teams that received information through text messages and had to keep track of situation development using paper maps. The method used was controlled experiments with the C3Fire microworld. All three projects had the same experimental approach, although three different types of participants took part in the experiments: university students, municipal crisis management organizations, and rescue service personnel.

A total of 304 participants took part in the three studies. In the first study, conducted in 2006, the research was tested on non-professional participants, 132 students formed into 22 groups (Johansson et al., 2007, 2010). In the second study, conducted from 2008 to 2009, the participants were professionals belonging to different municipal crisis management organizations, including both rescue service personnel and other municipal employees. In this study, 108 professionals formed 18 teams (Granlund et ah, 2010; Granlund & Granlund 2011a, 2011b). In the third study, conducted from 2010 to 2011, a total of 64 professionals formed 8 rescue service teams.

The organizations of interest were Swedish municipal crisis management organizations and their crisis management teams. The goal was to understand differences in the collaboration and work processes of teams that had access to unit and sensor information in their digital map systems at command center and command post level, compared to teams that had only paper maps in the command posts.

The participating professionals belonged to different municipalities in Sweden. Currently, many of these have made, or are about to make, investments in information and communication technologies that give the decision makers in the command centers and command posts access to GPS information. All this is done in order to enhance performance and control of work.

Digital maps are seen as support tools for crisis management. Management is understood to be more efficient with the introduction of these new technical supports. The reality is that the tools distribute large amounts of information automatically to the decision makers. All users at all levels of management have in many situations access to the same information simultaneously. What was originally seen as an aid in the management work can have unsuspected consequences. The tools change the requirements for leading and organizing emergency efforts.

The experiment designs of Study 1 and Study 2 were identical. For the third study the original design was amended with to the aim of increasing complexity and respond to rescue service professionals training needs. Study 1 and Study 2 had the same between-group design with one factor: (a) crisis management teams receiving unit positions and sensor information through a digital map, and (b) crisis management teams receiving information in the form of text messages, using paper maps (Figures 2.4 and 2.5).

Design Study 1 and 2, GPS condition

FIGURE 2.4 Design Study 1 and 2, GPS condition.

Design Study 1 and 2, paper map condition

FIGURE 2.5 Design Study 1 and 2, paper map condition.

In each group, three participants worked as a command center (CC) with one commanding officer and two liaison officers. Three participants worked as ground chiefs (GC). The CC had no direct contact with the simulation and controlled the simulated world indirectly via the GC. The GC directly controlled three units (fire brigades) each in the simulation, a total of nine units. Each unit could “see” a limited part of the world in the immediate surroundings. In the first condition, unit positions and the sensor information received by that unit were transmitted directly to the command post. The difference between the two conditions was the complexity of the type of support the commanding officer obtained, in terms of GPS with access to exact positioning of the resources in the simulated world, or a paper map without any automatic information.

Study 3 had the same between-group design with one factor as Study 1 and Study 2: (a) crisis management teams receiving information about unit positions and sensor information through a digital map, and (b) crisis management teams using paper maps. The differences between the designs were based on six points requested by the participating rescue service personnel.

(1) The organization of the participant group had four levels of command instead of three: command center (CC) at home base, command post (CP) on the field, and four ground chiefs (GC) directing units in the simulation (Figures 2.6 and 2.7). (2) Each group had eight participants. (3) The resources in the simulated world were ten firefighters, five water logistic trucks that supply water to the fire fighters, three excavators for digging firebreaks, three units for evacuating citizens, and one unit for reconnaissance purposes. (4) None of the resources are linked to any of the ground chiefs. A structure for who is using which resources and when must be set up by the

Design Study 3, GPS condition

FIGURE 2.6 Design Study 3, GPS condition.

Design Study 3. paper map condition

FIGURE 2.7 Design Study 3. paper map condition.

team. The complexity of the task of extinguishing the forest fire was increased. Also, the ability to communicate was not restricted: everyone could send messages to anyone. This forced the team to form a structure describing how interactions should take place and in what manner. (5) The simulation time was increased from 20 minutes to 40 minutes, but the whole training day has been reduced to one full training session and three full simulation cycles. (6) The fire spread at a slower pace and the firefighting units moved slower.

For the first study, where students participated, results showed no significant difference between teams receiving information directly in the digital map and teams working with paper maps (Johansson et al., 2010). The results showed that groups with an access to real-time support had a better performance than those who had to keep track of the situation using paper maps, in terms of the area lost to the forest fire. This suggested that it was easier to understand the situation and coordinate resources when receiving rapid feedback about unit positions and sensor information from the immediate surroundings of the units. This was also reflected in the communication between the participants, which revealed that a larger number of messages were sent in the condition without direct access to unit positions and sensor information. Specifically, the amount of questions differed between the two conditions, where the teams (paper map condition, see Figure 2.5) having to keep track of unit positions and fire development on paper maps sent a significantly larger amount of questions about unit positions and the development of the forest fire(s) (Johansson et al., 2010).

For the second study, where municipality crisis managers participated, there was no overall performance difference between automated information transfer and manual information management using paper maps (Granlund, Granlund, & Dahlback,

2011). The municipality crisis managers had an inconsistent result compared to participants in the first study. Observations suggested that the crisis managers recognized the simulations as training in crisis management or in communication, not as a computer game, which the student participants in the first study may have done. Further, the observations indicated that the professional participants perceived the simulation as more complex than the students, although the simulation setting itself was not changed. The professionals, as well as the students, knew they were observed and analyzed in a research project. They behaved like professionals when they solved their problems. They used strategies from real life, for example to discuss before they acted, not using all the resources directly, etc. They saw the simulated event's resemblance to a real situation rather than a computer game. Hence, the professional background of the crisis managers caused them to interpret the situation as training, and they tried to do what they normally would have done in a real situation.

In the third study, where professional fire fighters participated (Granlund & Granlund, 2011), the results indicated a surprising difference in the sense that teams where the commanders only received messages and worked with paper maps performed significantly better. It should however be noted that the amount of trials conducted was lower in this condition and that the experimental design was not identical to Studies 1 and 2 (as pointed out previously). A reason for this could be complexity. The automated information flow was directed to both CC and CP without prior classification as important or unimportant. This classification is in actual w'ork conducted by ground chiefs in the field. The teams in the condition where information flows were automatized had to allocate time to handle the information. The teams thus did not have time to adjust their ordinary management methods, basically adapted to paper maps, to the situation with direct information flows. Their amount of burned-out area was in the last simulation trial still increasing. That is a sign of the three simulation trials not being enough for learning to handle the complexity of receiving automated information flows.

The CC and CP in teams in the condition where information had to be handled manually had a situation that much resembled what rescue service commanders experience in reality. The complexity was experienced as lower and better adjusted to their ordinary methods. The commanders did not have to refine or alter their known methods too much, thus they performed better (Granlund & Granlund, 2011).

Using C3Fire to Support Role-Playing Exercises in Emergency Response Research

These studies were conducted at Linkoping University and differed from the previously mentioned studies in the sense that they did not follow the experimental tradition with contrasting conditions and controlled variables. Instead, a role-playing exercise approach was applied, focusing on utilizing components from C3Fire to create an immersive situation where professional participants could interact with each other (Trnka & Johansson, 2009). The C3Fire components were used to support data collection in terms of messaging, information dissemination, etc. (Trnka & Jenvald, 2006). An example of how this approach can be applied is described in Trnka & Jenvald (2006) where dispatchers from the county emergency call center, incident commanders, fire and rescue dispatch officers, police on-site incident commanders, and police dispatch officers participated in a scenario where a major forest fire threatens a local zoo and its visitors. In this simulation, the participants did not interact directly with the C3Fire interface, instead information about the progress of the fire and firefighting units was presented to the participants in the study by exercise facilitators (Trnka & Jenvald, 2006). The exercise facilitators injected information into the exercise either verbally or by text messages to the participants, mimicking the kind of information the emergency responders normally would get from units in the field and from the public. The focus was to understand how participants acted under time pressure and uncertainty with limited resources. Using C3Fire as input for the role-playing exercise included both planned and unplanned variations, thus requiring the participants to handle situations that changed over time in a nonpre- dictable way. Trnka and Jenvald (2006) showed that high realism of the simulation content, low-fidelity ecological settings, co-localization of the participants, and the use of domain, modeling, and simulation experts in planning and execution were necessary for a successful role-playing exercise. Trnka and Johansson (2009) used the same material and applied communication analyses such as episodic analysis (Korolja & Linell, 1996), socio-metric status (Houghton et al., 2006), and communication roles to assess coordination among participants. The results suggested that the participants used informal procedures within the command and control organization to perform various informal functions and roles across organizational and domain boundaries. Utilizing C3Fire in this way created a hybrid simulation where specific processes were represented within the microworld while other aspects of the simulation were conducted as role playing (Trnka & Jenvald, 2006).

Developing the Shared Priorities Instrument of Shared Strategic Understanding

Berggren and colleagues (Berggren, 2016; Berggren, Johansson, & Baroutsi, 2017) used C3Fire to develop a new measure of shared understanding in teams handling various dynamic tasks. Mainly using students as research participants, the instrument was developed through several iterations, where the main platform used was C3Fire. A brief summary of this development is discussed next.

The demand for a comprehensible, easy to use, useful instrument for assessing a team’s shared mental models has been identified over and over again (Cannon- Bowers, Salas, & Converse, 1993; Klimoski & Mohammed, 1994; Smith-Jentsch, Campbell, Milanovich, & Reynolds, 2001; Langan-Fox, Wirth, Code, Langfield- Smith, & Wirth, 2001; Mohammed, Ferzandi, & Hamilton, 2010; Mohammed, Klimoski, & Rentsch, 2000). One example of this is the authors’ own experience of a military brigade HQ meeting where the chief of staff (COS) provided a briefing and gave orders to the staff officers who acknowledged the orders and then left the staff meeting. Outside of the staff tent they were then asked individually what the intent of the COS was and what the shared goals of the brigade were. Responding to this question, the officers provided almost as many different answers as there were individuals. This feedback was then presented to the COS who recognized the need for an instrument that could help him evaluate if the command team had a shared understanding of the shared goals. Most existing instruments were either very difficult to utilize, took a long time to prepare, were only useful for a context- dependent setup, were more or less intrusive, and/or took a long time to analyze (Langan-Fox, Code, & Langfield-Smith, 2000; Mohammed, Klimoski, & Rentsch, 2000: Resick. Murase, Bedwell, Sanz, Jimenez, & DeChurch, 2010; DeChurch and Mesmer-Magnus, 2010; Wildman. Salas, & Scott, 2014).

Hence, the mission was to develop an instrument that was useful, understandable, easy to apply, and cost effective. The first iteration was an experiment in C3Fire (reported in Berggren, Alfredson, Andersson, & Modeer, 2007 and in Berggren, Alfredson, Andersson, & Granlund, 2008). Here, the instrument used lists of pre-defined items relating to different aspects of teamwork in the C3Fire scenario. The pre-defined lists were tested using three-member teams (with different roles) who were required to collaborate in order to perform well in the microworld setting. Four different conditions were tested (ranging from easy to difficult by manipulating the amount of information that was available for the participants). The participants were asked to rank order the lists, which were then compared among team members (within teams) to analyze degree of correspondence. The results revealed no differences. There were several factors identified explaining this: untrained participants, a too-weak relationship between items and behavior, and too difficult tasks.

The following iteration involved military teams working as tank crews (reported in Berggren, Svensson, Horberg, Jonsson, & Hoglund, 2009 and in Berggren & Johansson, 2010). The experimental organization included three tank crews that scouted an area in a peace-enforcing scenario. In this study, two different conditions were used. The first condition was that the teams communicated in the same command and control (C2) system, and the other condition was that one team could only use radio and paper maps. For this study the participants were asked to generate the items in the lists individually, that is, individually defined items. Within the teams they were then asked to rank order these lists. The outcome from these ranked lists was analyzed using Kendall’s measure of concordance (Kendall & Babington Smith, 1939). Here, a difference between the two conditions was present in the outcome of the instrument assessing shared understanding. That is, a difference between the two conditions could be seen regarding degree of shared understanding.

In the next iteration these findings were again tested in the C3Fire microworld (reported in Berggren, Johansson, Svensson, Baroutsi, & Dahlback, 2014 and Berggren et ah, 2017). This time six teams were trained over ten occasions in collaborating in the C3Fire microworld. These teams were then tested against six teams that were со-trained once prior to the experimental data collection. The participants were asked to generate the items in the lists individually, that is, individually defined items. Within the teams they were then asked to rank order these lists. The outcome from these ranked lists was analyzed using Kendall’s measure of concordance (Kendall & Babington Smith, 1939). Here, a difference between the trained and novice teams conditions was present in the outcome of the instrument. In addition, the generated items were analyzed regarding content, and the trained teams’ items were focused on team-level concerns, whereas the novice teams’ items were focused on the individual’s needs.

The shared priorities instrument assesses shared strategic understanding, that is “the ability of multiple agents to exploit common bodies of causal knowledge over time for the purpose of accomplishing common (or shared) goals” (Berggren, 2016, p. 125). The instrument was found to be usable, understandable, objective, flexible, and self-explanatory (Berggren, 2016), thus meeting several criteria that were asked for by the practitioners, i.e., military commanders. After the development phase of the instrument, it has been tested in several other domains: with nuclear power plant control room crews (Berggren, Johansson, & Ekstrom, 2016), emergency rooms (Berggren, Johansson, Allard, & Torensjo, 2016), at an emergency exercise (Prytz, Rybing, Jonson, Pettersson, Berggren, & Johansson, 2015), and for training of prehospital ambulance personnel (Berggren, Herrera Velasquez, Pettersson, Henning, Lidberg, and Johansson, 2018).

CONCLUSION

The aim of this chapter was to provide a description of how the C3Fire microworld has contributed to knowledge development concerning various aspects of team cognition and how the platform has been and can be utilized for research on team cognition. We have presented how the C3Fire microworld has been used to conduct different types of studies and how this has supported the development of team cognition research. Using microworlds to conduct research has, as has been argued (cf. Brehmer & Dorner, 1993), many advantages. Microworlds provide a platform where teams can be studied regarding structure, size, organization, roles, and different types of interaction. In this chapter we have shown how several aspects of team cognition have been explored in the C3Fire microworld, for example, team situation awareness, communication, shared mental models, coordination, and collaboration (in terms of command and control).

There are several results relating to team research that the content of this chapter has touched upon. Artman (1999) showed how microworlds can be used to investigate how various organizational configurations of teams affect performance and decision making. Microworlds were also utilized for the study of future command and control environments, as this required an approach that allowed for research on concepts that were too immature to be fully implemented from a technical point of view (Johansson, Persson, Granlund, & Mattsson, 2003). The three studies of how directed information flows concerning unit positions and sensor information presented in digital maps influences emergency management teamwork showed a number of interesting results. In novice teams, direct information improved performance, while this was not shown in studies involving professional participants, although communication was affected for all types of participants (Granlund & Granlund, 2011). The studies performed by Trnka and colleagues showed that a microworld can be used as a part of a hybrid simulation (Trnka & Jenvald, 2006; Trnka & Johansson, 2009). Further, the studies performed by Berggren and colleagues show how new measures of sharedness in teams can be developed and validated using microworlds as a research platform (Berggren, 2016).

Thus, microworlds can be used for aspects of team research such as method development, training, and testing of different psychological concepts (mental workload, situational awareness, motivation, expertise, trust, leadership, etc.), or as part of a larger experimental setup.

An important experience from working with microworld research is the lack of useful scenario descriptions in order to reproduce research findings. Hence, we call for better descriptions of scenario and design configurations to be able to replicate studies. This would be easy in these kinds of reproducible technical systems, where little is left to chance, except the behaviors of the participants. For research purposes, a standardized way of describing scenarios would make it easier to replicate studies and provide confirmations or refutations of earlier results. Such standardized descriptions could be implemented into any microworld platform, regardless of developer, for future researchers to use for replication so that findings generalize across time and across situations.

We also agree with McNeese and Pfaff (2012) on the importance of training participants prior to the actual data collection. This cannot be stressed enough, since reaching the sufficient capacity level of the participants will affect the outcome of the study. The situations and complexities require that the participants understand their part and have the skills necessary for controlling and interacting with the simulation.

A general observation is that participant background, i.e. whether the participants have an occupation or experience of situations resembling the microworld, seems to affect how they view the problems presented even when they are greatly simplified. For such participants, microworlds might be frustrating as they are overly simplistic and limit the possibility to utilize professional knowledge and tactics that could have been appropriate in a real-world situation or a more advanced simulation. This was observed for example in the studies conducted by Granlund, Granlund, and Dahlback (2011).

Using microworlds is an appreciative platform for team research. There is a learning effect of using a microworld platform for the participants, however, as the learning curve is quite steep the participants are easily up and running for the actual experiments in a short time. This compared to training teams of helicopter pilots or command teams, something that requires years of training and experience. Alas, as a researcher, you don’t want to waste valuable professional teams on testing new methods and instruments. When the methods and instruments have been validated and tested, then it is time to go into “the wild” with the research. This way you have feasible and useful hypothesises, concept models, and methods to use with the hard to access professional teams. Of course there is a difference between the real world of professionals and naive subjects and teams in microworld studies, but the benefits of testing material and methods to adapt them provides a cost-effective way of developing our team theories in a controlled and replicable environment.

REFERENCES

Alberts, D. S., Garstka, J. J., & Stein, F. P. (1999). Network centric warfare. Developing and leveraging information superiority (2nd ed.). Washington DC: CCRP Publication Series.

Artman, H. (1999). Situation awareness and co-operation within and between hierarchical units in dynamic decision making. Ergonomics, 42(11), 1404-1417.

Baker, D. P., & Salas, E. (1992). Principles for measuring teamwork skills. Human Factors, 34(4), 469-475.

Berggren, P. (2016). Assessing Shared Strategic Understanding. Linkoping Studies in Arts and Science, Dissertation No. 677. Linkoping, Sweden: Linkoping University Electronic Press.

Berggren, P, & Johansson, B. J. E. (2010). Developing an instrument for measuring shared understanding. In S. French, B. Tomaszewski, & C. Zobel (Eds.), Proceedings of ISCRAM 2010 (pp. 1-8). Seattle, WA: ISCRAM.

Berggren, P, Alfredson, J., Andersson, J., & Granlund, R. (2008). Assessing shared situational awareness in dynamic situations. NATO RTO HFM-142 Symposium, 1-7. Copenhagen. DK: NATO RTO.

Berggren, P, Alfredson, J., Andersson, J., & Modeer, B. (2007). Comparing measurements of shared situational awareness in a microworld with a developmental environment. 1FAC Proceedings Volumes, 10 (PART 1).

Berggren, P, Herrera Velasquez, M., Pettersson, J., Henning, O., Lidberg, H., & Johansson, B. J. E. (2018). Reflection in Teams for Training of Prehospital Command and Control Teams. ISCRAM 2018. Rochester. NY.

Berggren, P, Johansson, B. J. E., & Baroutsi, N. (2017). Assessing the quality of Shared Priorities in teams using content analysis in a microworld experiment. Theoretical Issues in Ergonomics Science, 18(2), 128-146.

Berggren, P, Johansson, B. J. E., & Ekstrom, E. (2016). Resilience through training - assessing cognition in teams. Proceedings of ISCRAM 2016.

Berggren, P, Johansson, B. J. E., Allard, O., & Torensjo, E. (2016). Training resilient medical teams. ECCE 2016. Nottingham.

Berggren, P, Johansson, B. J. E., Svensson, E„ Baroutsi, N.. & Dahlback, N. (2014). Statistical modelling of team training in a microworld study. Proceedings of the Human Factors and Ergonomics Society 58th Annual Meeting, (2011), 894-898. Chicago, II.

Berggren, P, Svensson, J., Horberg, U., Jonsson, S., & Hoglund, F. (2009). Shared priorities as a measure of shared understanding. Proceedings of the Europe Chapter Human Factors and Ergonomics Society (HFES) Conference 2009. Linkoping.

Bodeau, D., Fedorowicz, J., Markus, L., & Brooks, J. (2009). Characterizing and Improving Collaboration and Information-Sharing Across Emergency Preparedness and Response Communities. International Conference on E-Government, 192-200.

Boin, A., & McConnell, A. (2007). Preparing for Critical Infrastructure Breakdowns: The Limits of Crisis Management and the Need for Resilience. Journal of Contingencies and Crisis Management, 15(1), 50-59.

Brehmer, B. (1987). System Design and the Psychology of Complex Systems. In J. Rasmussen & P. Zunde (Eds.), Empirical foundations of Information and Software Science (pp. 21-32). New York: Plenum Publishing Cooperation.

Brehmer, B. (2000). Dynamic Decision Making in Command and Control. In C. McCann & R. Pigeau (Eds.), The Human in Command: Exploring the Modern Military Experience (pp. 233-248). New York: Academic/Plenum Publishers.

Brehmer, B. (2004). Some Reflections on Microworld Research. In L. R. Elliott & M. D. Coovert (Eds.), Scaled Worlds: Development, Validation and Applications (pp. 22-36). Aldershot: Ashgate.

Brehmer, B. (2005). Micro-worlds and the circular relation between people and their environment. Theoretical Issues in Ergonomics Science, 6(1), 73-93.

Brehmer, B., & Allard, R. (1991). Dynamic decision making: The effects of task complexity and feedback delay. In J. Rasmussen, B. Brehmer, & J. Leplat (Eds.), Distributed Decision Making: Cognitive models for cooperative work (pp. 319-334). Chichester: John Wiley & Sons.

Brehrner, B., & Dorner, D. (1993). Experiments with Computer-Simulated Microworlds: Escaping both the Narrow Straits of the Laboratory and the Deep Blue Sea of the Field Study. Computers in Human Behavior, 9, 171-184.

Brehrner, B., & Svenmarck, P. (1995). Distributed Decision Making in Dynamic Environments: Time Scales and Architectures of Decision Making. In J. P. Caverni, M. Bar-Hillel, F. H. Barron, & H. Jungermann (Eds.), Contributions to decision making. Elsevier Science Ltd.

Cannon-Bowers, J. A., Salas, E., & Converse, S. A. (1993). Shared mental models in expert team decision making. In N. J. Castellan (Ed.), Individual and group decision making (pp. 221-246). Hillsdale, NJ: Lawrence Erlbaum Associates.

Carlerby, M., & Johansson, B. J. E. (2017). The lack of convergence between C2 theory and practice. The 22nd International Command and Control Research and Technology Symposium, 1-20.

Cebrowski, A. K., & Garstka, J. J. (1998). Network-Centric Warfare: Its Origin and Future. US Naval Institute Proceedings, 124, 28-35.

Comfort, L. K. (2007). Crisis management in hindsight: Cognition, communication, coordination, and control. Public Administration Review, 67, 189-197.

Cooke, N. J., & Shope, S. M. (2004). Designing Synthetic Task Environments. In S. G. Schiflett, L. R. Elliott, E. Salas, & M. D. Coovert (Eds.), Scaled worlds: Development, validation, and application (pp. 263-278). Aldershot: Ashgate.

Cooke, N. J., Gorman, J. C., & Rowe, L. J. (2009). An ecological perspective on team cognition. In E. Salas, G. F. Goodwin, & C. S. Burke (Eds.), Team effectiveness in complex organizations: Cross-disciplinary perspectives and approaches (pp. 157-182). New York: Routledge/Taylor & Francis Group.

DeChurch, L. A., & Mesmer-Magnus, J. R. (2010). Measuring shared team mental models: A meta-analysis. Croup Dynamics: Theory, Research, and Practice, 14(1), 1-14.

Dorner, D. (1980). On the difficulties people have in dealing with complexity. Simulation & Gaming, 11(1), 87-106.

Dorner, D., & Schaub, H. (1992). Spiel und Wirklichkeit: Uber die Verwendung und den Nutzen computersimulierter Planspiele. Kolner Zeitschrift Fiir "Wirtschaft Und Pddagogik, 7(12), 55-78.

Dorner, D., Kreuzig, H., Reiter, E, & Staudel, T. (1983). Lohhausen. Vom Umgang mit Unbestimmtheit und Komplexitat. Bern: Huber.

Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64.

Funke, J. (1993). Microworlds based on linear equation systems: A new approach to complex problem. In G. Strube & K. F. Wender (Eds.), The cognitive psychology of knowledge (pp. 313-330). Amsterdam: Elsevier Science Publishers B.V.

Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking & Reasoning, 7(1), 69-89.

Funke, J. (2010). Complex problem solving: A case for complex cognition? Cognitive Processing, 11(2), 133-142.

Gonzalez, C., Vanyukov, P, & Martin, M. K. (2005). The use of microworlds to study dynamic decision making. Computers in Human Behavior, 21(2), 273-286.

Granlund, R. (1997). C3Fire A microworld supporting emergency management training. Licentiate Thesis, no. 598, Department of Computer and Information Science, Linkoping University.

Granlund, R. (2001). Web-based micro-world simulation for emergency management training. Future Generation Computer Systems, 17(5), 561-572.

Granlund, R. (2002). Monitoring distributed teamwork training. Linkoping Studies in Science and Technology, Dissertation No. 746. Linkoping, Sweden: Linkoping University Electronic Press.

Granlund, R. (2003). Monitoring experiences from command and control research with the C3Fire microworld. Cognition. Technology & Work. 5(3), 183-190.

Granlund, R., & Granlund, H. (2011a). GPS impact on performance, response time and communication - A review of three studies. ISCRAM201I, 8th International Conference on Information Systems for Crisis Response and Management. Lisbon, Portugal.

Granlund, R., & Granlund, H. (2011b). Using simulations to study the impact of GPS information in crisis response organizations. ISCRAM20I1, 8tlt International Conference on Information Systems for Crisis Response and Management. Lisbon, Portugal.

Granlund, R., & Johansson, B. J. E. (2003). Monitoring distributed collaboration in the C3Fire microworld. In S. G. Schiflett, L. R. Elliott, E. Salas, & M. D. Coovert (Eds.), Scaled worlds: Development, validation and applications. Aldershot: Ashgate.

Granlund, R., Granlund, H„ & Dahlback, N. (2011). Differences between students and professionals while using a GPS based GIS in an emergency response study. 14th International conference on human-computer interaction. Orlando, FL, USA.

Granlund, R„ Granlund, H„ Johansson. B. J. E., & Dahlback, N. (2010). The effect of a geographical information system on communication in professional emergency response organizations. ISCRAM2010, 7th international conference on information systems for crisis response and management. Seattle, WA, USA.

Granlund, R., Johansson, B. J. E., Persson, M., Artman, H., & Mattsson, P. (2001). Exploration of methodological issues in micro-world research-Experiences from research in team decision making. Proceedings of the first international workshop on cognitive research with microworlds, 73-82.

Gray, W. D. (2002). Simulated task environments: The role of high-fidelity simulations, scaled worlds, synthetic environments, and laboratory tasks in basic and applied cognitive research. Cognitive Science Quarterly, 2, 205-227.

Griine-Yanoff, T., & Weirich, P. (2010). The philosophy and epistemology of simulation: A review. Simulation & Gaming, 4I( 1), 20-50. 53470

Houghton, R. J., Baber, C., McMaster, R., Stanton, N. A., Salmon, P. M., Stewart, R., & Walker, G. H. (2006). Command and control in emergency services operations: a social network analysis. Ergonomics, 49, 1204-1225.

Johansson, B. J. E. (2005). Joint control in dynamic situations. Linkoping Studies in Arts and Science, Dissertation No. 972. Linkoping, Sweden: Linkoping University Electronic Press.

Johansson, B. J. E., Granlund, R., & Waern, Y. (2000). The communicative aspects of distributed dynamic decision making in the ROLF environment. 5th Conference on natural decision making. Stockholm.

Johansson, B. J. E., Persson, M., Granlund, R., & Mattsson, P. (2003). C3Fire in command and control research. Cognition. Technology & Work, 5(3), 191-196.

Johansson, B. J. E., Trnka, J.. & Berggren. P. (2016). A case study of C2 agility in the 2014 Vastmanland forest fire (No. FOI-R—4259—SE). Linkoping: FOI.

Johansson, B. J. E., Trnka, J., & Granlund, R. (2007). The effect of geographical information systems on a collaborative command and control task. Proceedings ISCRAM2007, 191-200. Delft, The Netherlands: ISCRAM.

Johansson, B. J. E., Trnka, J., Granlund, R., & Gotmar, A. (2010). The effect of a geographical information system on performance and communication of a command and control organization. International Journal of Human- Computer Interaction, 26(2-3), 228-246.

Kendall, M. G., & Babington Smith, B. (1939). The problem of m rankings. Annals of Mathematical Statistics, 10(3), 275-287.

Klimoski, R„ & Mohammed, S. (1994). Team mental model: Construct or metaphor? Journal of Management, 20(2), 403-437.

Korolja, N., & Linell, P. (1996). Episodes: coding and analyzing coherence in multiparty conversation. Linguistics, 34(4), 799-832.

Langan-Fox, J., Code, S., & Langfield-Smith, K. (2000). Team mental models: Techniques, methods, and analytic approaches. Human Factors, 42(2), 242-271.

Langan-Fox, J.. Wirth. A.. Code. S., Langfield-Smith, K., & Wirth. A. (2001). Analyzing shared and team mental models. In ternational Journal of Industrial Ergonomics, 28, 99-112.

Liu, D., Macchiarella, N. D., & Vincenzi, D. A. (2009). Simulation fidelity. In D. A. Vincenzi, J. A. Wise, M. Mouloua, & P. A. Hancock (Eds.), Human Factors in Simulation and Training (pp. 61-74). London: CRC Press.

McNeese, M. D., & Pfaff, M. S. (2012). Looking at macrocognition through a multimethod- ological lens. In E. Salas, S. M. Fiore, & M. P. Letsky (Eds.), Theories of team cognition: Cross-disciplinary perspectives (pp. 345-371). New York, NY: Routledge.

McNeese, M. D„ Bains, P„ Brewer. I., Brown. С. E., Connors. E. S., Jefferson, T.....Terrell.

I. S. (2005). The NeoCITIES Simulation: Understanding the design and methodology used in a team emergency management simulation. Proceedings of the Human Factors and Ergonomics Society 49th annual meeting, 591-594. Santa Monica, CA.

Mohammed, S., Ferzandi, L., & Hamilton, K. (2010). Metaphor no more: A 15-year review of the team mental model construct. Journal of Management, 36(4), 876-910.

Mohammed, S., Klimoski, R., & Rentsch, J. R. (2000). The measurement of team mental models: We have no shared schema. Organizational Research Methods, 3(2), 123-165.

Omodei, M., Elliott, G., Walshe, M., & Wearing, A. (2005). Networked fire chief: A research and training tool that targets the human factors causes of unsafe decision making in wildfires Mary Omodei, Glenn Elliott, Matthew Walshe and Alex Wearing. In B. Butler & M. Alexander (Eds.), Eighth international wildland fire safety summit. Missoula, MT: The International Association of Wildland Fire.

Osinga, F. (2010). The rise of military transformation Frans Osinga. In T. Farrell, T. Terry, & F. Osinga (Eds.), A transformation gap?: American innovations and European military change. Stanford, CA: Stanford Security Studies.

Ouyang, M., & Wang, Z. (2015). Resilience assessment of interdependent infrastructure systems: With a focus on joint restoration modeling and analysis. Reliability Engineering & System Safety, 141. 74-82.

Parush, A., & Ma, C. (2012). Team displays work, particularly with communication breakdown: Performance and situation awareness in a simulated forest fire. Proceedings of the 56th Human Factors and Ergonomics Society annual meeting, 383-387.

Persson, M., & Johansson, B. J. E. (2001). Creativity or diversity in command and control. In M. J. Smith, G. Salvendy, D. Harris, & R. J. Koubek (Eds.), Usability, evaluation and interface design - cognitive engineering, intelligent agents and virtual reality (Proceedings to HC12001) (Vol. 1. pp. 1508-1512). New Orleans.

Prytz, E. G., Rybing, J., Jonson, C.-О., Pettersson, A., Berggren, R, & Johansson, B. J. E. (2015). An exploratory study of a low-level shared awareness measure using mission- critical locations during an emergency exercise. Proceedings of the 59th Human Factors and Ergonomics Society annual meeting, 1152-1156.

Resick, C. J.. Murase, T., Bedwell, W. L., Sanz, E., Jime'nez, M., & DeChurch, L. A. (2010). Mental model metrics and team adaptability: A multi-facet multi-method examination. Croup Dynamics: Theory, Research, and Practice, 14(4), 332-349.

Salas, E., & Fiore, S. M. (2004). Team cognition: Understanding the factors that drive process and performance. Washington, DC: American Psychological Association.

Salas. E.. Elliott. L. R., Schiflett, S. G., & Coovert. M. D. (2004). Scaled worlds: development, validation and applications. Aldershot: Ashgate.

Salas, E., Fiore, S. M., & Letsky, M. P. (2012). Theories of team cognition. Cross-disciplinary perspectives. New York, NY: Routledge.

Schiflett. S. G.. Elliott. L. R.. Salas. E.. & Coovert. M. D. (2004). Scaled worlds: Development, validation and applications. Aldershot: Ashgate.

Smith-Jentsch. K. A.. Campbell, G. E.. Milanovich, D. M.. & Reynolds. A. M. (2001). Measuring teamwork mental models to support training needs assessment, development, and evaluation: two empirical studies. Journal of Organizational Behavior, 22, 179-194.

Sundin, C„ & Friman, H. (1998). ROLF 2010 A joint mobile command and control concept. Stockholm: Elanders Gotab.

Sundin. C., & Friman. H. (2000). ROLF 2010 The way ahead and the first step: A collection of research papers. Stockholm: Elanders Gotab.

Svenmarck, P, & Brehmer, B. (1991). D3Fire - An experimental paradigm for the study of distributed decision making. In B. Brehmer (Ed.). Proceedings of the 3rd MOHAWC Workshop on distributed decision making (pp. 47-77). Belgirate, Italy: Riso National Laboratory.

Swezey, R. W., & Salas, E. (1992). Teams: Their training and performance. Norwood, NJ: Ablex.

Tannenbaum, S. I.. Mathieu. J. E., Salas, E„ & Cohen. D. J. (2012). On teams: Unifying themes and the way ahead. Industrial and Organizational Psychology, 5(1), 56-61.

Trnka, J., & Jenvald, J. (2006). Role-playing exercise - A real-time approach to study collaborative command and control. International Journal of Intelligent Control and Systems, 11(4), 218-228.

Trnka, J., & Johansson, B. J. E. (2009). Collaborative command and control practice: Adaptation, self-regulation and supporting behavior. International Journal of Information Systems for Crisis Response and Management, 1(2), 47-67.

Wildman, J. L., Salas, E„ & Scott, С. P. R. (2014). Measuring cognition in teams a crossdomain review. Human Factors, 56(5), 911-941.

О The Dynamical Systems Approach to Team Cognition

 
Source
< Prev   CONTENTS   Source   Next >