Introduction: What is immersive journalism?

Astrid Gynnild, Turn Uskali, Sarah Jones, and Esa Sirkkunen

Immersive technologies are opening gateways to virtual realities that might change journalism forever. In the virtual world, journalism balances on the edge between imaginary approaches to fact-based creation and extended options for fakes. The journalistic maneuverings between reality and virtual reality are particularly intriguing to work with because they put the truth-seeking values of journalism into play. How can immersive technologies be applied to improve meaningful reporting and investigate storytelling in journalism? How can skills and knowledge of virtual reality empower news professionals? And how can journalism codes of ethics help shape the new platforms, shape a future in which journalism continues to play an important role in society?

Immersive journalism got a breakthrough in 2018, when virtual reality presentations were included in the series that was awarded the Pulitzer Prize in explanatory journalism. The staff of The Arizona Republic and USA Today Network were honored for “vivid and timely reporting that masterfully combined text, video, podcasts and virtual reality to examine, from multiple perspectives, the difficulties and unintended consequences of fulfilling President Trump’s pledge to construct a wall along the U.S. border with Mexico”. The symbolic effect of highlighting the value of virtual reality as a complementary tool in journalism was significant.

With this book we wanted to provide a comprehensive overview of immersive technologies in journalism at the turn of the 2020s. Immersive journalism is still in its infancy and it is so far uncertain in what ways and to what extent the immersive approach will expand in the years to come. As journalism scholars we believe that an important aspect of journalism research is to critically, constructively, and creatively investigate the potential implications of emergent technologies in journalism, even before they are widely adopted in societies. Thus, the volume builds on international research projects that started in 2016 and also integrates the latest research results by other scholars in the field.

Immersive journalism relates closely to the mobile expansion in global society. It is based on the premise of ubiquitous smartphones and is marketed as the new game-changer in the communication and media industry. Major news organizations around the world are experimenting with new opportunities for virtual presence and engagement among users - from daily 360-degree news videos to award-winning short VR documentaries. Immersive journalism is often called an “empathy machine” as well. Its audiovisual narratives have proved to be extremely effective in causing strong emotions among its users.

Originally, the term “immersive journalism” was coined by Nonny de la Pena et al. (2010, 291), and defined as “the production of news in a form in which people can gain first-person experiences of the events or situation described in news stories”. In this book, immersive journalism is defined more broadly as the use of immersive technologies, like 360-degree video, virtual reality, augmented reality, cinematic reality, and mixed reality in journalistic storytelling. Immersive journalism is an experiential approach that allows users to experience, and subsequently become immersed in, stories created not in the real world but in a virtual, augmented, or mixed reality.

When discussing immersive technology we thus refer to a broad spectrum of visual approaches in journalism. At the one end we have experiences created by 360- degree cameras. Whilst often called VR, 360-degree experiences are, more accurately, spherical films that allow interactivity through looking around and choosing where to look in an environment. At the other end of the immersive spectrum we have virtual reality (VR), which is more often than not computer-generated. With the VR experiences, there is more agency in the environment. Users may choose to take different paths, move more freely and interact with characters.To add to the terminology complexity, the umbrella term “cross reality” (XR) is gaining popularity, especially among engineering scholars and computer scientists. Cross reality refers to new forms of reality content creation in which all the above technologies may be mixed; digital objects are brought into the physical world and physical objects are brought into the digital world.

Because of its no-distance-at-all illusion, virtual reality experiences feel more real than other forms ofjournalistic storytelling. Evans (2019,11) argues thatVR is “a medium that offers something - immersion, interaction, even co-presence - that other media cannot due to the degree of intimacy and interaction thatVR can achieve”. Presence is a distinctive quality distinguishing immersive technologies from traditional film, computer games, television, and documentary. In journalism, the competitive advantage of immersive storytelling lies in its ability to create a sense of emotional connections to people, events and places. Bailenson (2018) argues that in the case of climate change deniers, for instance, the only way to change someone’s mind is to put people in a position to directly experience something in order to “see the light”. The perception of presence is derived from the notion of suspending all disbelief in the world (Pimental &

Texaria 1993), and one of a mediated world that is not mediated (Lombard & Ditton 1997).

Immersive journalism is thus a powerful new way to capture the attention of a multitude of users. The interactive traits of immersive technologies include hot spots that easily “teleport” users to new locations.The rapidly increasing options for virtually influencing audiences provide the journalism institution with genuinely new and extremely powerful storytelling tools.The need for ethical storytellers on multiple platforms will most likely grow in the years to come: journalism proves itself relevant to society by demonstrating that news professionals are at the forefront of immersive innovations, rather than passively observing what is going on. For journalism to provide knowledgeable storytelling and prove relevance, experiential approaches with new technologies would profit by starting from within; journalism innovation leads to innovation journalism (Gynnild 2014).

Immersive technologies are empowering in that they make journalism more relevant. In the era of touchscreens, individuals are used to navigate with their fingertips, and with immersive technologies new kinds of navigation, for example gazing, are easily added. At this point in history, audiences still expect journalism stories to be accurate, credible, and ethical. Even though immersive technologies are still in their infancy, we already know that users are easily persuaded by visual messages. Our brain believes so strongly in what it sees in VR that we might not be able to distinguish fake news from real news. Falsified videos of authorities such as former president Obama are hardly able to be differentiated from authentic clips (Diakopoulos & Johnson 2019), and boundaries between storytelling and storyliving are potentially getting blurred not only in gaming and social media but in journalism in general.

Immersive journalism experiences have evolved rapidly since 2012 when Nonny de la Pena provided the first, animated immersive journalism experience at the Sundance Film Festival. The video was a story on a diabetic man collapsing while standing in line at a food bank in Los Angeles. In 2015, immersive journalism's most innovative beat was probably The New York Times’ mini-documentary The Departed, which could be downloaded to viewers’ smartphones via the NYTVR app.When used via Google Cardboard it offered a glimpse of the possibilities with mobile immersive journalism. In 2016, the Oculus Rift or HTCVive offered the first high-end head-mounted display of immersive journalism experiences. By that time expensive HMDs also needed powerful computers and cables attached to the user. Only two years later, however, the first affordable stand-alone VR devices started to pour into the markets like Oculus Go,XiaomiMiVR,and Lenovo Mirage Solo.Three years later, in 2018, virtual storytelling was already on the list of Pulitzer winners in the United States, as the staffs of the Arizona Republic and USA Today Network were awarded the Explanatory Reporting prize.

Futurists and tech optimists have since then envisioned that immersive technologies will disrupt education, travel, health, and the media on a global scale. Leading tech companies have invested heavily in virtual reality since 2014, and CEOs have been hoping that users would quickly adapt to the new devices, applications, and platforms. But, as media history has taught us many times, consumers’ behavior is hard to change. Sometimes it takes decades to happen.This was true with the adoption of radio, television, internet, smartphones, and obviously also with immersive-technology-related devices such as wearables. Smartwatches, smart glasses, and smart rings are heavily marketed, but are still used mainly by innovators and early adopters.The marketing hype of virtual reality and other immersive technologies has not yet, at the turn of the 2020s, materialized into reality - at homes or in our daily lives. The number of people who have not ever tried HDMs is still far beyond those who have. By 2018, the total number of active virtual reality users was estimated to be about 170 million. About 28 million VR devices (head- mounted displays) were sold that year, and 34 million were projected for the year 2019 (Statista.com 2019a). Compared to the numbers of smartphones, of which more than three billion copies were sold in the same year (Statista.com 2019b), the use of virtual reality content is still, rightly, quite marginal in a global perspective.

Naturally, the experienced delays in public response have created uncertainties about future investment, not least in the media business. So far, a widespread hypothesis is that the equipment is too big and complicated, and that it creates a form of loneliness that has to be resolved for it to take off. Even the first step, of putting the head-mounted display on the face for the first time, has been a threshold for many people. And, if their first immersive experience was like a rollercoaster ride, they might have undergone virtual simulation sickness, also known as motion sickness, a phenomenon which hopefully will be resolved at later stages of technological development.

Indeed, as human beings we have interacted with media content in different ways. Already during the Stone Age, the cave paintings offered immersive experiences. In 1838, stereoscopic photos were used after Charles Wheatstone’s research demonstrated that the brain processes two different two-dimensional images from each eye into a single object of three dimensions. When two images are viewed side by side through a stereoscope, it gives a sense of depth and immersion. Later, individuals have experienced immersion in many other forms, from books, radio, films, television, computers, game consoles, and even smartphones. Immersion has been at the heart of a body of scholarly work; by bringing together immersion and interactivity, it is argued that total art can be achieved in virtual reality, with the emphasis lying in narrative (Evans 2019). Evans (ibid.) has narrated a short cultural history of virtual reality, starting from Victorian panoramas and ending with Second Life’s computer-generated worlds and avatars designed by Linden Labs in 2003. Second Life was not yet an immersive VR experience, though; rather, a kind of social gaming through the screen of a PC.

In the last few years, immersive journalism has started attracting vanguard journalism scholars inquiring into emergent technologies and their influence in journalism. (See, for example, Sirkkunen et al. 2016; Jones 2017; Sundar et al. 2017; Sanchez Laws 2017; Bosworth & Sarah 2019; Aitamurto el al. 2018; Kukkakorpi 2018; Aitamurto 2019; Hassan 2019; Mabrook & Singer 2019; Pavlik 2019; Sanchez Laws 2019.) In the first wave of immersive journalism research, the production consisted mainly of conference papers, master’s theses, and journal articles. Our investigations suggest that this opening phase of the new research field lasted from 2010 to 2019, when there was a notable shift in research investment. In the second wave the research field expanded into building more coherent overviews of immersive journalism via larger works. Melissa Bosworth and Lakshmi Sarah (2019) conducted an impressive number of interviews, analyzed and classified respectable numbers of immersive narratives and news. Their book offers an excellent review of virtual reality, augmented reality, and mixed reality storytelling cases from recent years. John V. Pavlik (2019) argues that a new form of mediated communication has emerged along with virtual reality, haptic technologies, interactive documentaries and drone media, and introduced the new terms “experiential news” and “experiential media”. Ana Luisa Sanchez Laws (2019) emphasizes the importance of theorization in immersive journalism on the implications of full bodily experiences.

This edited volume continues and enlarges the scope of immersive journalism research from 21 authors representing scholars from five countries (Finland, Norway, the United Kingdom, the United States, and Spain). The publication is the first international and interdisciplinary joint-venture project aiming at understanding the many layers of immersive journalism storytelling. The approaches vary from comprehensive case studies via philosophical discussions on the premises and perils of emergent technologies to a broad spectrum of perspectives on immersive journalism from professional practitioner, researcher, and educator perspectives.

Much of the research developed for this volume derived from two internationally ongoing research projects, and the volume also gathers the latest analyses from other scholars studying the implications of immersive technologies in journalism. News streams related to immersive journalism have been systematically monitored for several years, with the internet as the most valuable data source.

In the project “Engaging Services in Virtual Reality (VIRJOX)”at the University of Tampere and University of Jyvaskyla in Finland we developed, designed, and evaluated journalistic storytelling concepts for VR with an agile design process (Vaataja et al. 2018).The same group has continued to explore the various ethical aspects of immersive journalism in a follow-up project (EMORES).

The four-year project “Visual Surveillance Technologies in the News Media (ViSmedia)” at the University of Bergen, Norway, is grounded in the framework of Responsible Research and Innovation (RRI). RRI is a methodological framework that helps facilitate the co-creational, collaborative resources of universities, industry, education, and civic society (Owen et al. 2012, Stilgoe et al. 2013).The RRI is applied to different stages of technological research and innovation. The framework guides processes in which the involved actors engage in mutual actions to find sustainable solutions to the grand challenges of our time. Collecting empirical data on emergent technologies, prompting public debate, adjusting the course, and providing responsible foresight is thus a main aim of RRI.

The volume further conceptualizes immersive filmmaking practices, using original productions as examples, including The Town that Blew Away (Jones, Aesthetica

Film Festival, 2017) and Google-supported Digital News Initiative projects onVR Journalism, including the Coventry Blit' VK.

The 16 chapters are divided into four sections that deal with essential issues of immersive journalism: trends and experiential practice, ethics, production and design, and education.

In the first section, an overview of immersive journalism is presented. Although we are seeing the presence of immersive journalism in every continent (Sanchez Laws 2019), there is a limited understanding of practices and case studies in Latin America and Asia.The United States is certainly leading the way as the main testbed for media content. These ideas are picked up in the first section. Esa Sirkkunen, Jorge Vazquez-Herrero, Heli Vaataja, and Turo Uskali explore the use of 360- degree videos in journalism, through a content analysis drawing on the practices of The New York Times and Euronews together with interviews with journalists making immersive content. This is followed by a case study that focuses on the work of Euronews. Joakim Vindenes and Astrid Gynnild analyze the production and workflows in Euronews and how they have applied a low-cost approach to immersive journalism. In the final chapter of this section a global overview of immersive journalism is presented by Sarah Jones. In this, the need for accessibility is highlighted to bring the technology' to different communities so that experiences are authentically created.

It is then that we move to arguably one of the most important sections: ethics. It begins with a chapter by Turo Uskali and Pasi Ikonen, who summarize what we know about the emotional implications of virtual reality experiences. Their discussion is followed by a case study on Project Syria. Project Syria was created by Nonny de la Pena in 2014 and uses virtual reality technologies to put the audience “on scene”, enabling people to feel as if they are truly witnesses to the violent events in Syria. In this case study, Siri Flatlandsmo and Astrid Gynnild concentrate on accuracy in immersive journalism and why this is implied in VR experiences. Then follows a chapter by Deborah Johnson, who discusses emerging technologies in the light of anticipatory' governance and ethics. It then focuses on the potentials for surveillance and manipulation of virtual experiences and the impact that this can have on an audience.

The claims that immersive journalism can be an empathy machine have been central to arguments for news industries to develop the technology in order to engage new audiences. In the final chapter of this section Sarah Jones explains why immersive journalism does not necessarily increase empathy and instead needs to be thought of in new ways for the technology and journalism to reach new audiences.

We move then to look at the production and design of this emerging form of journalism. With the demand on journalists to become multi-skilled and tell stories across platforms, it is important to reflect and critically analyze the production processes within immersive journalism and the implications around this.

This section begins with David Dowling looking at the concepts of place- based journalism. It considers the implications for interactivity and production aesthetics and how branding plays a role within production. A second case study by

Ilona Ilvonen, Joel Vanhalakka, and Nina Helander follows the creation of value in immersive journalism in The New York Times and their VR application NYTVR. The process of design is the focus of the next chapter by Chelsea Kelling and six other contributors from the project VIRJOX. The chapter culminates in building the hierarchy of needs of user experiences, something that designers should take seriously into account when developing immersive journalism and immersive products in general.

A lot of scholarly thought around immersive media has come from game studies, and there is certainly a lot that can be learned within immersive journalism from these traditions. In Chapter 12, Jonne Aijoranta, Raine Koskimaa, and Marko Siitonen take a look at pioneering examples of immersive gaming and how these impact journalism. This is developed further in the following chapter, where the focus turns to Augmented Reality. Within AR, we are looking at the application and situating of digital objects within our own environment. Pasi Ikonen andTuro Uskali focus on how this now plays a role in journalism.

The final section concerns teaching and learning. For educators reading this book, there is a need to continue to ensure that students are familiar with new trends and skills. It is important that students are thinking critically about emerging technologies and how it can reach new audiences, but also that core journalistic skills are not lost within the excitement of a new technology. In the first in this section, Turo Uskali and Pasi Ikonen provide an overview of teaching approaches in five different countries and how the educators bring immersive journalism into the classroom. This is followed by a chapter by Lars Nyre and Joakim Vindenes, who discuss immersive journalism as witnessing. They conceptualize the designs and strategies behind first-person experience in VR journalism and present experiments from an innovative pedagogy approach to teaching journalism.

The book ends with a chapter on the future of immersive journalism. As the technology develops quickly, it is important to think more broadly about the implications of immersive journalism on the media industry. To this end, trends and future trajectories for immersive journalism are analyzed, looking at where and how this industry, which is full of questions, debates, concerns, and opportunities, can develop and establish itself in responsible ways.

References

Aitamurto, T. 2019. “Normative Paradoxes in 360-degree Journalism: Contested Accuracy and Objectivity.” New Medio & Society 21( 1): 3-19.

Aitamurto, T., Z. Shou, S. Sakshuwong, J. Saldivar, Y. Sadeghi, & A. Tran. 2018. “Sense of Presence, Attitude Change, Perspective-Taking and Usability in First-Person Split- Sphere 360-degree Video.” In: Proceedings oj the 2018 CHI Conference on Human Factors in Computing Systems. New York: ACM.

Bailenson, J. 2018. Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do. 1st ed. New York: WW Norton & Company.

Bosworth, M. & L. Sarah. 2019. Crafting Stories for Virtual Reality. London: Routledge.

Diakopoulos, N. & D. Johnson. 2019. “Anticipating and Addressing the Ethical Implications of Deepfakes in the Context of Elections.” October 23. Available at SSRN: https:// papers.ssrn.com/sol3/papers.cfm?abstract_id=3474183 [Accessed 15 March 2020].

Evans, L. 2019. The Re-Emergence of Virtual Reality. 1st ed. London: Routledge.

Gynnild, A. 2014. “Journalism Innovation Leads to Innovation Journalism: The Impact of Computational Exploration on Changing Mindsets.” Journalism 15(6), August: 713-30. doi: 10.1177/1464884913486393

Hassan, R. 2019. “Digitality, Virtual Reality and the ‘Empathy Machine’.” Digital Journalism, published online 2 January. doi:10.1080/21670811.2018.1517604

Jones, S. 2017. “Disrupting the narrative: Immersive journalism in virtual reality.” Journal of Media Practice /«(2-3): 171-185. doi:10.1080/14682753.2017.1374677

Kukkakorpi, M. 2018, n.p. “Immersive Journalism: Presence and Engagement in Conflict News Stories: Close Reading of The New York Times’ Virtual Reality Stories.” Master’s thesis. Finland: Media and Communication Studies, University of Helsinki.

Lombard, M. &T. Ditton. 1997. “At the Heart of It All: The Concept of Presence!' Journal of Computer-Mediated Communication 5(2). September.

Mabrook, R. & J.B. Singer. 2019. “Virtual Reality, 360-degree Video, and Journalism Studies: Conceptual Approaches to Immersive Technologies.” Journalism Studies 20(14), January. https://doi.org/10.1080/1461670X.2019.1568203

Owen, R., P. Macnaghten, & J. Stilgoe. 2012. “Responsible Research and Innovation: From Science in Society to Science for Society, with Society.” Science and Public Policy 39(b): 751-760. https://doi.org/10.1093/scipol/scs093

Pavlik, J. V. 2019. Journalism in the Age of Virtual Reality: How Experimental Media Are Transforming News. New York: Columbia University Press.

Pimentel, K. & K.Teixeira. 1993. Virtual Reality: Through the New Looking G/dss.Windcrest/ McGraw-Hill/TAB-Books.

Pulitzer.org. 2018. “Staffs ofThe Arizona Republic and USA Today Network.” www.pul- itzer.org/winners/staffs-arizona-republic-and-usa-today-network [Accessed 15 March 2020].

Sanchez Laws, A.L. 2017. “Can Immersive Journalism Enhance Empathy?” Published online 20 October. Digital Journalism 1-16. doi:10.1080/21670811.2017.1389286

Sanchez Laws, A. L. 2019. Conceptualizing Immersive Journalism. London: Routledge.

Shin, D. Sc F. Biocca. 2017. “Exploring Immersive Experience in Journalism.” New Media & Society 1 -24. doi: 10.1177/1461444817733133

Sirkkunen, E. &T. Uskali. 2019. “Virtual Reality Journalism.” In: T.PVos Sc F. Hanusch (eds), The International Encyclopedia of Journalism Studies. Hoboken, NJ: Wiley Blackwell.

Sirkkunen, E., H. Vaataja, T. Uskali, & P.P. Rezaei. 2016. “Journalism in Virtual Reality: Opportunities and Future Research Challenges. Academic Mindtrek ’16.” In: Proceedings of the 20th International Academic Mindtrek Conference. New York: ACM, pp. 297-303.

Statista.com. 2019a.“Unit Shipments of Virtual Reality Head-mounted Displays Worldwide from 2017 to 2018 and in 2022 (in million units).” www.statista.com/statistics/509154/ head-mounted-displays-worldwide-shipments/ [Accessed 22 September 2019].

Statista.com. 2019b. “Number of Smartphone Users Worldwide from 2016 to 2021 (in billions) .’’www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/ [Accessed 22 September 2019].

Stilgoe, J., R. Owen, Sc P. Macnaghten. 2013. “Developing a Framework for Responsible Innovation.” Research Policy. 42(9): 1568-1580. https://doi.Org/10.1016/j.respol.

2013.05.008

Sundar, Shyam S., Jin Kang, & Danielle Oprean. 2017. “Being There in the Midst of the Story: How Immersive Journalism Affects Our Perceptions and Cognitions.” Cyberpsychology, Behavior, and Social Networking 20(11): 672—682.

Vaataja, H., M. Turunen, I. Ilvonen, E. Sirkkunen, T. Uskali, C. Kelling, T. Keskinen, A. Burova.V. Makela, A J. Karhu (eds). 2018. VIRJOX. Engaging Services in Virtual Reality. Finland: Tampere University ofTechnology. https://tutcris.tut.fi/portal/en/publications/ viijox(9bf3cfba-f2be-4ecf-a800-b7ca8c04faad).html [Accessed 14 October 2019].

 
Source
< Prev   CONTENTS   Source   Next >