Two types of data were gathered from the study. The first was quantitative and comprised of the answers to the post-test questionnaire. The second was qualitative and comprised of the participant comments during the application usage and the post-test interview.

Post-test questionnaire

The questionnaire had a total of 21 items.The statements and their responses can be seen in Figure 11.1. The responses are shown as percentages of the corresponding five-point Likert scale value (completely agree, somewhat agree, neither disagree nor agree, somewhat disagree, and completely disagree).

All participants agreed to some degree that they are interested in art, reflecting that they would be in the real-case user group for the application. Most participants agreed that the experience was pleasant (86%) and that using the application was easy (91%), and 81% agreed that they would recommend the experience to their friends or loved ones. Moving from one spot to another via the icons was reported as mostly logical (76% agreed), 67% of the participants agreed to some degree that the consequences of their actions resulted as expected, and 62% of the participants agreed that the transition from the museum to the cathedral felt natural. In contrast, the item that was received most negatively was about the image quality: only 19% agreed that the quality was good, while 71% disagreed to some extent.

The answers to the items pertaining to presence (questions 6 and 7) were somewhat more divided, with 67% reporting that they felt like they were there in the virtual space and 19% disagreeing. Just under half (47%) felt aware of their outside surroundings while in the virtual experience, 33% did not feel aware, and about 19% neither agreed nor disagreed. Similarly, the feeling of being immersed in the story was also split, with 52% agreeing, 14% neutral, and 33% disagreeing to some extent. When asked where participants would like to use a similar VR application, the items with the largest agreement were, first, in an educational establishment; next, at home; then, at a museum; and lastly, at a public cafe. Most of the participants agreed that they would like to know more in-depth details about the story (76%) and that they would like to get to know other artists and their work in a similar way (86%). Three-quarters of the participants (76%) agreed that the music was pleasant, and that the narration was interesting. Nearly all the participants (95%) felt that the environments they visited left an impression. Finally, none of the participants reported feeling nauseous during or after use of the application.

Think-aloud comments and post-test interview results

In this section, we present the analysis results based on the comments made by the participants during and after application use. All of the comments were transcribed from the experiment recordings. Each statement was extracted into single comments so that they could be reviewed and grouped, totaling 434 comments. The comments were analyzed in a cyclical manner (Saldana 2009), which allowed for examination of the data in several iterations. Participant comments were first transcribed, validated, and then coded according to their subject matter.The codes were then sorted into groups so that similar comments were together in one group. The initial transcription and validation was conducted by one researcher and

Post-test questionnaire results

FIGURE 11.1 Post-test questionnaire results.

reviewed by a second researcher, while the coding and grouping was conducted by two researchers in an affinity wall type of setting, then further reviewed by a third researcher to increase accuracy and validity. The results of this process can be seen in Table 11.1. In the table, the main groupings are in bold on the left, subcategories in the middle, and further descriptions on the right. The number of comments is in parenthesis next to the main groupings and subcategories.

As Table 11.1 shows, the majority of the comments made by the participants was about the usability of the application (19 out of 21 participants). While 20/ 75 of these comments stated that the application was generally easy to use, most (55/75) of the category’s comments concentrated on low affordability and difficulties activating the icons used to traverse the virtual environments. This somewhat contradicts the results of the questionnaire, where three-quarters of the participants agreed that the movement from one spot to another was logical. Exploration was also commented on by most of the participants (19/21) during or after using the application.The comments from this category were grouped into six subcategories representing different attributes, such as interest towards exploration,“I still have to check this [icon] out” (transl.) and elements the participants wanted to see and learn from textual and visual information in the environment,“[...J I would’ve wanted to go read those texts, but they were so far you couldn’t see them [...]” (transl.).

TABLE 11.1 The categories that resulted from the analysis of the post-test interview data. The categories are listed from most frequently mentioned to least frequently mentioned.


Degree of functionality and interactivity of application elements


Icon Activation


Icon responsiveness

Ease of Use


Level of ease perceived by participants

Icon Affordances


Ability to recognize & understand icons


Desire to explore and facilitation of exploration


Interest in Exploration


Desire to look around further

Interest in Textual Information


Wishing to read more about artworks

Ease ofVirtual Visit


Facilitation of virtual tours

Free Exploration


Independence in examining the VE

Desire to Learn More


Minting to obtain addition or re-hear information

Lack of Fine Details


Minting to see closer details of artworks


Level of acceptance of visual elements


Visual Execution


Technical aspects of 36D photography

Image Accuracy


Dick of Visual Precision

Expectation of Graphical Representation


Visual quality of artwork did not meet expectations

Image Quality


Elements of general image quality



Pixilation and blurriness


Influence of story and narration on overall experience


Interest in Narration


Degree to which narration appealed to participants

Supplementing the Museum Atmosphere


Impact of narration on museum atmosphere

Unclear Topic


Inability to discern narration subject


Significance of music and narrator on virtual experience


Narrator and Music Imbalance


Inability to hear narrator oner music

Impact of Music


Effect of music on general atmosphere

Voice of Narrator


Inability to hear narrator oner music


TABLE 11.1 (Cont.)



Bodily response to virtual environment and device





Adverse reactions to lack of body in the VE



Feelings of nausea




Reflections on sensitivity to VR-induced arousal



Reactions to perceived height in the VE




Discomfort caused by head- mounted display


Emotional valence and degree of interest towards the experience




Feeling pleasant, comfortable or easy



Having fun in the experience



Feeling amazed and intrigued



Feeling impressed



Showing general interest


Loss of reality and degree of absorption in the VE




Extent to which the experience felt realistic

Detachment from Real World


Loss of presence in reality



Becoming engrossed in the VE


Movement within and between an environment




Attitudes towards transitions unthin the environments





Attitudes towards transitions between environments


Feeling disoriented and uncertain




Loss of sense of location in the VE



Obscure purpose and questioning of expected actions


The extent of presence in theVE


Feeling Present


The sense of “being there"


Reactions based on personal memories and knowledge




Relating virtual environments/ elements to their real counterparts

Similarity to Guided Tours


Associating experience until real- world museum behaviors

Visual quality received mentions from 18/21 of the participants. These comments were concentrated on how the participants perceived the experience and the issues with it, such as remarks on visual flaws, insufficient image accuracy, and blurriness.The storytelling elements received comments from 16/21 of the participants.The comments were grouped into three subcategories: Interest in Narration, Supplementing the Museum Atmosphere, and Unclear Topic. The second subcategory discusses how the narration felt suitable in the museum atmosphere and how in turn that affected the experience positively, such as, “It made me feel that I wasn’t in a hurry anywhere” (transl.). Participants also made comments on the narration being unclear, as the narration started automatically when entering a scene with narration, without warning.

The category of Audio received comments from 12/21 participants that concentrated on the quality of the audio, including the voice of the narrator and the background music, and how the music affected the experience. One participant mentioned that the background music in the cathedral, “Was very peaceful and surely an effective way to escape the mundane” (transl.). Finally, 17/21 participants made comments regarding their Physical Comfort during and after using the application. A combined total of 18 comments by eight participants noted feelings of vertigo or body disassociation, such as,“It’s strange since I can’t see my arm” (transl.).

Although the previous study (Kauhanen et al. 2017) and the current study are not directly comparable because of differences in the questionnaires, there appear to be many improvements with the addition of the storytelling elements and other added features. In the previous study, 58/288 (20%) comments described feeling disoriented while in the virtual environment. Comments from the current study related to disorientation and confusion amounted to only 18/434, or 4% of the total comments, resulting in a clear decrease in disorientation. Similarly, comments related to immersion also increased from 1/288 (0.3%) to 26/434 (6%), suggesting that participants in this study were somewhat more engrossed in the viewing experience.

< Prev   CONTENTS   Source   Next >