Introduction

The aim of this paper is to give an account of the application of immersive video projected within immersive environments for the educational purposes. The immersive systems that are in the focus of this study use a screen as a method of presenting photographic quality content. What is more, the observers field of view creates the illusion of being in another place or taking a ‘virtual travel when viewing such a screen. These environments are used for scientific and entertainment purposes, but similarly to 19th century panorama paintings, virtually transport spectators to different locations and also become a form of presentation of the world that does not forget its educational merit [1]. Edutainment projects for immersive environments aim at large-scale immersive screens in order to gather large audience. A number of analog planetariums existing in the world were converted into digital projection systems [2] to fulfill the demand for immersive presentations. However, little attention has been paid to 360° cylindrical screens, in particular in combination with non-linear storytelling based on 360º video. The existing 360° projection systems [3, 4], envelope viewers in panoramic forms of storytelling. In the pages that follow, it will be argued that object-based panoramic interactive narrative has not been applied to such immersive cylindrical screens, therefore this paper presents a concept for motivating and more effective learning via interactive narratives. Several attempts have been made to hyper-narrative interactive cinema [5, 6], but they refer mostly to a rectangular screen. However, Yu [7] highlights that immersive environments provide greater spatial context than rectangular screens (cinema, television) and may inspire learners by providing new pedagogical methods. Additionally, the research to date has tended to focus on linear storytelling for fulldome environments [2, 7] rather than for non-linear narratives presented within cylindrical environments.

Discover Krakow is an example of 360° film that enables armchair travel (understood as a metaphor of viewing; a new way of seeing the world) to Krakow in Poland. This city is the second largest city in Poland and is the most visited place in this country. Krakows historic centre was inscribed to the prestigious UNESCO World Heritage List in 1978. The researcher created a panoramic film that transports the viewer to this city. This experience of being in other space can be achieved by the use of immersive environments. The projection of this 360° film was performed on five different screens. Three of them were based on 360° screens (a,b,d), one on a fulldome projection screen (c) and one on a large hemispherical screen (e). These include: (a) Cyclorama (Montreal); (b) Arena 360 (Plymouth); (c) Immersive Vision Theater (Plymouth); (d) ICCI360 (Plymouth) and (e) iDome (Hong Kong).

Table 1 compares the dimensions and technical details of the five immersive environments where ‘Discover Krakow was presented. Cyclorama (a) was developed by Vision3D lab at the University of Montreal [8]. Arena360 (b) and ICCI360 (d) were 360° enterprises organized by ICCI department from the University of Plymouth. Immersive Vision Theater (c) – IVT – is a permanent 40 seat structure at the University of Plymouth and enables a digital fulldome projection. iDome (e) [9, 10] is a hemispherical projection system that gives a fully immersive experience for a small group of spectators. It was developed at the UNSW iCinema Research Centre and tested by the researcher at the City University of Hong Kong in the School of Creative Media. As shown in Table 1, Arena360 and ICCI360 have similar structures, but the second screen is larger and is designed for larger audience. Only IVT is equipped with seats, other screens are experienced in standing position which is one of the significant aspect of education within immersive environments, discussed later in the paper.

This paper has been divided into seven parts. The second section in this paper (after this introduction) describes the idea of the panoramic video as a basis for educational projects. Immersive environments are the focus of the third part of this paper. The fourth section illustrates the process of creating an interactive 360° film in Krakow in Poland. Then, the idea of interactive narrative is presented which has the potential to be applied to education for collaborative group-based learning environments delivered over immersive environments. Finally, the last part of this paper summarizes the potential of immersive environments towards educational purposes, where students could be motivated for further research.

360° video

Video panoramas (also called 360° video or panoramic video) overcome the passive limitations of the traditional video and provide the user with the possibility to interactively explore a moving image by changing the viewing direction. The horizontal field of view (HFOV) of video panoramas covers 360 degrees, while the vertical field of view (VFOV) is less than 100°. This type of panoramas are necessary for 360° screens while iDome needs spherical video, where VFOV is 180°. Sarmiento and Quintero [11] focus on the creation of 360° video in 3D modeling software because panoramic cameras are still high-priced. Ladybug cameras from Point Grey Research (1) enable the creation of spherical video. The company has released three models of spherical video cameras to date. The maximum resolution of footage recorded with the latest model is 5400x2700 pixels (spherical format) and this resolution is also not sufficient for some projection systems, where the resolution of approx. 9000x1000 (cylindrical format) is necessary for the display of high-definition footage. The solution to this limitation is the application of a set of cameras (Sensocto camera [12], Spherecam [13], Totavision camera [14]), but the stitched 360° cylindrical film is not appropriate for iDome.

Computer-based interactive panoramas appeared in the 1990s [15], as a result of the advances in digital photography. No longer was the creation of rotundas necessary to preview the panoramic work. The first video panoramas appeared at the beginning of 2000s [1]. It was a laborious process to present such recording in an interactive form. 360° video can be demonstrated on a computer screen (using panoramic viewers) or on an immersive screen.

Panoramic viewers provide opportunity to display 360° images on a computer monitor. The use of this type of software based on Flash technology (KrPano, Lucid Viewer (Figure 2), Pano2VR) does not offer full immersive experience for the viewers, because they can only perceive a miniature part of the panoramic image. HTML5 based panoramic viewers will only improve the performance of the display of large files of video panoramas over the Internet. Immersive environments (discussed in the next section) eliminate disadvantages of flat computer screens and provide a wide field of view, which creates the illusion and sensation for learners as in the panoramas presented in rotundas.

Historical panoramas failed to provide visual explanations of the images presented, whereas in digital technology, these possibilities are almost endless. The combination of panoramic images and panoramic video enhanced by elements of non-linear narratives (defined in the fifth section of this paper) projected in immersive environments creates a new potential for educational projects by inspiring observers to explore the additional information that are available only at specific times during the presentation. For example: exploration of geographical spaces, heritage sites or application of history events by merging 3D modeling and video panoramas as presented in [16].

This project, about UNESCO World Heritage site, is not based on computer-generated content or virtual reality renderings, but on recordings from 360° video cameras. The following section focuses on application of immersive screens that are used not only for passive consumption of the story, but particularly for active exploration of interactive narrative produced by the application of a spherical video camera.

Immersive environments

The idea of immersion, firstly introduced through the employment of cave paintings and panoramic baroque ceilings in churches (3), can now be experienced within immersive environments where digitally projected or displayed images surround the observer and enrapture its senses [17]. The effect of total immersion of the depicted scene presented in a contemporary panoramic arena has not changed in the aspect of preparing a space for panoramas illusion. The panorama cannot have visible edges and intrusive elements of the outside world (e.g. light) are blocked out in order to create dark spaces. Observation platform is no longer necessary, because the application of digital projection systems and the 360° painting does not need to be lit from the back.

The production of contemporary wrap-around spaces is based on panoramic rotundas, but no longer static images are presented for the observers. Immersive environments discussed in this paper are based on the photographic quality visual content. This paper attempts to indicate that immersive screens provide an appropriate method of presenting non-linear narratives that motivate learners. For several years researchers have aimed to construct environments that would enable people to believe that they are in a different location or that they participate in a narrative. The idea of such an imaginary space was realized via CAVE [18], however cylindrical environments with stereo projection [3, 19, 20] seem to realize the dream of first panorama painters to localize viewers inside a presented story.

Stereo technology was not applied to this project, because omni-directional stereo video recording is still in realization. The diagram (3) reveals the immersive advances in technology starting from cave images thousands of years ago until the possible development of gigapixel video panoramas (not applied yet) and spherical broadcasting in the future, excluding elements that were the catalyst for these changes. For instance, the first digital 360° image appeared when specialist software for stitching images was released in 1994, which started a new era for digital panoramas. The development of equipment (panoramic cameras, flexible films) and technology was also significant in the process of generation of 360° imagery and 360° video which were the basis for the creation of the content for immersive environments.

In his major study, Grau [21] identifies the development of immersive spaces and states that immersion is created when the art work and the projector converge (the medium and the message) are impossible to separate. In this manner the medium (in most cases a screen) becomes invisible.

This is also one of the aims of contemporary creators to involve the audience so they are hypnotized by a wrap-around screen and a 360° film seems to provide with the strongest impression of being at the different location. This stimulating process generates the unity of space and time. For the purpose of this paper, the immersion is interpreted as the impression of being surrounded by 360° panoramic imagery and video. Therefore, immersive environments are spaces that facilitate this impression and offer the display of interactive narratives and new forms of digital presentations. This paper explores the opportunity for the inclusion of interactive narratives within a high quality photographic environment. What is more, this study consolidates the findings from the practical research and proposes the potential of educational values in such environments.

The ultimate goal of immersive environments that improve the process of learning is to provide opportunities for the creation of spaces that:

• are perfect displays for presenting seamless images;

• a viewer no longer determines the number of projectors involved in the creation of the illusion of reality [22];

• create the feeling of being there;

• provide spectators with the real experience that they could not otherwise have [23];

• install the participant in a hermetically closed-off image space of illusion [17];

• reach a large number of audiences, similarly to a painted panorama which was a mass medium of the 19th century;

• present the content in the way that the medium (screen) is invisible;

• generate possibilities to access supplemental information.

These factors explain that immersive spaces with non-linear narratives are appropriate pedagogical methods. However, in order to design a transportable panoramic immersive environment for a classroom in a school a few aspects have to be considered in advance. These are:

• desired size of audience;

• number of projectors;

• software;

• position of projectors;

• specification of projectors (brightness, resolution etc.);

• size and the curvature of the screen;

• powerful computers for running software and controlling projectors and speakers;

• type of spatial sound;

• material for the screen;

• a room (size, shape and facilities) or a temporary building for accommodating the immersive screen.

A budget is another factor that should appear in the list above, because educational projects have to be created within a limited time-scale and money. The application of computers and digital HD projectors is commonly used for enveloping the observer in the image space, however new technologies such as LED or plasma screens could be potentially employed in the creation of immersive spaces in the near future and reduce the cost of these enterprises in the education sector. Digital projection causes a number of problems:

• shadows created by intersecting the light created by projectors;

• differences in brightness, caused by different working periods of expensive projector lamps;

• limited resolution;

• dividing sections;

• geometric misalignments;

• color variations.

Google Liquid Galaxy has already presented an immersive arena based on a number of LCD screens that are arranged in a circle and offer an immersive effect for Google StreetView and Google Earth imagery [24]. The website of the project describes how to build and synchronize computers in order to create 360° immersion.

To sum up this section, painted panorama changed the way of narrating events. Artworks, presented as circular paintings were not created for an individual, but rather for a significant audience. Furthermore, the point of view was not fixed, but mobile. Additionally, the spectators were surrounded by the 360° painting which was no longer just in front of the observer. The same rules for perceiving 360° artworks are applied to digital immersive environments where a significant number of viewers are hermetically enclosed within the screen and have the opportunity to move inside this space in order to experience the same narrative from different perspectives.

The 360° film discussed in the next section challenges the traditional rules that were established more than 100 years ago. These are a sitting arrangement and a rectangular screen. These two hypnoses apply partially to fulldome sceens (the audience still has to sit in order to experience immersive stories) and fully to cylindrical screens, because spectators can freely move and no longer observe a rectangular display. 'Discover Krakow' proposes a new method of experiencing films.

Discover Krakow

Discover Krakow is the title of the 360° film recorded using a spherical video camera – Ladybug2. The researchers hometown has one of the largest squares in Europe (Figure 4) and this is also a location where the immersive film starts and finishes. The tourists approach was a leading idea for the generation of a panoramic linear film which was then extended to create non-linear 360° stories around this city.

The aspect of non-linearity helps the audience gathered within immersive screen to collaborate in order to acquire additional information about the city. The audience assembled within an image arena are encouraged to solve clues and puzzles in the different parts of the city and in this way expand their understanding of the site that they have never visited before. The observers can gain knowledge not only of the history of the city and well-known buildings, but also they have an opportunity to study about historical people who used to live here.

The spectators do not have the power to decide about further developments of the narrative as discussed in [25, 26], but they have an opportunity to explore additional information that appear as they are watching an immersive film. The audience is encouraged to remember reference numbers that appear during the linear story and represent additional stories that could be played immediately as they appear on the screen or at the end of the 360° video.

This 360° recording of a 45-minute tour on a horse carriage through the streets of a historical centre of Krakow was adequately planned and an additional four persons were welcomed to participate in this experiment (Figure 5). Their task was to keep the tripod in the middle of the horse carriage. The setting time of Ladybug camera is at least 15 minutes, due to numerous cables which have to be connected and a laptop to be powered up and this creates a number of restrictions during the recording outside the office (limited capacities of a battery and a hard drive).

Figure Preparations before the recording of 360 video in Krakow

The process of recording 360° films can be equate with the production of films in the early cinema period (beginning of 20th century), where movie makers struggled with transporting heavy camera equipment. This caused a number of restrictions. Similarly, Ladybug camera does not allow a number of experiments because of the use of a laptop that constantly captures data coming from the camera (approx. 2-3 GB/min). More flexible recordings were possible due to a number of technological advances in camera technology, so the cameras became smaller and more portable. This is also a route for further developments of devices that can record 360° video.

The creation of the 360° film Discover Krakow offers a number of possibilities for further development, both for a panoramic viewer and an immersive screen. Individual frames presented in Figure 6, taken from the edited 360° film, are appropriate for the panoramic viewer, iDome and a fulldome projection system. In order to prepare video files for the multi-projector presentation in a 360° cylindrical image environment, they have to be cropped to the cylindrical format and by doing this the vertical field of view (VFOV) of images has to be changed (7).

The visual part of the film had to be enriched with a voice-over that was recorded in a studio. The audio recording provides the explanations of objects of interests and presents legends related to specific places in the city. This is a third-person narration of the omniscient narrator and as the narrative progresses from the beginning to the end, the voice-over seems to unfold stories, and also possible alternatives, before the observers. The spectators located in the centre of a panoramic screen could, in fact, feel that they are situated in a horse carriage and they are on a journey through the city.

The following section focuses on the conversion of a linear 360° video recorded with a 360° video camera to an interactive narrative which has the potential to be used in schools or as a training material to groups that manage the city.

360° interactive narrative for education

Interactive storytelling has an opportunity to become a significant support in education in the 21st century. Cinema is already more than 100 years old, computer games are increasingly common, and there is a need for interactive solutions, especially in schools that will encourage students in the new exploration of subjects.

Interactive storytelling is an important cross-disciplinary area for research and entertainment [27]. It is also an ambitious form of art because it links interactivity, visual art and traditional narrative [28] which is a crucial element in a new approach to teaching methods. Del Favero, Kenderdine and Shaw have developed a number of approaches to interactive storytelling via the use of immersive environments (e.g. co-evolutionary narratives), however little attention has been paid to educational application of 360° films. Ozkeskin and Tunc [29] have already presented an inspiring approach to teaching students about the risk of earthquakes and the 360° video of Krakow was used for this purpose (the authors do not mention the source of the video of Krakow and New York in their article) in order to evaluate the risk for people during unexpected disaster. Ozkeskin and Tunc's study would have been far more useful if the author had considered an interactive storytelling approach to understanding potential earthquakes in different locations.

Ryan [30], in the theory of story graphs, does not fully examine the inclusion of additional narratives that are only available during the specific times of the narrative journeys. This paper represents the concept of object-based panoramic interactive narrative. As shown in 8 the line between points A and B consists of a linear narrative (this is a 360° video that is played between two still panoramas: A and B). This paradigm is based on a number of objects situated on the way from A to B. Circles in 8 indicate the distance to the specific points (points of interest; stories) in the city. These points appear only if they are visible from the point of view (C or D) of spherical camera and contain reference numbers (E, F, H, I) which change their position on the screen because of the movement between points A to B. Narrative G is not visible and not available from points of view (C or D), but is in a range when the camera is on the way between points C and D.

Figure The story graph of object-based panoramic interactive narrative

This type of narrative can be a part of a map-based panoramic interactive narrative described in [25], where video panoramas are used to move from one point in the city to another. Here, in the concept defined in this paper, such panoramic video is enhanced with more detailed non-linear narratives that are orbiting around the line A-B and appear only in the specific points and for a specific time. For instance, in the case of Discover Krakow additional narrative about the trumpeter from the tower of St.Marys Church is available to watch (also as 360° film) when the church is visible in the film. Most viewers wanted to watch the main journey from the beginning to the end, returning to the additional narratives at the end of the presentation.

The exploration of immersive environments towards educational purposes

The Cyclorama in Vision3D lab (Montreal) covered the right part of the field of view of the observer. The screen was at the head and sitting (on the ground) was a recommended method of being immersed in this type of projection. The observers were watching panoramic content at the appropriate angle. Comparing the screen in Montreal to Arena360 or ICCI360 a few issues are noticeable. The position of the 360° screen was much above heads of the spectators and instead of looking in front as in Cyclorama (Figure 9), the audience within screens installed in Plymouth had to watch the content with their heads looking up (Figure 10). What is more, Arena360 and ICCI360 had dividing bars and the audience could see five separate screens instead of one seamless screen as the one in Montreal. The use of dividing bars does not present seamless content and the immersiveness is limited.

Figure The relation of spectators position to the Cyclorama in Vision3D lab in Montreal.

The researchers website www.360stories.net - presents five videos of the discussed immersive environments. Four videos were recorded inside these image spaces using Ladybug2 or Ladybug3 camera. Only one video was recorded using normal camcorder and presents Discover Krakow projected on iDome at City University of Hong Kong (11).

Figure The relation of spectators position to the Arena360 in Plymouth

These are the interactive media:

Cyclorama (Montreal):

Media 1: http://360stories.net/krakow-cyclorama

Arena 360 (Plymouth):

Media 2: http://360stories.net/krakow

Immersive Vision Theater (Plymouth)

Media 3: http://360stories.net/krakow-ivt

ICCI360 (Plymouth)

Media 4: http://360stories.net/krakow-icci360

iDome (Hong Kong)

Media 5: http://360stories.net/krakow-idome

Figure Presentation of Discover Krakow on iDome to group of students in Hong Kong

The fully rendered movie about Krakow was demonstrated to the students at the University of Plymouth in Arena360 (Figure 10). The time of journey was shortened from about 45 minutes to 13 minutes and 46 seconds by changing the frame rate in video editing software. A questionnaire was provided to these students in order to receive feedback about their experience.

Kenderdine et. al. [31] proposed a method of evaluating specific qualities of the experience of spectators of the The PLACE-Hampi project and offered a new strategy for evaluation of immersive environments. The PLACE-Hampi - project uses only a part of 360° screen, where one projector mounted on a motorized platform is used, whereas in the researchers approach full 360° display system is filled with not only panoramic images, but mainly panoramic video. The researcher in his questionnaire was interested in the potential of 360° films and user's reactions to a virtual travel to a new place, rather than in the reactions of the body, awareness of other visitors within 360° space or dwelling time as asked by creators of the PLACE-Hampi.

The feedback from students (who were invited to Arena360) suggests that the video panorama kept their interest as 74% (14 of 19) of participants were satisfied with the length of 360-degree story (Figure 76 - question 21). There were 19 students who filled in this questionnaire. Moreover, students agreed that such a presentation is an appropriate way for creating a tourist guide (17/19 = 89%) encouraging them to visit the presented location. What is surprising, is that this audience (1st year students) needed no additional information at the beginning of the presentation (15/19 = 78%). Additional feedback was also provided by students in the last part of the questionnaire. These are some of students comments:

• screen height not fully immersed most of the students compared the experience to the one in Immersive Vision Theatre; where the screen covers a large part of the field of view;

• cold a heating system was not installed in the Arena360, which took place in early February;

• contextual information needed students proposed the potential for adding extra information and description;

• music pessimistic/depressing half of the students did not prefer this type of background music for the travel genre presentation.

The next event (ICCI360 Festival in September 2010) improved some issues of the presentation, but the screen (much bigger than in Arena360) remained in the same position in relation to the audience (they have to look up to see panoramic works).

The recording of the audience while they are watching 360° content using spherical video camera provides additional information which is worth further examination (the direction of looking or their interest in particular parts of the presentation). There are a few conclusions after watching this panoramic narrative on a 360° screen:

• there is no singular direction of observation in the circular screen; when the movement is presented it took some members of the audience (especially for those who were far from the centre) about 1-2 minutes to realize what is demonstrated and where to look to follow the narrative journey;

• long gaps between speeches where no information was provided are also noticeable; the audience starts to explore other parts of the content projected, not focusing in one direction (mainly the direction of the movement);

• a great part of the audience were not aware of the possibility to move during the presentation and stayed for most of the time in the same place for the period of the display.

However, these findings are limited to a 360° screen (Arena360 was very similar construction to ICCI360). There are numerous factors that need to be taken into account when exploring iDome and IVT because of different approaches to presenting 360° stories. The CAVE environments could be the next environment for testing the 360° film.

A number of film makers and also panorama photographers provided the cutting-edge work for the Arena360 and the ICCI360 events and their work included: animations, movies, still photographs or time-lapse photography. Only a few artists created their work in the correct format where the potential of the 360° screen was fully explored, while most film makers used 5 screens to present the same content (or slightly changed) on these 5 screens. Interactive versions of their work are published on the researchers websites.

This illustrates that observers are still hypnotised by the rectangular screen and it is still a challenge to create a content using 360° video camera, because a completely new language of immersive cinema has to be difined. Neafus [32] is working on that description of a new rules for immersive cinema, because the production of immersive films is no longer defined in frames as described by Arjon [33] or Messaris [34].

Immersive Vision Theater allowed the audience to sit and admire the narrative about Krakow, while other environments do not seat the audience. Here the attachment to a chair is evident, however no longer it is the attachment to a rectangular screen. One of the aims of immersive environments was to encourage the collaboration between users or students gathered in front or inside the environment in order to motivate them for further investigation. Standing position in immersive environments brings new opportunities for education. IVT does not provide the opportunity for spectators to contact each other and change positions which is one of the main tasks of collaborative watching of the interactive narratives presented using panoramic video.

iDome, that the researcher had an opportunity to test in October 2010 is a much different environment than the screens discussed so far. This 3m diameter projection hemisphere enables a small group of viewers not only to experience the immersiveness, but also to interact with the panoramic video. This interaction is created for example by the use of a navigable ball situated in front of the screen. When a person from the audience notices something important he or she can change the field of view. This means that all users will see the result of his/her action, while within 360° screens such a problem does not exist because every spectator can watch different part of 360° film presented on a 360° screen or a fulldome screen.

If the researcher (provided with an adequate budget) was given a task to create the immersive screen for panoramic works that would be of assistance for teachers and lecturers, he would select the following features from the cylindrical screens, which have been visited and tested.

iDome the size and the shape of the screen that occupies large amount of the observers field of view and also the precision of the display;

Cyclorama (Montreal) design of the structure (it took about a few hours to dismount it and a similar time to mount it in another location; free software (LightTwist) is an advantage;

Arena360 social aspect of watching panoramic movies together; presentation of the work not only created by one person or an institute but with the international artists contribution in the form of time-lapse photography, animations and films created with different 360° cameras.

The researchers visits to these panoramic screens have facilitated the following conclusions about the generation of an idealized immersive screen based on a digital projection. The findings from this study have a number of significant implications for future practice. The target screen should have:

• a curved shape;

• a significant height;

• a surround system;

• the potential of interaction with the content displayed;

• ability to provide special effects (changing temperature, wind, water) which are good tools for getting into the spectators and sucking them into the image space;

• calibration techniques for improving the misalignments in projections;

• a seamless structure (no divisions between screens).

It has been already indicated that the budget is the limitation for the creation of such an idealized immersive projection surface, especially for educational projects.

Conclusion

Returning to the hypotheses posed at the beginning of this paper, it is now possible to state that object-based panoramic interactive narrative is a new approach for motivating students for further explorations about the presented topic using immersive environments.

To sum up, the researcher projected his 360° film Discover Krakow on 5 immersive environments on three continents, mostly to the spectators that have never visited the Polish city. Almost all viewers were interested in going to this place to explore the site on their own. This study has shown that 360° film was not a substitute to their travel.

Large field of view and interaction with the content provided in order to create unique interactive narratives can enhance learning processes because the spectators participate in collaborative group-based learning environments delivered over immersive environments.

References and Notes

1. ^M. Dunleavy, C. Dede, in Handbook of Research on Educational Communications and Technology. 4th edition, Volume 2, M.J. Bishop, J. Elen, Eds. (Macmillan, New York, in press)
2. ^A. Kamarainen, S. Metcalf, T. Grotzer, A. Browne, D. Mazzuca, M.S. Tutwiler, C. Dede, EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips. Computers and Education (in review)
3. ^L. Garner, M. Gallo, M, Field trips and their effects on student achievement and attitudes: a comparison of physical versus virtual field trips to the Indian river lagoon. Journal of College Science Teaching , 34(5), 14-17 (2005).