Augmented reality (AR) provides an extra channel of computer-generated sensory input into the perception of the user, augmenting the physical sensory input using sound, video, graphics or GPS data. By contrast, “virtual reality replaces the real world with a simulated one” [1, 2]. According to Azuma [3], an AR system combines real and computer-generated information in a real environment, interactively and in real time, and aligns virtual objects with physical ones. “AR is a subfield of the broader concept of Mixed Reality (MR)” [4], which combines simulations predominantly taking place in both virtual domain with the objects or locations in the real world. Augmentation in AR is performed in real-time and in semantic context with environmental elements, referring to the scores in sports competitions or the identity of the people as seen on TV. The Columbia Touring Machine [5] was the first example of Mobile AR Systems (MARS) with a head mounted display. Using AR technology, the user can interact with the surrounding real world, read and manipulate the objects digitally. Artificial information about the environment and 3D objects can be overlaid on the real world [6] as demonstrated in some medical [7] and military applications [8].
MARS present a dynamic way for people to interact with computers and digital information. Main advantages of MARS are the portable nature of handheld devices and built-in cameras, while the only perceivable disadvantage is the physical constraints of the user such as having to hold the handheld device out in front of them. Therefore, MARS offer a novel way for interaction which is radically different from static desktop computing, and promise to be the first commercial success for AR technologies.
However, the main challenge in MARS design is to provide a generic framework flexible enough to allow rapid development of applications to be used by different types of people. Especially for MARS to promote independent learning, the creation of learning scenarios should be completely independent of coding. The scenario production challenge in training simulations applies to the military to a great degree, however, it is even more pronounced in the civilian emergency management community where there is a lack of organizational uniformity, standardized tactics, techniques and procedures. For example, Elms et al. [9], addressing the collaborative team training challenge, developed and tested the Emergency Management Exercise System (EMES), a computer-based training tool for formal emergency management training in both civilian and military (U.S. Air Force) venues.
The obvious advantage of a generic framework for MARS would be its applicability to various domains. VR and AR training is used by many organizations, such as emergency services, mining companies and armed forces. For example, the US Army uses them as a complement to more traditional training materials because they accelerate learning, stimulate interest, and communicate better than text [10]. Examples can be seen in Generic Virtual Training (GVT) in France by Gerbaud et al. [11] and in The Infantry Immersive Trainer (IIT) in US by Dean et al. [12]. IIT is a AR system designed to extend training capabilities for US Marines across a wide range of military operations. These VR and AR systems have been shown to be effective for training, and their distributed game-based architectures contribute an added benefit of wide accessibility. As stated by Cameron [8], “In military applications, AR can serve as a networked communication system that renders useful battlefield data onto a display system such as soldier’s goggles in real time.”
A generic framework will have capabilities for reconfiguring scenarios with various databases so as to produce meaningful exercises for different applications. Thus, it can be used for a variety of training venues and e-learning. The lack of generic frameworks to design and manage the content for the training scenarios is the biggest problem in MARS design. Tan et al. [13] presented the 5R adaptation framework for a location-based mobile learning system, which provides a standard structure for adaptive mobile learning systems. This framework takes learner, location, time, and mobile device into the learning content generation process and implements a wide-range adaptation in the mobile learning environment. However, its adaptability for MARS is limited and it does not address the scenario building challenge.
Our aim in this paper is to investigate the development models for MARS, propose a generic people-centric framework (4Any framework) for the next generation HCI paradigm, to develop a pilot system and to test its usability with a number of field studies. In this paper, we demonstrate the potential of the framework in terms of flexibility of system architecture and usability in recording landmarks in a number of heritage sites, as well as how 4Any framework has been applied in an ongoing project ArcHIVE 4Any using a number of field studies in Chalon sur Saone, France, Anzac Cove, Gallipoli, Turkey, and Sydney, Australia. ArcHIVE 4Any was implemented by using Django framework. ArcHIVE 4Any uses GPS location, accelerometer and gyroscope of smartphones, tablets and other mobile devices. It facilitates the communication between the user and a heritage site offering a CyberGuide at specific GPS locations. In other words, it allows users to see relevant digital information of landmarks in various GPS locations during their visit, such as text, image and video.
Although CyberGuide in ArHIVE 4 Any application is a demonstration of the flexibility and extensibility of the framework, it does not have scenario engineering capabilities. We plan to add this module in future versions of the system. Therefore, the focus of this paper is the system architecture, rather than scenario engineering. Scenario engineering for each user will be performed after collecting data in the usability studies, regarding the personality profiles and navigational routes followed by each participant in two tasks: navigation in digital heritage and surveying (marking) digital heritage locations to deliver information. These tasks are addressed for two different types of users: learner (citizen) and instructor (surveyor).
Our idea is that citizens in a cultural context can be treated as curators, and their navigation in heritage collections conveys narratives. Our long-term aims are: (1) Navigation Analysis in the exploration of cultural heritage in VR and AR, (2) Narrative Analysis to compare the temporal and spatial parameters of navigational patterns in VR and AR, (3) Analysis of User Experience to measure learning outcomes as well as the impact of autonomy in navigation, (4) Model Development to obtain the optimum user experience and learning outcomes. The first three aims will inform the 4th to obtain the optimum user experience through alternating the temporal and spatial parameters of the medium such as AR and VR, as they present different temporal and spatial parameters to measure differences in presence, transportation, motivation, autonomy, and more importantly, user navigation associated with narratives. However, the scope of this paper is limited with Mobile AR and we leave the debate of temporal and spatial parameters outside the scope of this paper. It is important to note that we have developed the framework to address the needs for a more general purpose, targeting scenario engineering.
In this paper we propose to use technical tools such as ArcHIVE 4 Any application to draw conclusions from collected large volume of data for the creation of narrative. We propose to overlay geometrical, geographical, and navigational information, as well as user’s personality and map these to segmented parts of each associated heritage object to understand the relationships between navigational patterns, heritage objects, and user cognition. The 4Any framework contributes to the knowledge by taking into account multiple users and collecting different routes to each user in this version. In future versions, using this information, we will present a model offering different routes to each user, keeping digital narrative completely independent of coding and leaving building the database task to the users through crowdsourcing.
In the remaining sections of this paper, we will first review location-based mobile systems, discuss development of MARS, and people-centric computing paradigm. Then, we will introduce A People-Centric Framework (4-Any) for Mobile Augmented Reality Systems.