A people-centric framework for mobile augmented reality systems (MARS) design: ArcHIVE 4Any

Abstract

This paper introduces a generic people-centric mobile augmented reality systems (MARS) framework to leverage the application system design and implementation. Proposed 4Any framework could be used as a meta architecture to guide the development of MARS. The main challenge in MARS design is providing a generic framework flexible enough to allow rapid development of applications to be used by different types of people. The paper demonstrates how 4Any framework has been applied in an ongoing project ArcHIVE 4Any using two field studies in Chalon sur Saone, France and Anzac Cove, Gallipoli, Turkey. ArcHIVE 4Any was implemented by using Django framework. The goal of this paper is to demonstrate the potential of the framework in terms of flexibility of system architecture and usability in recording landmarks in a number of heritage sites. In 4Any Framework we assume learning has to occur anywhere, anyway, anytime for anybody. The actors in this framework are specified as Instructor, Learner, Device and Programmer. Each actor initiates an action with respective entities Information, Media, Location and Time. What mediates each action is the metadata layer between the entities and people profiles using the same system. Metadata layer provides mapping between the underlying structures and definitions of associated content between the actors and entities. This way, each user sees a different face of the same system. The 4Any system architecture separates the system into four modules: Instruction, Learning, Device and Coding. There are 2 main sides of this framework to support learning anytime, anywhere, anyway by anybody: Pedagogical and Technological. The framework is composed of four layers that correspond to each row: Actors, Profiles, Metadata to be shared, and Scenario. With the rapid development of mobile technologies, new features of mobile devices will become available. The usability test results with ten Instructor profiles suggest that 4Any framework will be able serve as a vehicle for people-centric computing, to be used by anybody, anywhere, anyway, and anytime. Thus, the paper contributes to the knowledge in Human Centric Computing by taking into account multiple users and presenting different faces to each user demonstrating an example of the People Centric Computing paradigm.

Background

Augmented reality (AR) provides an extra channel of computer-generated sensory input into the perception of the user, augmenting the physical sensory input using sound, video, graphics or GPS data. By contrast, “virtual reality replaces the real world with a simulated one” [1, 2]. According to Azuma [3], an AR system combines real and computer-generated information in a real environment, interactively and in real time, and aligns virtual objects with physical ones. “AR is a subfield of the broader concept of Mixed Reality (MR)” [4], which combines simulations predominantly taking place in both virtual domain with the objects or locations in the real world. Augmentation in AR is performed in real-time and in semantic context with environmental elements, referring to the scores in sports competitions or the identity of the people as seen on TV. The Columbia Touring Machine [5] was the first example of Mobile AR Systems (MARS) with a head mounted display. Using AR technology, the user can interact with the surrounding real world, read and manipulate the objects digitally. Artificial information about the environment and 3D objects can be overlaid on the real world [6] as demonstrated in some medical [7] and military applications [8].

MARS present a dynamic way for people to interact with computers and digital information. Main advantages of MARS are the portable nature of handheld devices and built-in cameras, while the only perceivable disadvantage is the physical constraints of the user such as having to hold the handheld device out in front of them. Therefore, MARS offer a novel way for interaction which is radically different from static desktop computing, and promise to be the first commercial success for AR technologies.

However, the main challenge in MARS design is to provide a generic framework flexible enough to allow rapid development of applications to be used by different types of people. Especially for MARS to promote independent learning, the creation of learning scenarios should be completely independent of coding. The scenario production challenge in training simulations applies to the military to a great degree, however, it is even more pronounced in the civilian emergency management community where there is a lack of organizational uniformity, standardized tactics, techniques and procedures. For example, Elms et al. [9], addressing the collaborative team training challenge, developed and tested the Emergency Management Exercise System (EMES), a computer-based training tool for formal emergency management training in both civilian and military (U.S. Air Force) venues.

The obvious advantage of a generic framework for MARS would be its applicability to various domains. VR and AR training is used by many organizations, such as emergency services, mining companies and armed forces. For example, the US Army uses them as a complement to more traditional training materials because they accelerate learning, stimulate interest, and communicate better than text [10]. Examples can be seen in Generic Virtual Training (GVT) in France by Gerbaud et al. [11] and in The Infantry Immersive Trainer (IIT) in US by Dean et al. [12]. IIT is a AR system designed to extend training capabilities for US Marines across a wide range of military operations. These VR and AR systems have been shown to be effective for training, and their distributed game-based architectures contribute an added benefit of wide accessibility. As stated by Cameron [8], “In military applications, AR can serve as a networked communication system that renders useful battlefield data onto a display system such as soldier’s goggles in real time.”

A generic framework will have capabilities for reconfiguring scenarios with various databases so as to produce meaningful exercises for different applications. Thus, it can be used for a variety of training venues and e-learning. The lack of generic frameworks to design and manage the content for the training scenarios is the biggest problem in MARS design. Tan et al. [13] presented the 5R adaptation framework for a location-based mobile learning system, which provides a standard structure for adaptive mobile learning systems. This framework takes learner, location, time, and mobile device into the learning content generation process and implements a wide-range adaptation in the mobile learning environment. However, its adaptability for MARS is limited and it does not address the scenario building challenge.

Our aim in this paper is to investigate the development models for MARS, propose a generic people-centric framework (4Any framework) for the next generation HCI paradigm, to develop a pilot system and to test its usability with a number of field studies. In this paper, we demonstrate the potential of the framework in terms of flexibility of system architecture and usability in recording landmarks in a number of heritage sites, as well as how 4Any framework has been applied in an ongoing project ArcHIVE 4Any using a number of field studies in Chalon sur Saone, France, Anzac Cove, Gallipoli, Turkey, and Sydney, Australia. ArcHIVE 4Any was implemented by using Django framework. ArcHIVE 4Any uses GPS location, accelerometer and gyroscope of smartphones, tablets and other mobile devices. It facilitates the communication between the user and a heritage site offering a CyberGuide at specific GPS locations. In other words, it allows users to see relevant digital information of landmarks in various GPS locations during their visit, such as text, image and video.

Although CyberGuide in ArHIVE 4 Any application is a demonstration of the flexibility and extensibility of the framework, it does not have scenario engineering capabilities. We plan to add this module in future versions of the system. Therefore, the focus of this paper is the system architecture, rather than scenario engineering. Scenario engineering for each user will be performed after collecting data in the usability studies, regarding the personality profiles and navigational routes followed by each participant in two tasks: navigation in digital heritage and surveying (marking) digital heritage locations to deliver information. These tasks are addressed for two different types of users: learner (citizen) and instructor (surveyor).

Our idea is that citizens in a cultural context can be treated as curators, and their navigation in heritage collections conveys narratives. Our long-term aims are: (1) Navigation Analysis in the exploration of cultural heritage in VR and AR, (2) Narrative Analysis to compare the temporal and spatial parameters of navigational patterns in VR and AR, (3) Analysis of User Experience to measure learning outcomes as well as the impact of autonomy in navigation, (4) Model Development to obtain the optimum user experience and learning outcomes. The first three aims will inform the 4th to obtain the optimum user experience through alternating the temporal and spatial parameters of the medium such as AR and VR, as they present different temporal and spatial parameters to measure differences in presence, transportation, motivation, autonomy, and more importantly, user navigation associated with narratives. However, the scope of this paper is limited with Mobile AR and we leave the debate of temporal and spatial parameters outside the scope of this paper. It is important to note that we have developed the framework to address the needs for a more general purpose, targeting scenario engineering.

In this paper we propose to use technical tools such as ArcHIVE 4 Any application to draw conclusions from collected large volume of data for the creation of narrative. We propose to overlay geometrical, geographical, and navigational information, as well as user’s personality and map these to segmented parts of each associated heritage object to understand the relationships between navigational patterns, heritage objects, and user cognition. The 4Any framework contributes to the knowledge by taking into account multiple users and collecting different routes to each user in this version. In future versions, using this information, we will present a model offering different routes to each user, keeping digital narrative completely independent of coding and leaving building the database task to the users through crowdsourcing.

In the remaining sections of this paper, we will first review location-based mobile systems, discuss development of MARS, and people-centric computing paradigm. Then, we will introduce A People-Centric Framework (4-Any) for Mobile Augmented Reality Systems.

Location-based mobile systems

MARS also differ substantially from other styles of computer interaction, such as VR, even though it may share some traits with it. As stated by Hollerer [14], “commercial location-aware systems have explored the utility of a variety of coarse position-tracking approaches, ranging from monitoring infrared signals, to getting location information from the nearest wireless phone base stations, in order to provide local weather and traffic updates, or tourist information.” One of the most important aspects of mobile devices is their potential to support location-aware or location-based computing [15] that opens up new possibilities in the way we interact with computers and gather information. “Location-based mobile systems have taken the advantage of the mobile devices to enhance learner’s interaction with the learning context” [13].

There are three applications that are noteworthy regarding location-based mobile systems: One of these is iWalk-NorthCornwall [16]. It is devised 133 walks mostly in North and Mid Cornwall, ranging from 2–10 miles, totalling 618 miles. Each has a map of the route, detailed directions, photos and some information about the area. They are classified by length, steepness grade, type of scenery (coastal, moorland, riverside etc.) and amenities nearby. The walks are free to browse on the web and print out for your own use. These walks are available as guided walk mobile apps for Android or Apple IOS platforms.

The other is the i-Tour project a finalist for the Smart City Prize at SMAU held on 12th December 2013 in Naples. SMAU is divided into different sections, each of them targeted to a specific audience. Smart City section is dedicated to mayors, councillors and officers of municipalities and representatives of the public administration. The goal of i-Tour [17] is to develop an open framework to be used by different providers, authorities and citizens to provide intelligent multi-modal mobility services. i-Tour client is expected to support and suggest the use of different forms of transport (bus, car, railroad, tram, etc.) taking into account user preferences as well as real-time information on road conditions, weather, public transport network condition. i-Tour utilises a recommender system based on the information provided by the whole user community and encourages sustainable travel choices by providing rewarding mechanisms for users choosing public travel options. i-Tour mobility client applications feature a user-friendly interface accessible from PCs, PDAs and Smartphones. The i-Tour project is now halfway through and some important objectives have been achieved such as public transport estimation load, modeling of the multi-modal transport system, development of the serious game interface, an analysis of the privacy threats. Low-level services that support transport data access have been implemented.

Another example sharing same developmental principles with Archiev4Any is Historypin [18], a digital, user-generated archive of historical photos, videos, and audio recordings. In Historypin, users are able to use the location and date of their content to “pin” it to Google Maps; where Google Street View is available, they can overlay historical photographs and compare it with the contemporary location. This content can be added and explored online and via a series of Smartphone applications. The website has over 200,000 assets and recollections “pinned” to the Historypin map around the world, with higher contributions in the UK, USA and Australia. The project was created with funding from Google and was launched at the Museum of the City of New York in July 2011. Historypin is very much focused on community engagement using imagery to get people connecting through stories and locations.

Development of MARS

Although the crowdsourcing idea to create a digital heritage is the same as Historypin, ArcHIVE 4Any theoretically focuses on learning from collected information in the database for the customised creation and presentation of digital narrative to each user to maximise learning outcomes. We cannot treat any research domain in IT in isolation from the involvement of other disciplines. The development of an AR Scenario requires interdisciplinary collaboration, such as integration of research studies in Arts, Science, or History-related content, Media and Computing with Theories of Narrative. The difficulty for this integration lies in an apparent opposition between traditional narrative and interaction: a traditional narrative involves an ordered sequence of events which is immutable, while in an interactive experience, the users create their own story. Adapting stories to the interactive medium requires resynthesis of the story elements to create a generative computational foundation from which a number of possible scenarios can be interactively generated. These scenarios heavily depend on scene dynamics. One of the key characteristics of MARS is that both virtual and physical objects are part of the user interface and they are perceived in the real world. As stated by Hollerer [14] “This raises several issues such as control, scene dynamics, consistency, need for embedded semantic information, and display space.” In this project, our goal is to take some of these parameters and investigate how to design the “embedded semantic information” to work more efficiently with augmented “display space” and “scene dynamics.”

5R Framework [13] considers “learning content” which refers to embedded semantic information, “location” to scene dynamics, as well as “device profile” to display space in Hollerer [14] ’s thesis. There are other generalizable computational models for characterizing the dynamics of unconstrained scenes. System dynamics, for example, deals with internal feedback loops and time delays that affect the behaviour of a complex system. These elements help describe how even seemingly simple systems display baffling nonlinearity. Shroff et al. [19] propose using the theory of chaotic systems to capture scene dynamics. They propose dynamic attributes which can be augmented with spatial attributes of a scene for semantically meaningful categorization of dynamic scenes (the video data). Compiling a dataset of in-the-wild dynamic scenes, they tested the framework and found that it provides the best classification rate among other well-known dynamic modeling techniques. Dynamic attributes include amount of activity in the video (high or low), flow granularity of the structural elements that undergo motion (coarse or fine), and regularity of motion of structural elements (irregular or regular). These two frameworks have common attributes that can be integrated into one. In MARS, we need to establish concrete semantic relationships between virtual and physical objects in order to characterize behaviour of the user interface [14]. In fact, since many virtual objects are designed to annotate the real world, these virtual objects need to store information about the physical objects to which they refer. Only a small portion of the virtual space is visible to the user at a GPS location. “The representation of that portion of augmented space depends on the user’s position, and head-orientation, personal preferences and other interactions with the augmented world” [14]. It is possible to control the scene dynamics with a scenario which is an outline of the interactive plot that demonstrates the pattern of possible events or story possibilities in an interactive narrative.

People-centric computing paradigm

Human computer interaction (HCI) had a user-centric focus in 1980s, and became human-centric in 1990s. The focus of HCI has changed towards actor-centric in 2000s due to the wide use of information systems and became more person or people-centric in the last decade. It no longer serves to only one person but serves people (citizens) including many groups of person. Therefore, while personalisation and personification of an interface is important, making it multi-faceted for the use of many people has been gaining significance.

Abstracting the user experience from the device is a need so that it persists between the user and virtual or real computing platforms, across many devices, such as, smart phones and tablets, google glasses and many more to follow in near future. This could be done using a Digital Persona which may be a special simcard or a chip that carries the Digital ID for the mobile authentication of each person in a device similar to a USB stick. Persona is a term used in Jungian psychology. It refers to a personal façade that one presents to the world. In drama, persona refers to an actor’s portrayal of someone in a play. In February 2014, a Digital Persona of this kind, a mobile Fingerprint scanner that holds the capacity to carry a Mobile ID with a Silicon Fingerprint Sensor was released under the name of Touch CHIP TCS1 sensor [20]. TCET module in this device provides a self-contained biometric matching system for all skin types, and it is compatible with ISO and ANSI standards. It would be interesting to use this device not only for biometric matching, but also for detecting psycho-physiological characteristics of a person. Perhaps, in the next decade, HCI will have a more mind-centric or x-centric focus with the novel ideas to personate our interaction with the technology, and in the following decade this person-ation may create a x-centric focus.

Regarding the future desktops, Microsoft Summit in April 2013 [21] states that due to the popularity of the current developments in ubiquitous computing and mobile applications, we will have to adapt to use a hybrid system in human computer interaction. The hybrid system can be called people-centric as it should have:

• A user-centric approach layer on top of device management.

• Multi-faceted user approach layers underneath (user-owned, user-enabled devices with privacy and security protocols).

4-Any: a people-centric framework for mobile augmented reality systems

The 5R adaptation concept for location-based mobile learning is stated as: at the right time, in the right location, through the right device, providing the right contents to the right learner. Similar to this approach, we have developed 4-Any Framework in which we assume learning has to occur anywhere, anyway, anytime for anybody. The actors in this framework are specified as Instructor, Learner, Device and Programmer. Each actor initiates an action with respective entities Information, Media, Location. Time is considered as another layer with the repetition of these components at another time. What mediates each action is the metadata layer between the entities and people profiles using the same system. Metadata layer provides mapping between the underlying structures and definitions of associated content between the actors and entities. This creates a 4 × 4 matrix as demonstrated in Fig. 1 with Actors and Entities form x and y axes respectively, and Time is the repetition of all in the z axis to represent these events occurring Any Time. The 4Any system architecture separates the modules as 4 main parts that refer to each column: Instruction, Learning, Device and Coding.

Fig. 1

The 4Any Mobile AR System Framework

There are two main sides of this framework to support learning anytime, anywhere, anyway by anybody: Pedagogical and Technological. The framework is composed of 4 layers of Entities that correspond to Actors, Profiles, Metadata to be shared, and Scenario in each row. A Scenario includes a capsule of Information, Media, Location and Code initiated at a specific Time by a specific Actor (see Fig. 2 for the visualisation of a Scenario).

Fig. 2

Scenario layer mediating Information, Media, Location and Code in Figs. 2, 3 as experienced by Learner and Instructor: ArcHIVE hotspots and virtual walkthrough in Chalon sur Saone using the MARS system

This framework presents an opportunity for the design component of the Scenario independently at any point in time. Information, media, location and code can be updated independently any time. Each specific actor that can be generalised as anybody has access to the respective entity at any location (anywhere), using any media (anyway), anytime. Time is controlled and regulated by updates anytime by anybody which happens again anyway and anywhere. Thus, the 4Any framework provides a simple flexible architecture for people-centric computing paradigm.

ArcHIVE 4Any system architecture

We developed a pilot system (ArcHIVE 4Any) to test the 4Any framework. A server based design approach was used for the application for ease in publishing to many platforms at once and centralizing for immediate deployment of updates. As seen in Fig. 3, ArcHIVE 4Any system architecture provides a Metadata layer to retrieve, deploy, create, update or delete customization information, such as custom object definitions and page layouts which is tailored to the user profile defined by the Actor layer.

Fig. 3

ArcHIVE 4Any Version1 (server-based) system architecture using the 4Any Mobile AR System framework

Metadata layer consists of a number of APIs. These APIs interact in a way similar to the Google Metadata API concept. Through the use of the HTML5 markup, javascript and libraries such as JQuery Mobile, the system can access the smartphone and tablet sensors (GPS, camera and microphone) needed to crowdsource information and media which sits on the pedagogical side of the 4Any framework in Fig. 1. Conversion tools such as PhoneGap can allow compilation of the application to run natively on the application if this is needed. PhoneGap is a free and open source framework that allows you to create mobile apps using standardized web APIs for various platforms such as iOS, Android, Blackberry 10, Firefox, and Windows 8. Dashed lines in Fig. 3 represent no direct access to Metadata layer by the Instructor and Learner Profiles. The access to Metadata layer is through Device and Programmer profiles. Instructor and Learner have access to Information and Media only through Scenario layer utilising the major access line in the metadata layer designed by the programmer. Device records the location information.

We will introduce each box stated as Information, Media, and Location in a separate figure in the remaining section of this paper.

Code

Code encapsulates Information, Media, and Location (Positioning) data as described in the 4Any framework. A Model View Controller paradigm provided by the Django 1.6 framework was implemented. Django uses the python scripting language to quickly provide a web/database interface that is well supported, scalable, and easy to maintain. The application interfaces with a MySQL database at the moment, which was chosen for ease and speed of setup for the pilot. However, in production this would be changed to PostGIS as it provides extra positional functionality which may prove more accurate and convenient and is well supported by third party applications, allowing for easy expansion of map creation features. Retro changing of the database is relatively easy with this setup.

Location (positioning)

The coordinates are obtained as latitude and longitude from the device interpreted through the browser using the Bing Maps API. One can expect Android and iOS devices to be displaced by around 20 m in accuracy, and because of this, the web application has an interface that allows repositioning in retrospect using an aerial view. Location concept can be visualised in Fig. 4.

Fig. 4

Location box in Figs. 2, 3 as experienced by Instructor: Attributing your current location to a relic on an iPhone using Safari

Media

Images may be uploaded for each relic, and can be many megabytes in size. This resolution may be needed when examining a relic, but is not needed when displaying all the relics. Having a smaller version of the image file is essential for a responsive web page. Sorl Thumbnail (a Django application) was used to quickly generate thumbnails from the uploaded images into the relic list page, allowing for a list of images to be shown that links to a high resolution image when selected. Video, Audio and shared URLs to hosting sites such as Vimeo and YouTube can be stored with each relic. Media concept can be visualised in Fig. 5.

Fig. 5

Media box in Figs. 2, 3 as experienced by Instructor: Uploading a photo for a relic from an iPhone using Safari

Information

We used relational tables in this application. The focus of the design is the relic table, which is related to four other tables, namely the historical site with a bounding area, the period with start and end dates of the heritage item, the archeologist who entered/found the relic and a relic type with a name and description. Other attributes can be added to any of these tables, for example the relic type may have. Information concept can be visualised in Fig. 6.

Fig. 6

Information Box in Figs. 2, 3 as experienced by Learner and Instructor: The landing page in Chrome on MacBook Pro

For a location-based scenario, assuming Information in the Scenario Layer holds the values that differentiate between different modelling scenarios to track values for actual actions and planned actions, we can use simple data models linked with a primary key “Location”, listing “Actions” and “Action-Plan” at a certain GPS location as fact tables. The link from the “Actions” table to the location “Location” is the usual link from the fact table to the primary key of the “Location” table. “Location” has a granularity attribute “Coordinates in 10 m” on a GIS map and so do the other tables. While the “Actions” table is linked to the “Coordinates in 10 m”, we link the “Action-Plan” table to the “Coordinates in 10 m” attribute and define the proper key mappings for that regarding Media and Code (Real and Virtual object models), executing the processes in “Action-Plan” of Information unit to suggest other sites to visit and checking the dimensions of the augmented space.

Archive 4-Any implementation

The system is located at: http://www.archive.science.mq.edu.au/archives/.

The development was created in 20 different locations. Detailed mappings were added for 4 major pilot sites. The pilot usability testing has been conducted with 10 surveyors in 4 sites to improve the system, however we need further testing with learners. Therefore, we created an online survey for the web application (ArcHIVE 4Any) using Django 1.6 framework. Online survey has the same Model View Controller paradigm and interfaces with a MySQL database. It has been lodged on http://www.archive.science.mq.edu.au/survey/. Here we provide technical details of Online Survey implemented to explain the system.

Model

Survey has several categories and each category has a number of questions. Figure 7 demonstrates the Online Survey Model. Each Question types can be varied: text, select (unique answer), radio, integer (rating), and checkbox (multiple answers). The choices are separated by “,” while entering data, the function get_choices() is provided to retrieve them into a list.

Fig. 7

MySQL database design for the usability SURVEY

Response table contains relevant information of participants. The focus design is the AnswerBase table, which is related to Question table and Response table. It has subclasses of different type of answers.

MySQL request templates and Excel templates calculate results of personality and satisfaction questions. Export MySQL request result and copy it to corresponding Excel templates, scores of personality or satisfaction would be calculated automatically. Then, we could utilise the analysis plan below to draw conclusions.

Controller

Controller is the bridge between Model and View, which can manipulate models and update views. We have created a form from Response model. The form receives initially the survey id and displays related questions in order. These questions are displayed according to their type: If it is a selective question, a radio question or a checkbox question, the choices will be displayed under the question. If it is a text question, a text field will display. A required message will appear if a mandatory question has not been answered. A function “save” has been created for saving the answers into right data model with Response information.

View

In the case of Django framework, View is the Python callback function for a particular URL. It puts the form we created into a variable, checks its validity, saves it and defines the page to redirect.

Template

Templates are html pages which the users finally see. “{% %}” is used to define a dynamic block and “{{}}” to add a dynamic content. The variables defined in View of same URL can be used directly in Template. For example, if you defined a variable X = “welcome” in View, you can put {{X}} into Template, then users will see the string “welcome” just like a normal html page.

Admin

One of the advantages of using Django framework is that one can customise the admin page. This page allows administrator to manage the database easily without any knowledge of MySQL command. The database can be administered by any instructor (surveyor) through the standard Django administrative interface seen in Fig. 8 and accessed through the following link: http://www.archive.science.mq.edu.au/admin/.

Fig. 8

Information Box in Figs. 2, 3 as experienced by Instructor and programmer: ArcHIVE Database

Application storyboard

The landing page for the ArcHIVE 4Any application presents you with the historical sites available. Clicking on the black “Show Location” button will display the site’s location on the map (see Fig. 6). Selecting the Anzac Cove historical site provides a scrolling list of the relics found at that site, and when you click on the relic’s black button, the map is centered on the relic, and a pointer to the relic with its thumbnail image is displayed (as seen in Fig. 9). Clicking on the description of the relic allows you to edit it and move its position. The example in Fig. 10 shows the relic being repositioned on the tip of the pyramid with the relic identified by a house icon and the surrounding relics represented by orange dots. The house image can be dragged to its correct location. To add a relic you hit the big orange “Add Relic” button when listing the relics for an historical site. You are given a map with the positions of the surrounding relics, you can get your current location, and you can drag the house icon (bottom left corner of icon to drag) to the exact position of the relic in the same way as the edit page shown in Fig. 6.

Fig. 9

Information Box in Figs. 2, 3 as experienced by Learner: Map centered on Ariburnu with thumbnail and pointer

Fig. 10

Location box in Figs. 2, 3 as experienced by Instructor: Repositioning relic on tip of pyramid

Thus, in a specific GPS location, any instructor/learner can use their mobile device to enter relics. On the “Add Relic” page, hitting the “Get My Location” button will position the house icon at your current location, which you can then drag to the correct position of the relic. Figure 4 shows the iPhone running the application which is requesting access to the GPS. You can then take a photo to attribute to the relic. Figure 5 shows the web page accessing an iPhone’s camera.

Usability testing and field studies with instructors

As a crowdsourcing tool, ArcHIVE 4Any is open to public who have a mobile device with Internet connection (smartphone, tablet, etc.). The web server is located at the following link: http://www.archive.science.mq.edu.au/archives/.

We tested the development in 20 different locations and developed detailed mappings for 4 major pilot sites with 10 instructors: first one in Anzac Cove, Gallipoli, Turkey located at http://www.archive.science.mq.edu.au/archives/18/, the second in Chalon sur Saone, France, http://www.archive.science.mq.edu.au/archives/8/, the third in the Rocks area, Sydney http://www.archive.science.mq.edu.au/archives/20/, and the fourth in Manly Quarantine Station, Sydney http://www.archive.science.mq.edu.au/archives/19/. We had 3 Instructors (surveyors) entering data in Anzac Cove, 3 in Chalon sur Saone, and 4 in Sydney in both the Rocks area and the Manly Quarantine Station, doing 86 mapping in total. They all reported that the system is functional and easy to use. Now we plan to conduct usability testing with visitors to be discussed in the next section.

Anzac Cove, Gallipoli has 46 mappings, Manly Quarantine Station has 16 mappings and Chalon sur Saone and Rocks have 12 mappings each. As can be seen in Figs. 11a, b, 12, 13, 14, 15, the test results were promising and we have improved the usability of the system in a few iterations. Figure 11b demonstrates the concept of a Scenario that is a pop up information that appears in a specific GPS location. In the current version of ArcHIVE 4Any, scenario is user driven. In the future versions, this process will be automated by using personality profiles of the users, and the Scenario will be adapted to the user profile.

Fig. 11

a, b Media and information boxes in Figs. 2, 3 as experienced by Learner and Instructor: ArcHIVE hotspots and marked GPS locations in Chalon sur Saone

Fig. 12

Information Box in Figs. 2, 3 as experienced by Learner and Instructor: ArcHIVE hotspots in Chalon sur Saone and the information listed

Fig. 13

Information Box in Figs. 2, 3 as experienced by Learner and Instructor: ArcHIVE hotspots in Chalon sur Saone and the information delivered in zoom in for the User

Fig. 14

Information box in Figs. 2, 3 as experienced by Instructor and Programmer: ArcHIVE hotspots in Chalon sur Saone and the database entry for the Surveyor/Instructor

Fig. 15

Media box in Figs. 2, 3 as experienced by Learner and Instructor: ArcHIVE hotspots and virtual walkthrough in Chalon sur Saone using the MARS system

In ArcHIVE 4Any, while users are visiting a heritage site, they are able to display pre-recorded video scenes and digital information in various GPS locations, as seen in Fig. 15. This can be done online prior to visiting the location as well. Thus, smart phones and tablets serve as wireless motion trackers and display devices to provide the viewer with free movement capability within the digital landscape of the heritage site, while they are exposed to the specific historical scenes as well as digital information at a specific GPS location. This suggests a powerful sense of time-independent presence in the digital landscape, as if they are actually walking through the heritage site or trenches in the war time.

Usability testing and field studies with learners

We currently conduct Usability Testing with Learners. The potential participants are random visitors of various historical sites. We use a number of questionnaires to measure users’ satisfaction of our application as well as some open-ended questions to identify the weaknesses of the system. In addition to questionnaires such as System Usability Scale (SUS) by Brooke [22], we collect personal information, such as gender, age, nationality, profession, etc., using personality questionnaires (including 50 questions) for creating users profile, using a 50-Item Set of IPIP Big-Five Factor Markers. There are three steps involved in this study:

• 1. Participants are required to complete an online pre-exposure questionnaire. Results will be used to create user profiles.

• 2. Participants are asked to use the application to accomplish one task:

  • TASK A: Participants are asked to visit landmarks with their mobile device and note the order of landmarks.

  • TASK B: Participants are asked to add a landmark and relevant digital information, such as written text, images and videos.

• 3. Participants are required to complete an online post-exposure questionnaire. Results will be used to evaluate usability satisfaction and to improve the system.

The results of questionnaires are automatically stored in our private server for further analysis. The focus of this analysis is usability satisfaction and the factors affecting it. Personal information, such as age, gender, profession, may change more or less the attraction of participants by a historical site. Participants who are interested in history may request profound explanations than others. Their frequency of visiting historical sites may also affect their satisfaction of the contents and depth of information in the application. If the learner is fairly familiar with technology, especially with mobile devices, their expectation of a heritage site may be higher than the others. Personality of participants may affect the order of visiting heritage locations and cause the variations and clusters in navigational paths. This can be used for developing automated scenario engineering tools.

Conclusion

This paper introduces a generic Mobile Augmented Reality Systems (MARS) framework to leverage the application system design and implementation. Proposed 4Any framework could be used as a meta architecture to guide the development of MARS. The paper demonstrates how 4Any framework has been applied in an ongoing project ArcHIVE 4Any using a number of field studies in Chalon sur Saone, France, Anzac Cove, Gallipoli, Turkey, and Manly Quarantine Station and The Rocks in Sydney, Australia. In 4Any Framework, we assume learning has to occur anywhere, anyway, anytime for anybody. The actors in this framework are specified as Instructor, Learner, Device and Programmer. Each actor initiates an action with respective entities Information, Media, Location. Time is considered as another layer with the repetition of these components at another time. What mediates each action is the metadata layer between the entities and people profiles using the same system. Metadata layer provides mapping between the underlying structures and definitions of associated content between the actors and entities. This way, each user sees a different face of the same system and a multi-faceted approach is achieved. The 4Any system architecture separates the modules as four main parts that refer to each column: Instruction, Learning, Device and Coding.

There are two main sides of this framework to support learning anytime, anywhere, anyway by anybody: Pedagogical and Technological. The framework is composed of 4 layers that correspond to each row: Actors, Profiles, Metadata to be shared, and Scenario. The usability test results with 10 Instructor profiles suggest that 4Any framework will be able serve as a vehicle for people-centric computing, to be used by anybody, anywhere, anyway, and anytime. Thus, the paper contributes to the knowledge in Human Centric Computing by taking into account multiple users and presenting different faces to each user demonstrating an example of the People Centric Computing paradigm.

Strengths and limitations

Although CyberGuide in ArHIVE 4 Any application is a demonstration of the flexibility and extensibility of the framework, it does not have scenario engineering capabilities. We plan to add this module in future versions of the system. Scenario engineering for each user will be performed after collecting data in the usability studies with learners, using the personality profiles and navigational routes followed by each participant in two tasks: navigation in digital heritage and surveying (marking) digital heritage locations to deliver information. These tasks are addressed for two different types of users: learner (citizen) and instructor (surveyor). Our idea is that citizens in a cultural context can be treated as curators, and their navigation in heritage collections conveys narratives. Our future aims are: (1) Navigation Analysis in the exploration of cultural heritage in VR and AR, (2) Narrative Analysis to compare the temporal and spatial parameters of navigational patterns in VR and AR, (3) Analysis of User Experience to measure learning outcomes as well as the impact of autonomy in navigation, (4) Model Development to obtain the optimum user experience and learning outcomes. The first three aims will inform the 4th to obtain the optimum user experience through alternating the temporal and spatial parameters of the medium such as AR and VR, as they present different temporal and spatial parameters to measure differences in presence, transportation, motivation, autonomy, and more importantly, user navigation associated with narratives. However, the scope of this paper is limited with Mobile AR and we leave the debate of temporal and spatial parameters outside the scope of this paper. It is important to note that we have developed the framework to address the needs for a more general purpose, targeting scenario engineering as an extension in future.

In the current version of ArcHIVE 4Any, scenario is user driven. In the future versions, this process will be automated by using personality profiles of the users. Figure 2 demonstrates the concept of a Scenario that is a pop up information that appears in a specific GPS location.

In an AR Platform, the users are expected to use their phones like a camera to be able to see the real world as the pan around. The application should utilize the users’ GPS location and determine exactly what their mobile device is pointing towards, and provide them augmented information about the target heritage object directly on their mobile devices. This capability should be added to ArcHIVE 4Any in future.

The digital information presented in an AR application is called a layer. Layers can provide services, such as finding a specific historical location using GPS locations, reviews of historical facts, and customised virtual tour guides. Layers can also provide an experience with interactivity, 3D objects and sounds for simulations of historical facts and engaging guided tours. Currently, we have used information presented in static guided tours published by the local authorities in both field works. However, we plan to update the information currently given in the fieldworks using experts in specific heritage locations in future versions of the development.