Collaborative behavior, performance and engagement with visual analytics tasks using mobile devices

Abstract

Interactive visualizations are external tools that can support users’ exploratory activities. Collaboration can bring benefits to the exploration of visual representations or visualizations. This research investigates the use of co-located collaborative visualizations in mobile devices, how users working with two different modes of interaction and view (Shared or Non-Shared) and how being placed at various position arrangements (Corner-to-Corner, Face-to-Face, and Side-by-Side) affect their knowledge acquisition, engagement level, and learning efficiency. A user study is conducted with 60 participants divided into 6 groups (2 modes $\times$ 3 positions) using a tool that we developed to support the exploration of 3D visual structures in a collaborative manner. Our results show that the shared control and view version in the Side-by-Side position is the most favorable and can improve task efficiency. In this paper, we present the results and a set of recommendations that are derived from them.

Introduction

Visualizations are external aids that are intended to support users’ analytical and exploratory activities [1]. Increasingly, these activities are conducted by teams. Supporting collaborative activities remains one of the great challenges within interactive visualizations and visual analytics [2, 3]. Collaborative visualizations aim at facilitating analytical activities by more than one user, and allocating different parts of the task to different people, either distributed in various places or co-located in the same physical space.

Researchers have been exploring the use of mobile devices, particularly tablets, to overcome the limitations of large displays for collaborative activities [2, 4]. Today’s mobile devices are quite powerful. With mobility, portability, wireless connectivity, and touch-enabled input, tablets have been widely used and are claimed to be suitable devices to support co-located collaborative workspaces [5, 6]. There has been increasing amount of research focusing on techniques to better support co-located collaborative activities with tablets [7,8,9,10,11], including for learning in groups. Researchers and educators have been trying to incorporate tablets in their teaching activities [12,13,14,15,16]. For example, students as a group could complete a task by simply interacting with their own device since all the interactions could happen within one single virtual space. Such an approach has been well received by teachers and students because it can lead to a general improvement in students’ learning outcomes [17].

Most collaborative applications are about drawing or document annotation [12, 18,19,20], while research on the use of mobile tablets to enable collaborative exploration of visualizations of more complex concepts is still largely underexplored. Although some prior research explored the effect of sharing content among users on task performance [21], there is little research that has compared Shared and Non-Shared mode using tablets in learning settings. Besides, previous work has identified the arrangement of users as one of several factors that influence co-located collaborative tasks [22, 23]. Some early research on position arrangement suggesed that different collaborative coupling and position distance would affect user behavior and performance during collaboration. However, these related work investigated the position arrangements and collaborative coupling around a shared device (like a large tabletop or wall display) [24,25,26]. Research that focused on mobile devices is still largely underexplored. Therefore, this research investigates cross-tablet collaboration in a co-located setting. We focus on two interaction modes and for three position arrangements, and explored how they affect collaborative tasks with visualizations in a co-located setting. We investigate the effect that two factors, shared/non-shared view/control and position placement, have on the learning improvement and engagement of learners. To do this, we have developed a tool that can support group exploration of 3D geometrical shapes in terms of their individual structural properties and transformative processes. We run an experiment with two collaborative modes: (1) shared and (2) non-shared interaction and view, and having three position arrangements: (1) Corner-to-Corner (C-C), (2) Face-to-Face (F-F), and (3) Side-by-Side (S-S). The shared mode supports two users in interacting simultaneously; they share the view and control of the visual objects. In the non-shared mode, one user cannot see from his/her tablet what the other is doing and the control is not shared.

The main contributions of the paper include: (1) an application for mobile tablets that allow users to explore 3D geometrical shapes in terms of their individual structural properties and transformation processes; (2) an exploration of task performance, knowledge acquisition, engagement levels, and collaboration behaviors and patterns of users using mobile devices to explore visualizations; (3) an understanding of the positive affordances and limitations of each mode of interaction and position arrangements; and (4) a set of implications from our findings for future developments of collaborative visualizations and applications in mobile devices that can support positive collaborative interaction in co-located settings.

Background and related work

Co-located collaborative visualizations

Collaborative scenarios can be generally categorized along two axes: space (co-located vs. distributed), and time (synchronous vs. asynchronous) [2, 27]. Research that has focused on co-located collaboration can be divided into two groups based on the display type: single display (normally large displays) [28, 29], or multiple displays (based on networked devices) [2, 30, 31]. Some systems have explored integrating mobile devices (for individual view) with tabletops (for group view) in solving collaborative tasks [32,33,34]. Selwyn-Smith et al. [35] explored how cross-device environments consisting of digital tabletops, mobile tablets, and laptops can support co-located collaborative tasks in block-based programming. Besides, a study [36] with three different conditions (a digital tabletop, personal tablets, and both tabletop and personal tablets) was conducted to understand how these devices support sense-making for groups. Their results show a negative correlation between utilization of personal devices and group performance. Numerous useful visual analytics tools have been designed to help domain experts solve analytical problems. For example, Dong et al. [37] designed a collaborative touch-table application (iPCA-CE) by adopting an existing single-user desktop analytical tool (iPCA). With the system, users can actively transit from individual desktop to shared collaborative environments without losing track of their analysis. Konkel et al. [38] explored collaborative and interactive visualizations of genomic and related scientific datasets. For their tablet-based application, they investigated the exploration of genomics work using a vertical screen, three tablets, six Sifteobased tokens, and a smartphone. Bishop et al. [39] presented Construct-A-Vis, a tablet-based tool designed to help explore the feasibility of group-based free-form and constructive visualization activities for elementary school children. They noted that shared visualization processes further prompted productive discussions and collaborative behaviors.

Isenberg et al. [2] defined collaborative visualization as the shared use of computer-supported interactive visual representations of data by more than one person to do sense-making tasks. Some researchers have focused on the challenges in visual representations and tried to provide different representation possibilities to explore collaborative aspects in co-located tasks (e.g. [40, 41]). Desolda et al. [42] explored composition techniques to allow combining spatial-aware mobile devices to foster sensemaking activities. Phil et al. [43] presented a visual analytic tool, ActiVAte, which can facilitate human-machine collaboration, inspection, transparency, and trust of the learning process through human-machine collaboration. The VizCept system, for example, allows users to work in their own workspace and collaborate with others using representations like concept maps and timelines [44]. Others have focused on interaction techniques to support collaboration like linking meta-visualizations to help collaborators be aware of each other’s actions [45], and to eliminate the influence of fixed orientations in collaborative tabletops [4].

Some researchers began to explore the use of tablets to support the collaborative exploration of visualizations. Hemmings et al. [46], for example, have developed the distributed system IDCVS which allows real-time collaborative analysis with visualizations of large data sets for tablets. While representing a positive development, the studies of collaborative visualizations on tablets are still very limited compared to those for tabletops and research is still needed to understand how they should be designed and deployed.

Shared vs Non-Shared view and control

Shared virtual workspaces [36, 47, 48] and augmented reality (AR) systems [49, 50] allow users to concurrently interact with visualized data. Greenberg [51] has surveyed and discussed systems that are aimed at providing shared views among distributed worksites. As Isenberg et al. [2] highlighted, co-located synchronous collaboration comes with inherent interaction challenges that arise when multiple people have the possibility to synchronously interact together. These interaction conflicts bring to the foreground social norms and conventions that are difficult to overcome. The shared view mode of tablets is a possible solution to solve these issues, because when using multiple tablets users can have a shared view, while still choose to be located at a comfortable distance of other users.

How the tasks are presented on the devices when users need to collaborate can affect their behaviors and performance. In addition, they are affected by their perception of territoriality issues and how the workspaces (both individual and group) are provided in these devices [52, 53]. For example, Larsen-Ledet et al. [54] suggested that territoriality issues related to users’ workspaces could serve as a framework to guide the development of social structures and maintain them during cooperative activities. Some researchers have explored the collaborators’ behaviors with one shared tabletop workspace [55, 56] and cross-device workspaces [57, 58]. There seems to be little research that has explored shared view and control of collaborative visualizations in co-located settings [59, 60]. Hinckley [61] has developed several interaction sensing techniques that support transitions between the shared and personal working areas, for example, tilting the tablet toward oneself to enlarge the personal workspace. Similarly, Kaufmann [62] has investigated two modes, independent (i.e., every student can only see the elements constructed by him/herself) and collaborative (i.e., everything is visible to everybody), for Construct3D, a geometry construction tool designed for mathematics and geometry education. These papers proposed systems that share related or the same views across devices but have not provided any user evaluation. Huang et al. [63] developed the coisTable tabletop system that supports co-located collaborative activities. It divides personal and shared workspaces using additional physical representations that are intended to help users distinguish with ease the boundaries of different application spaces.

Chung et al. [64] presented a cross-display visual link technique which can allow users to see connections of related information across displays. Their results showed that the visual links of tasks across multiple displays effectively helped users organize and synthesize related information scattered across devices. Kraut et al. [65] have argued that a shared visual space is essential for solving complex collaborative visual problems because it can improve the awareness of the state of task and facilitate communication of the group. In addition, results in [66] indicate that there exists an overall positive attitude towards paired tablets, but it has not led to significantly better task completion times because their views are not entirely synchronized and shared. In short, the above review shows that there seems to be very limited work that has investigated thoroughly the effects of shared and non-shared view and control on co-located collaborative visualizations in mobile devices.

Position arrangement during collaboration

Arrangement of users and collaborative coupling refer to where they are placed, and the way collaborators interact with each other. It mainly reflects users’ need to work tightly with or independently from one another. How users are arranged influences their performance in completing collaborative tasks. For example, Yiannis et al. [67] conducted a study that investigated the effects of strong versus loose coupling on middle school students’ learning performance. Their findings showed higher conceptual learning for the students participating in the strong coupling condition. Early research on collaboration around physical tables determined that face-to-face or right-angled seating was preferred for conversations due to their support for visual contact [68]. Previous work on collaboration around a tabletop display found that a face-to-face arrangement better supported non-verbal communication than sitting side-by-side [69]. Rodden et al. [70] noted that a side-by-side arrangement helped to ease the social awkwardness of strangers, as there was less expectation for direct eye contact. Tang et al. [26] have identified six coupling styles in tabletops and concluded that groups with these coupling styles when working together prefer common and global views. Isenberg et al. [24] have extended these coupling patterns into eight and grouped them as close and loose collaboration. They observed that pairs with frequent synchronizations produced discoveries that are more successful in solving experimental tasks.

Jakobsen and Hornbæk [25] have explored coupling patterns of collaborators when working around a wall display. They observed users’ visual attention, proximity, and verbal communication and suggest that proximity between members is related to how tightly coupled they are working together. Liu et al. [60] have explored five coupling styles around these displays but their styles correspond to very different dimensions–collaborative coupling and shared interaction support. Their results show that a shared interaction technique can facilitate collaboration by allowing collaborators to work more tightly together even when not in close proximity.

Research on collaboration coupling in tablets has also looked at gaze or eye contact and verbalization [49]. Plank et al. [66] have observed how users positioned their tablets from their personal territory to group territories by rotating or tilting the tablets to share content. Marquardt et al. [9] have explored the coupling styles for tablet users using two sociological constructs: F-Formations (the distance and relative body orientation among multiple users) and micro-mobility (how people orient and tilt devices towards one another for sharing ideas).

Previous research has found that different coupling patterns yield different communication strategies together with how closely the collaboration is. As one crucial element of coupling patterns, the relative positions of collaborators reflects the relationships between them and how they work together. However, how position arrangements influence collaboration behaviors and sense making in a co-located setting remains largely under-explored. There does not seem to be that much work examining whether the different positions will result in different collaborative behavior, experience, and performance. Our research aims to extend the investigation of position arrangements of collaborative coupling with mobile tablets under two settings, shared and non-shared view, to further explore the collaborators’ behaviors and interaction patterns.

Solid visualization tool

Visualization tools can allow learners to visualize and manipulate abstract geometric structures and their spatial relationships and are useful to help them with their mathematical reasoning and learning [71,72,73]. In addition, peer collaboration is useful for challenges that require spatial reasoning [74]. Therefore, we developed a visualization application to facilitate the collaborative process of exploring, analyzing and learning 3D geometric shapes (see Fig. 1). These solids can be obtained from one another and their relationships are multilayered, not only dealing with planes of symmetry but also how the shapes can be derived from each other by truncating/augmenting their vertices and edges [75].

Fig. 1

The interface of the 3D solids visualization tool

Visualization application overview

The tool has been developed in Unity3D. The main goal is to assist users to visualize and explore 3D shapes. This tool is used to allow us collect experimental data and explore the differences among the various conditions. As a visualization tool, no direct or explicit instructions about how they should interact with the tool are provided. Users have to explore and analyze by themselves using the tool’s visual interactive functions and interpret the results and draw their own conclusions [76, 77]. Figure 1 shows the interface of the tool, which is implicitly divided into 5 sections: (1) Synchronize function: The transformation of three solids will be synchronous when changing from one same shape to another same shape; (2) User ID: It indicates how many users are using the tool at the same time; (3) 3D solid visualizations: There are three solids, cube, tetrahedron and Octahedron; (4) Solid transition maps (STM): They can support active navigation within and between visualizations. Thumbnails of shapes are connected by lines to indicate the transitional processes of how shapes can be derived from each other; and (5) Network Connection: Shared/Non-Shared mode can be chose by clicking “Connect/Disconnect”.

Solid visualizations

The 3D solid visualizations placed in the middle of the interface present structural information of the solids. Each solid is rendered with three different colors indicating the process of truncating their vertices and/or edges. On a corner of each solid, there is a modified solid transition map that shows the current stage of transformation of the solid. Also, it serves as an implicit link to the map located below. Figure 2 shows an example of this relationship: the cube with the map attached to the top-right corner (LEFT) and its transition map (RIGHT).

Fig. 2

The introduction of the basic solid. (LEFT) Rhombi-truncated cuboctahedron obtained by truncating all the vertices and edges of a cube; (RIGHT) Solid transition map of the cube indicating the current state of the transformed solid. The black dot on the solid’s triangle (LEFT) shows the location of the current solid on its transition map (RIGHT)

Morphing of the solids

Users can dynamically and continuously morph or transform a 3D solid via the STM. Each map has an interactive selected solid (see Fig. 2 for the highlighted circles on the maps). Users can interact with this selected solid and move it anywhere on the map. All changes are reflected on the solid—this interaction is referred to as “indirect manipulation”, where users communicate with an object via a second object [78]. On the other hand, direct manipulation, which gives a more direct engagement with an object [79], is implemented as an interactive dot on the local STM of each solid. Using the dot users can morph the solid directly by freely moving it within the local STM. By embedding both direct and indirect manipulation, users can choose the STM to morph the shape indirectly, which provides a clearer sense of global transformations, or choose the interactive dot to have direct engagement with the enlarged solids.

Dynamic-linking of multiple visualizations

Twin Solid is the shape that has the same structural properties as the current selected solid but is different in how it is obtained [71]. Such property represents more abstract complex connections between multiple STM, as identical shapes can be obtained from different base solids but through different transformation processes. In our tool, the STM not only can highlight the selected solid, but also the twin solid(s). As shown in Fig. 3, for any selected solid on a map, the corresponding twin solid(s) will also be shown and highlighted on the other two maps. Such function can facilitate global knowledge acquisition across multiple maps–that is, the solids and the maps all become dynamically linked to help visualize the existence of twin solids. Users can switch off the “Synchronization” toggle (on the left corner of the tool) to turn off the dynamic linking of visualizations.

Fig. 3

An exploration scenario. It shows the selected solid (currently on the Octa-Map, most left solid) and one twin solid on each of the other two maps (the Cube and Tetra maps)

Shared interaction and view between multiple tablets

As mentioned before, the application has two versions, Shared and Non-Shared control and view. The shared mode supports two (or more) users to interact simultaneously with shared view and control of the visual objects using connected tablets so that they can see what each other is doing in real time. In the non-shared mode, one user cannot see from their tablets what the other is doing. If one user is interacting with an object, it is highlighted on the other user’s tablet in the Shared version, but no in the Non-Shared one (see Fig. 4).

Fig. 4

The two diffrent modes of interaction. a) Interaction in the Shared version (synchronous change in two applications); b) Interaction in the Non-Shared version (asynchronous change in two applications)

Experiment design

Participants

Sixty students (32 females) aged between 18 to 28 years old ($\text {M} = 19.90$, $\text {SD} = 2.11$) were recruited to participate in the study. We advertised our study through email listservs, in classes, and via online forums in a local university to ensure that participants had diverse educational backgrounds. None of them had used the tool before the experiment. Based on the results of our pre-study survey, 50% of the participants thought they had a good sense of space and 76.57% thought they were good at cooperation.

A between-subjects design was used–i.e., one participant could only interact with one position arrangement and one version of the application to avoid any carry-over effects. There were 2 (versions) $\times$ 3 (positions) groups, that is 30 participants for each version of the tool and 10 participants for each position arrangement in each version (see Table 1). For each version, there are 3 position arrangements, C-C, F-F and S-S (as shown in Fig. 5). The position for each pair was randomly set. The pairs were allowed to discuss freely during the collaboration process.

Table 1 Overall descriptive data for all 60 participants

Fig. 5

The three position arrangements based on how participants sat during the experiment. a Side-by-Side (S-S); b Face-to-Face (F-F), and c Corner-to-Corner (C-C)

Apparatus

The application was run on a Google Pixel C tablet with a 2560 * 1080 resolution multi-touch screen. Finger touch was the only input mode as it is the most common way for interacting with these devices. Also, we prepared the tasks printed on paper. During the experiment, each pair sat around one table according to a predefined position arrangement. We used a video camera to capture participants’ interactions with the tool and their verbal and non-verbal communications. The interactions with the tool were also screen-captured for later assessment.

Tasks and procedure

Pre- and post-test tasks

In the pre-test and post-test stages, the task was to test the user’s spatial geometric capability by cutting and augmenting the vertices and edges of three solids (cube, octahedron and tetrahedron). We set up 12 tasks (1 mark for each task; 12 marks in total). According to the solid given in the question, users were asked to select the correct one that can be obtained by cutting or augmenting the vertices and edges of solids from the options. Two users were required to complete the test independently.

Collaborative tasks

In the collaboration stage, the task was still to cut and augment the vertices and edges of the three solids, but the questions were different from that in the pre- and post-test. Several sample tasks are provided (see Fig. 6). During collaboration, each pair was asked to collaboratively get the correct answer, and two users must provide the same answer for each question by discussing together. There were 12 tasks during collaboration (1 mark for each task; 12 marks in total), which were shown on paper.

Fig. 6

Some sample tasks used in this study

Procedure

Each session was divided into six phases: (P1) Informing participants of the purpose of the study and the ethics regulations governing it, and then completing the consent form plus a short questionnaire to collect anonymized demographic data ($\sim 5 \,\text {mins}$); (P2) Pre-test with 12 knowledge-based questions about the geometric shapes within 10 mins. If users can not finish all tasks within the time limitation, the tasks that they did not complete will be given 0 mark; (P3) Performing 12 predefined collaborative tasks with the tool on the task sheet ($\sim 30 \,\text {mins}$); (P4) Completing the engagement questionnaire ($\sim 5 \,\text {mins}$); (P5) Performing post-test which was the same as the pre-test (10 mins); and (P6) A brief interview on their experience of the experiment ($\sim 5 \,\text {mins}$). The whole experiment took about one hour to complete for each pair. A training session was provided to participants before the collaborative task for them to become familiar with the interface and interactive features. After that, we would not provide any further assistance to participants.

Hypotheses

We tested the following four hypotheses in this experiment:

H.1.

Shared group would perform better than the Non-Shared group on task performance:

Prior research has emphasized the importance of collaboration on supporting learning. We expected that the same would apply to collaborative problem-solving using mobile devices. We anticipated that shared view/control could facilitate thinking and sharing ideas while solving problems. By making one’s interactions continuously visible to each other, it would lead to a more efficient problem-solving and exploration.

H.2.

Side by Side position is most suitable for improving task performance:

Because of the near distance, users can more easily communicate and share their ideas with little movement and they can communicate much more than in other positions, which would improve task performance to some extent.

H.3.

Shared mode would lead to higher scores than Non-Shared one on engagement during collaboration:

With shared view/control, participants could to see the effects of their partner’s actions. As such, shared view/control would facilitate communication and interaction, which could contribute to a higher engagement level. Besides, because of the shared control, users would easily share their ideas without moving, which would avoid interrupting the user’s operation and provide a higher sense of immersion in the collaborative task.

H.4.

Side by Side position would be the most preferred arrangement:

For the position arrangements, we expected that the Side-by-Side position would lead to higher level of engagement during the collaborative task because participants would feel more natural and it would be more convenient to communicate in near distance.

Results

By assessing user’s performance on the pre- and post-test, collaborative tasks, and ratings on engagement and collaboration questions, we were able to quantify users’ knowledge acquisition of the domain and their perceived level of engagement and collaboration. We next present the results of the quantitative measures, and then combined those with the qualitative measures to cross-validate user performance and collaboration patterns. For simplicity, we will use M, SD, and SE to denote mean, standard deviation and standard error values. The significance level is set to 0.05.

Test score and completion time

Test improvement score

As mentioned earlier, in the pre- and post-test, all participants were asked to finish 12 tasks (12 scores in total) within 10 mins. Between the two tests, participants needed to complete some tasks cooperatively under the condition assigned to them. Our results show that all participants, regardless of their groups, experienced different levels of improvement after interacting with the tool. Overall, participants in the Shared version had a higher mean improvement ($\text {M} = 2.10$, $\text {SD} = 2.155$, $\text {SE} = .393$) than Non-Shared version ($\text {M} = 1.47$, $\text {SD} = 2.113$, $\text {SE} = .386$). In each version, there were some differences on mean improvement between position arrangements. Overall, C-C led to the highest improvement ($\text {M} = 2.50$, $\text {SD} = 1.650$, $\text {SE} = .522$ for Shared version, $\text {M} = 2.50$, $\text {SD} = 1.780$, $\text {SE} = .563$ for Non-Shared version), and F-F got the lowest improvement ($\text {M} = 1.40$, $\text {SD} = 2.836$, $\text {SE} = .897$ for Shared version, $\text {M} = 0.70$, $\text {SD} = 1.418$, $\text {SE} = .448$ for Non-Shared version), as shown in Fig. 7.

Fig. 7

The mean of improvement in scores (between the pre- and post-test) in different conditions

Fig. 8

Distribution of the raw data for score improvement in different conditions. (C-C: Corner to Corner; F-F: Face to Face; S-S: Side by Side)

Figure 8 shows the distribution result of the improvement in scores between the pre- and post-test. We can see that regardless of the version, the participants in C-C achieved the best mean improvement, while F-F led to the worst improvement, a two-way ANOVA (Analysis of Variance) test yielded no significant effect of positions ($\text {F}_{2,54} = 2.371$, $\text {p} = .103$, $\eta \text {p}^2 = .081$, observed power $= .459$). The mean improvements in all positions for the Shared version were better than those in the Non-Shared version, however, there was no significant effect of versions ($\text {F}_{1,54} = 1.357$, $\text {p} = .249$, $\eta \text {p}^2 = .025$, observed power $= .208$) and no significant interaction effect of version $\times$ position ($\text {F}_{2,54} = .410$, $\text {p} = .666$, $\eta \text {p}^2 = .819$, observed power $= .113$) according to a two-way ANOVA test.

Time and score on the collaborative tasks

As stated before, participants working collaboratively had to complete a set of tasks. We observed their behaviors and communication patterns and recorded the scores and the time for completing tasks (12 scores in total). Answers to the questions and the time spent could provide insights on the task efficiency and allow us to assess how they were affected by their positions and interactions with the tool.

Fig. 9

The mean time and score for collaborative tasks in different conditions. (C-C: Corner to Corner; F-F: Face to Face; S-S: Side by Side; Ctime: the time for collaboration; Cscore: the score for collaboration)

In Fig. 9, the Shared group generally spent less time ($\text {M} = 29.800$, $\text {SD} = 2.704$, $\text {SE} = .698$) to complete the tasks but still got a slightly higher mean score ($\text {M} = 9.800$, $\text {SD} = 1.373$, $\text {SE} = .355$) than Non-Shared one (Time: $\text {M} = 33.200$, $\text {SD} = 4.769$, $\text {SE} = 1.231$; Score: $\text {M} = 9.133$, $\text {SD} = 1.356$, $\text {SE} = .350$). Overall, S-S participants spent the least amount of time ($\text {M} = 30.200$, $\text {SD} = 1.619$, $\text {SE} = .512$) and got the highest mean score ($\text {M} = 10.100$, $\text {SD} = 1.524$, $\text {SE} = .482$). On the other hand, C-C participants needed the most amount of time ($\text {M} = 33.700$, $\text {SD} = 6.533$, $\text {SE} = 2.066$) and achieved the lowest score ($\text {M} = 8.800$, $\text {SD} = 1.398$, $\text {SE}= .442$).

A two-way ANOVA test yielded a significant difference on completion time among versions ($\text {F}_{1,24} = 8.128$, $\text {p} = .009$, $\eta \text {p}^2 = .253$, observed power $= .781$) and positions ($\text {F}_{2,24} = 3.441$, $\text {p} = .049$, $\eta \text {p}^2 = .223$, observed power $= .589$). There was also statistically significant interaction effect between version and position on time spent ($\text {F}_{2,24} =4.284$, $\text {p} = .026$, $\eta \text {p}^2 = .263$, observed power $= .691$). Overall, the Shared version was better than the Non-Shared one on task efficiency. A further Tukey HSD (Honestly Significantly Different) post hoc test showed that S-S obtained the best performance (least amount of time and highest scores) on the tasks than the another two (see Table 2). Therefore, S-S position was the best group on task efficiency and C-C position was the lowest condition.

Table 2 Turkey HSD’s test results for the difference between mean task performances

The average scores by the Shared and Non-Shared pairs were very similar. Although there were some differences among the mean scores in different conditions, there was no statistical significance on the scores among the different versions ($\text {p} = .178$, observed power $= .265$) and positions ($\text {p} = .108$, observed power $= .443$), as indicated by the results of the two-way ANOVA test. Similarly, no statistically significant interaction effect was found between version and position ($\text {p} = .559$, observed power $= .138$).

Perceived engagement levels during collaboration

Overall engagement ratings

The engagement level effectively reveals learners’ persistence, academic achievement and satisfaction with their learning [6, 80,81,82,83,84]. A 7-point Likert scale (from level 1 to 7) user engagement questionnaire was used to measure participants’ engagement levels based on 7 categories (via self-reporting) [85]. The questionnaire for assessing collaboration experience was derived from the work of Isenberg et al. [24] and Jakobsen and Hornbæk [25]. After making some adaptations, our final questionnaire had 23 questions on engagement level. Videos were also used to record participants’ behavior. These types of data would help us assess how engaged the participants were during the analytical process.

The Shared group obtained a slightly higher overall score for engagement ($\text {M} = 41.817$, $\text {SD} = 5.862$, $\text {SE} = 1.070$) than the Non-Shared group ($\text {M} = 41.256$, $\text {SD} = 5.503$, $\text {SE} = 1.005$). It was also noteworthy to mention that S-S, regardless of the version used, got the highest engagement scores ($\text {M} = 42.025$, $\text {SD} = 6.012$, $\text {SE} = 1.344$). Nevertheless, a two-way ANOVA test revealed no statistical significance on the overall engagement scores among the two versions ($\text {p} = .709$, observed power $= .066$) and three positions ($\text {p} = .831$, observed power $= .077$). No statistically significant interaction effect was found between version and position on engagement either ($\text {p} = .466$, observed power $= .175$).

Ratings on the engagement categories

Figure 10 shows the results based on the 7 categories of engagement for the two different versions of the tool. Overall, Shared version ($\text {M} = 5.974$, $\text {SD} = .837$, $\text {SE} = .153$) received a slightly higher rating than Non-Shared one ($\text {M} = 5.894, \text {SD} = .786, \text {SE} = .144$) on the mean engagement level. We can also see that the range was smaller. The Shared version was rated higher on Exploration, Attention, Satisfaction, Comfort, and Communication. It could be inferred that participants felt more involved in the exploration process. However, it seemed that participants in the Non-Shared group generally thought that they contributed more.

Fig. 10

The original data of the two different versions on 7 categories of engagement

Fig. 11

The original data for the three different positions on 7 categories of engagement

From Fig. 11, we can see that the 3 position arrangements received good scores on almost all categories ranging between 5 and 7. When compared with other categories, Contribution got the lowest scores in all positions. We can also see that the engagement ratings by users of S-S had a larger range for almost all the 7 categories and had higher median. Overall, the results show that S-S received a higher mean engagement ($\text {M} = 42.025, \text {SD} = 6.012, \text {SE} = 1.344$).

Although there were some differences on mean of engagement level (overall and sub-scales) among different versions and positions, we did not found any statistically significant differences among 2 $\times$ 3 conditions (all $\text {p} > 0.5$), as indicated by the results of the two-way ANOVA test.

Subjective feedback

Position preference

In this study, participants had to complete tasks at a predefined position arrangement. After the experiment, we also collected their preferences on different positions if they could choose a position freely. The results showed that S-S (41.94%) was the most popular choice; F-F received a slightly lower proportion (37.09%); the lowest was C-C (20.97%).

Collaboration experience

Our questionnaire also had some questions about their collaboration experience. 64.51% thought the relationship among partners would affect the learning performance in collaboration. 64.52% said they were interested in this collaborative mode. During the experiment, one user mentioned this study was really interesting many times.

In addition, we asked some questions on issues of territoriality. Most users thought private and public space were both important when collaborating. Although a shared space is essential for collaboration, some private area or space is still needed to allow them to have independent thinking and exploration.

Discussion

Shared and Non-Shared control and view

We found that the average improvement from pre- to post-test of Shared participants was higher than the Non-Shared group, which aligns with H.1. This shows that Shared type of collaborative mode with mobile devices may be more beneficial for learning. However, although the Non-Shared group did not improve a lot, users performed well in completing the tasks correctly both in Shared and Non-Shared versions.

When collaborating, participants in the Shared group had been able to complete learning tasks in a shorter time but still gained more understanding than the Non-Shared group. From this result, we can conclude that sharing visualizations of tasks among users can help them to share their perspectives in time with their collaborators, which would also contribute to improvements in task efficiency. They can communicate with each other by sharing ideas even if they are not too close. Otherwise, users have to position their mobile devices from their personal territory to group territories by rotating or tilting the devices to improve readability or share/highlight contents, which would lead to time waste and affect the smooth communication between users to some extent. This may be the reason why most users preferred Shared mode during collaborative learning.

Besides, participants provided high scores on their engagement level. It seems that a collaborative setting made users feel engaged. Participants in the Shared mode provided a slightly higher scores. In this sense, these participants considered shared interaction more focused and involving. However, the ANOVA test yielded no significant difference between Shared and Non-Shared versions, which contradicts our H.3. One possibility is that our experiment setting is around a table, which made the distance between paired users very close. Therefore, the users in any position could possibly see their partners’ device when they move the tablet, and users can show theirs ideas and activities to their partner, which made user engagement levels similar across different modes.

A slightly higher score from the Shared group on Communication, Exploration, Satisfaction and Comfort could imply that having a partner would ease the mental burden for participants to complete the tasks. They may feel more comfortable and supported. However, their lower rating for Contribution gave some insights to the potential drawbacks of shared view when using tablets. When two participants have separate analytical and thinking orientations (e.g., one participant would like to morph the shape while the other would like to rotate the current shape), it can lead to some attention anxiety, which may negatively affect the learning and collaboration process.

Position arrangements

Participants placed at C-C and S-S positions got higher improvements of test scores, which partially aligns with H.2. This means that these position arrangements may be suitable for collaborative learning tasks.

It is also worth mentioning that participants in the S-S position completed the tasks collaboratively within the shortest time but achieved the highest scores compared to the other two positions. Therefore, S-S seems the most suitable in this kind of learning scenarios. Normally, users placed at S-S positions were closer in distance, which made it more convenient for them to share their ideas. This also led to a higher task efficiency. In terms of the two versions, S-S in the Shared version may be the most suitable for collaborative learning. In addition, because the ANOVA test yielded a significant effect of Version and Position on completion time, it is perhaps important to consider both the mode and position of users when completing collaborative tasks. In the research of collaboration around a tabletop display [69], the ‘face to face’ position had better performance in communication than ‘side by side’ arrangement, which is different from our results, S-S spent less time on collaborative tasks than F-F position.

The three position arrangements received relatively high scores on all 7 categories. Overall, S-S had the best ratings on engagement and this shows that this position had a higher acceptance level, which aligns with H.4. However, we did not find any significant difference among all positions on user engagement, although we expected users would prefer the S-S position more. Similarly, we think that the reason was still close proximity between participants regardless of their position arrangements. The distance between users in any position was close and similar, which may have caused user engagement levels similar across different positions.

However, in terms of our participants’ subjective preference on the position arrangement, the results show that S-S was the most popular choice; F-F received a slightly lower proportion; and C-C was the least preferred. Most participants wanted to be in the S-S position if they had a choice. Prior research has shown that the side arrangement is also common for collaborative interaction around a tabletop [34]. This result may be caused by that S-S position is more normal behavior when two or more people are together, therefore, S-S had a higher subjective acceptance level among users.

Collaboration patterns and behaviors

For the Shared condition, we observed that one of the reasons for collaborators to speak more was that they tended to explain their actions to their partners or to inform their partners of their intentions. Although they actively shared ideas with each other, they also interacted with their own devices most of the time. This suggests that these participants want to have some control of their interaction. For example, these two comments show this: “Collaborating is fun, but sometimes I have to wait for my partner to finish...” and “It could be useful if I can choose to share my interactions or not”.

For the Non-Shared pairs, we observed that their communication has not been as much or as fluid as the Shared pairs. It was frequent for them to see the manipulations performed by the partner directly on the partner’s tablet when they communicated with each other. We often saw them re-orienting and tilting their own devices towards their partner to facilitate information and idea sharing. Sometimes they directly manipulated the visual objects on their partners’ devices to demonstrate their thinking and reasoning process. However, it is worth mentioning that participants in the Non-Shared condition felt that they had contributed more to solving the tasks. That is, they might have tended to work as single individuals, rather than collaboratively, and because of this they felt that they had contributed more to the problem-solving process. This also aligns with the answers from the engagement questions in that the individuals in the shared group felt that they made less overall contribution.

We also observed the users behavior/patterns during collaboration. These positions allowed different levels of visibility to the screen of their partners, which could have affected the efficiency of grounding in communication. As we expected, when collaborators were placed at S-S position, they worked more closely together. For example, based on the qualitative analysis, participants in S-S position communicated with and demonstrated their interactions to their partners more frequently. In S-S position, because of the near distance, even in Shared mode, users preferred to show personal device to the partner for sharing their ideas. When they were in F-F and C-C positions, they had more eye contact. However, in these two positions, it is not convenient for users to see the device of the partner, they had to move the tablet to show the operations in Non-Shared mode.

Implications of our findings

Based on our results, we are able to distil these three design implications for deploying collaborative visual analytic systems in a co-located cross-device setting.

• If the goal is to maximize learning within a limited time, shared interaction/view and both C-C and S-S positions could be chosen.

• If the goal is to maximize task efficiency in completing a set of tasks cooperatively, S-S position with shared view/control could be considered.

• If there are multiple dynamically linked visualizations, it is useful to provide a function for users to switch between the shared and non-shared modes.

Limitations and future work

There are several limitations in this research. Although the size of our sample population is line with experiments of similar nature [44], it may be useful to see if a larger sample would lead to similar or different results. Another limitation was that, instead of controlling strictly the time during collaboration, we allowed participants to finish all tasks without time limitation. Although the time they spent among different conditions were similar, it may also affect their learning performance. Also, our study has focused on a co-located synchronous collaborative setting and as such our findings may not apply to situations where users need to perform visual analytics tasks in a remote or asynchronous manner. Additionally, as with other similar studies, our experiment was conducted in a lab setting. Given the potential use of the tool in educational settings, it will be useful to have the experiment done in a natural setting (such as a classroom). This would allow us to explore other factors that may affect the use of the tool (like the influence of a teacher and integration of related class content). Groupware technology is developing further, and it will be useful to explore and do comparative assessments with other emergent technologies like augmented reality [86] and virtual reality [87, 88]. Virtual reality technologies are becoming more and more accessible. Visual analytics tasks may happen anytime, anywhere collaboratively without space and time constraints. Our next step is to conduct a similar study in immersive virtual reality environment, in which users will be non-co-located physically, but virtually co-present in a shared virtual workspace. We would also like to see how we can integrate AR with tablets to see how this coupling could benefit collaborative interaction with visual representations and extrapolate design principles for an AR-tablet integrated environment. For example, with virtual reality, we can explore the design of virtual shared/non-shared spaces, collaboration patterns, and new gesture based interactions with 3D objects [89,90,91,92]. Finally, our experiment involved pairs of participants. It will be interesting to explore if the patterns found in our study remain the same for 3 or even more users working together in co-located collaborative visual tasks.

Conclusion

This paper reports the results of an experiment that examines 2 $\times$ 3 groups of paired users who are required to work together to complete a set of analytical tasks using mobile tablets. We used a visualization tool that supports the exploration of the properties of 3D shapes as the testbed for our study, which investigated the influence that two factors, shared/non-shared control/view and position arrangements, have on users’ knowledge acquisition, engagement level, and learning efficiency. Our results show that paired users with a shared view have better performance when completing the set of tasks collaboratively. Shared view and control could also improve the learning and task efficiency. Additionally, the results show that the side-by-side (S-S) position leads to less time spent on collaborative tasks and greater perceived user preference. Overall, results show that the shared version used in the S-S position is suitable for supporting collaborative cross-tablet explorations of visual content.