Ontology mapping and semantics of web interface signs

Abstract

Although, intuitive interface signs are crucial elements of a good user interface (UI), prior research ignored these in UI design and usability evaluation process. This paper argues that ontology (the set of concepts and skills for understanding the referential meaning of an interface sign) mapping is critical for intuitive sign design. An experiment with fourteen participants and twelve signs (total 168 cases) has been carried out. The data has been analysed using both qualitative and quantitative approaches. The findings suggest that ontology mapping is important for understanding the semantics of interface signs. The paper concludes with design implications and future research plan.

Background

Every interactive application, especially web pages, incorporates a large extent of navigational links, labels, icons, symbols, short texts, thumbnails, command buttons, images, etc. [1]. These elements of a web user interface (UI) are called ‘interface signs’ (see Fig. 1). According to Peirce [2], each sign should have its own triadic relation that consists of (a) the representamen corresponding to the representation or form of a sign, (b) the object corresponding to the referential meaning or underlying functionality, and (c) the interpretant corresponding to the meaning (or a sign) generated in the mind of the interpreter or user. Thus, the key criteria to consider anything as a sign are: (1) it should have some meaning or stand for something else, and (2) it should be interpreted by someone. Based on this, a whole website (e.g. an e-commerce website) can be considered as a sign since a website as a whole can convey messages to its users; and a particular webpage (e.g. a product list of an e-commerce website) can also be a sign since a webpage as a whole can provide messages to the users. Similarly, small elements of a web page (e.g. a navigational link or label of product item) can be considered as signs since these elements convey meaning or functionality to the users.

Fig. 1

Retrieved from http://www.mist.ac.bd in August, 2016

Snapshot of Military Institute of Science and Technology (MIST) homepage shows some interface signs marked by ovals.

In this paper, we focus on the small elements of user interfaces that are defined as interface signs. For example (see Fig. 1), the small element ‘Contact’ at homepage of Military Institute of Science and Technology (MIST) refers to a unique meaning or functional message (i.e. shows the contact details of head of the departments, deans of the faculties, and other administration personnel) to users; thus ‘Contact’ can be considered as an interface sign. An interface sign can be a single sign like ‘Contact’, ‘Clubs’, ‘Home’, and ‘Facebook icon’. It can also take a form consisting texts and icon/symbols like ‘Important Notice with the icon’, ‘Academic with the icon’, etc.

The interface signs act as communication artefacts in web UI to convey information about web content and system functionality. End-users interact with user interfaces through interface signs, which make them crucial elements of web user interfaces. Users need to understand the meaning of the sign in question immediately. Therefore, designers should design interface signs to be intuitive for end-users, so that end-users can understand the referential meaning of interface signs accurately and perform their desired tasks properly. The user-intuitive interface sign is an interface sign that makes it easy and intuitive for end users to accurately understand its referential meaning [3]. According to Islam and Bouwman [3], a user intuitive interface sign needs to be transparent and reflect the actual content if it is to meet the expectations of users, allowing them to go directly to the content that is essential to them.

Additionally, interaction between users and web interfaces is mediated via interface signs since the content and functions of web systems are directed primarily through interface signs. Thus, at low level, users interpret the interface signs of user interfaces to understand the system’s logic and to perform tasks [4]. It is thus important to design user-intuitive interface signs so that users can interpret the meaning (semantic) of the interface signs easily and accurately. Understanding the meaning of an interface sign properly allows users go directly to the content of interest. Taken together, user-intuitive interface signs are critical for improving both ease of use and usefulness of a web application.

In information systems (IS) research, there has been an absence of research into the core design attributes that likely influence constructs like usefulness [5] and satisfaction [6]. This is surprising especially in light of the calls that such IT design attributes should be a focal area for IS research [7–9]. Consequently, our research focuses on a website’s core design attributes (i.e. the interface signs).

Indeed, prior research suggests designing user-intuitive interface signs and evaluating the intuitiveness of interface signs as essential in UI design and usability evaluation process. For example, Bolchini et al. [10] suggested interface sign as one of the major dimensions of web UI design and usability evaluation. A study by Islam and Tétard [11] shows that, if the interface signs are not intuitive, end-users spend a long time for finding the most suitable sign to obtain the information they are looking for. This leads to increased interaction variation, which may create usability problems and affect usefulness. However, very few studies explicitly focused on interface signs in UI design and evaluation [12–14].

A number of prior studies, conducted on interface signs focused on the importance of sign ontology [12, 13]. The term ontology is defined as the set of concepts and skills for understanding the referential meaning of an interface sign [15]. From the users’ perspective, ontology refers to the knowledge or concepts that users use to understand and interpret the meaning of an interface sign. From the designers’ perspective, ontology refers to the knowledge or concepts presupposed and referred to by an interface sign. We argue that the ontologies used by the users through which they interpret a sign may differ from the ontologies used by the designers when they design the system. For example, an interface sign Spam in an email application may be well designed in terms of colour, layout, position, etc. but will not make any sense to the users who do not know what the concept of Spam refers to. Another example, an interface sign Member in a museum website is designed for providing subscription-functionalities with the museum website to receive updates via email. It may happen that a user interprets the meaning of this Member sign as to be a member of the real-world museum to get some benefits while visiting the museum. Here, the ontology pointed by the interface sign (from designers perspective) is not matched with the ontology used by the users to interpret the meaning of this sign. Thus, the user interpretation to grasp the meaning of this sign will not be accurate.

In this paper, we show that if the users’ ontology (presupposed knowledge) is matched with the ontology referred to by an interface sign, the users understand the semantic (meaning) of an interface sign easily and accurately. This in turn sheds further light on why ontology mapping is important in relation to UI design and evaluation.

This paper is organized as follows. Previous works related to this study are discussed briefly in “Related work” section. In “Research model” section, the research model is presented. The study design is discussed in “Study design” section; that includes discussion on test participants and data collection procedure, data coding, the qualitative and quantitative analysis with the results. The discussion, implications and ideas for future work are presented in the final section.

Related work

This section provides a brief overview of previous related research on interface sign design and evaluation and then the related literature on ontology in relation to interface signs in UI design and evaluation.

A number of studies have been conducted for interface sign design and evaluation during the last decades. Previous studies on interface sign mainly strive to investigate which type of sign (icon, text-based, icon with text) works better to improve the ease-of-use of a software system. For example, according to Norman only icons are least well; icon with text signs works best, and only text sign works better than only icons (see [16]). Some other studies found that only text as well as only icon is not sufficient to design better web interfaces, while icons are generally complemented by a text label to provide hint to its referential or indexical meaning [17, 18]. Therefore, earlier research did not provide any definite suggestion to choose a specific type of sign for a specific context, e.g. some researcher suggested icon is better and some suggested icon with text label is better.

Some other studies have found several underlying factors that impact the recognition or understanding rate of interface sign. These include past experience [19], graphical representation [20–22], culture [23–25], metaphor [23, 25], cognitive features [25–27], and age [20, 21, 23]. However, a complete list of factors is still missing to design the understandable interface signs to improve the overall system usability.

Therefore, ‘ontology’, one of the key factors of interface sign design, is introduced in HCI to make them more intuitive for end users. Speroni [15] introduced the concept of ontology for information intensive websites. According to him, the most common ontologies used in information intensive websites are: InterLocutor/Institution Ontology (knowledge related to the owner of the website), Commonsense Ontology (knowledge belonging to the user’s background, and referring to common and everyday terms), Internet Ontology (knowledge related to the internet use), Web Domain Ontology (knowledge related to a specific web domain), Website Ontology (knowledge related to a particular website), and Topic Ontology (knowledge related to a specific topic or subject of a website), and Context Ontology (knowledge related to a specific context of interface sign). In another study, Speroni et al. [14] showed how the proposed ontologies could be used for evaluating the information intensive website using the expert inspection technique.

Later, Bolchini et al. [10] developed and tested a few heuristics (page-level heuristics and site level heuristics) to design and evaluate the web interface signs grounded on the Speroni’s [15] ontology. They also suggest that these heuristics could be used to conduct expert inspection to assess the usability of web interfaces. The significance of considering the concept of ontologies in the development process of information systems is broadly discussed by Sharp [28]. Islam et al. [12] analysed the interface signs of museum websites to explore users’ perceived difficulty in interpreting the meaning of interface signs that belong to different ontologies. They found that users’ interpretations difficulty varies with the ontology referred to by an interface sign, for example, the sign that belongs to Internet ontology are comparatively easier to interpret than the sign that belongs to website ontology for the users having average level of familiarity with all ontologies. In another study, Islam [29] showed that how re-designing an interface sign also changes the ontology referred to by an interface sign; and which in turn changes the users’ difficulty in interpreting the meaning of the interface signs. The prior work carried out on ontology is briefly presented in Table 1.

Table 1 Summary of prior research

The literature review suggests that very few studies have been conducted on interface sign ontology. The studies have been carried out by considering the Speroni’s [15] ontologies and focused on information intensive web interfaces. Speroni [15] proposed an example list of most common ontologies used in information intensive web UI. He further stated that the set of ontologies can be different depending on different websites. In response to this research opportunity, Islam [13] and Islam and Bouwman [30] proposed a total of twelve ontologies for web interface signs, and a set of principles of ontology mapping in interpreting the meaning of web interface signs. The set of ontologies identified by Islam [13] and Islam and Bouwman [30] is briefly presented in Table 2.

Table 2 List of ontologies proposed by Islam [13] and Islam and Bouwman [30]

In sum, the related work highlighted that several studies have been conducted on interface sign design, however, a very limited number of research are focused on the issue of ontology. Therefore, this study is focusing on the ontologies of interface signs to show why and how ontology mapping is important in relation to user interface design and evaluation.

Research model

This study adopts the ontologies presented by Islam [13] and Islam and Bouwman [30] for ontology mapping. In principle, this research carries out an experiment to show the effect of ontology mapping on understanding the meaning of web interface signs. We define three situations based on whether the designer’s ontology (ontology referred to by an interface sign) and user’s ontology is matched.

• Matched Ontology: Matched Ontology occurs when ontology/ontologies referred to by an interface signs matched with the one(s) owned by the users.

• Conflicted Ontology: Conflicted Ontology occurs when user’s ontology/ontologies and the one(s) referred to by an interface sign conflict with each other

• Unfamiliar Ontology: Unfamiliar Ontology occurs when user is not familiar with the ontology/ontologies referred to by an interface sign.

We argue that the intuitiveness of a sign depends on ontology mapping. For example, the meaning of the ‘HOME’ sign in a webpage refers to ‘getting the main/home page of a website’. A designer uses Internet Ontology when they include it in a website. If a user also uses Internet Ontology (Matched Ontology occurs), then he/she is expected to perceive the sign as intuitive. Alternatively, if the user uses Real World Ontology (Conflicted Ontology occurs), he/she may feel the sign less intuitive and make wrong interpretation by considering it as someone’s house. Finally, if the sign is not known to user (Unfamiliar Ontology occurs), he/she must feel the sign as less intuitive and fail to interpret it. This implies that Matched Ontology cases will have highest intuitiveness score, followed by Conflicted Ontology and Unfamiliar Ontology. To capture this, we propose the following hypotheses:

H1a

Users who experience Unfamiliar Ontology perceive lower intuitiveness than those who experience Ontology Conflict.

H1b

Users who experience Matched Ontology perceive higher intuitiveness than those who experience Ontology Conflict.

We also suggest that higher intuitiveness of an interface sign is associated with higher interpretation accuracy of understanding the meaning of interface signs. An intuitive sign allows users understand the referential meaning without clicking on it since it become transparent and reflects the actual content to users, allowing them to go directly to the content that is essential to them [4, 15, 31]. Consequently, we propose the following hypothesis:

H2

Intuitiveness is positively associated with Accuracy.

The overall research model with the above-mentioned hypotheses is shown in Fig. 2.

Fig. 2

Research model

Study design

Test participants and data collection procedure

We have conducted an experiment in a usability-testing lab at Åbo Academi University (ÅAU) and also in a post-graduation research lab at Military Institute of Science and Technology (MIST). We selected 12 interface signs (see Fig. 3) from home pages (English version) of two university websites (Universität Trier and RWTH Aachen University) and two museum websites (Drents Museum and Hunebed Centrum Museum). Then, we recruited 6 (5 undergrad and 1 graduate) students and 8 (5 undergrad and 3 graduate) students as test-participants to conduct the experiment at ÅAU and MIST, respectively. Participants were aged from 20–30 years (mean = 25.57; standard deviation = 2.77).

Fig. 3

Set of interface signs used in the experiment

We conducted a sign test with each participant via semi-structured interview. The following activities were followed in each test session with each participant: (1) test subjects filled a pre-test questionnaire that addressed their socio-biographical profile, experience with the use of computer, mobile and internet, as well as familiarity with the studied websites, (2) a short introduction was given to inform the test participants about the test in general (e.g., the test procedure, their role, etc.), and (3) the participants were asked to answer a set of questions for each interface sign (signs were presented within their original context during experiment). In practice, the participants were asked to answer the following questions without clicking on the signs.

• What could be the referential meaning of this sign?

• Why do you think this is the meaning of this sign?

• When interpreting this sign how intuitive are they to you (less intuitive, average intuitive or highly intuitive; score: 1(not intuitive)—9 (extremely intuitive))?

• Do you have any suggestion for redesigning this sign that would make the sign more intuitive in terms of interpreting its meaning?

Each session took about 25–35 min and was audio–video recorded. The interview data was transcribed and then analysed, first using qualitative analysis and then quantitative analysis using partial least squares (PLS) technique.

Data coding

We used the following technique for ontology mapping. First, we identified the ontology/ontologies referred to by the interface signs (designer’s ontology). Two researchers experienced in website design browsed the websites and listed the possible ontology referred to by a particular interface sign independently. For example, ‘Online shop’ refers to Internet Ontology. Then, the researchers met to compare the list of ontologies. The lists matched with each other with only a few disagreements. These disagreements were resolved by discussion. Second, we modelled the users’ profiles based on their familiarity with ontologies. A set of pre-test questionnaires were used to understand users’ familiarity with ontologies, e.g., a question ‘how long have you been using the internet?’ gave an indication of users familiarity with Internet Ontology. The model of the profiles of the test-participants is shown in Table 3. The users’ profiles along with the answer to the question “why do you think this is the meaning of this sign” helped us to code which ontology the users use to interpret a particular sign. Again, two researchers were involved in this coding process and the disagreements were resolved by discussion. The designer’s ontology along with the user’s ontology guided us categorize each case in one of the three ontology mapping groups (Matched Ontology, Conflicted Ontology, and Unfamiliar Ontology). At the end of the process, we got 66 instances of Matched Ontology, 48 instances of Unfamiliar Ontology, and 54 instances of Ontology Conflict.

Table 3 Model of the profiles of test-participants

The answer to the question “what could be the referential meaning of this sign” was coded in five levels: accurate (5), moderate (4), conflicting (3), erroneous (2), and incapable (1) by both researchers independently. The accuracy levels are presented in Table 4. Again, the disagreements were resolved by discussion.

Table 4 Definition of the accuracy levels

Qualitative analysis

The results of qualitative analysis showed that when participant’s familiarity with ontologies matched with ontologies referred to by an interface sign, then participants interpret the meaning of that sign accurately and experienced less difficulty to interpret. For example, the ontology/ontologies referred to by ‘Exhibition’, ‘FAQ’, ‘Online shop’, and ‘Departments’ signs (see Fig. 3) and the one(s) owned by them were matched. They interpret the meaning of this sign accurately and the mean intuitiveness scores for these signs were 8.29, 9.00, 8.03 and 9.00 respectively. ‘FAQ’ and ‘Online shop’ signs point to Internet Ontology and all participates were highly familiar with this ontology (see Table 3), thus they interpret these sign accurately. One participant responded the following while interpreting the meaning of ‘FAQ’ sign:

“…I know this sign. I have seen this sign almost all websites. This sign generally provides a set commonly asked questions with answers…”

The results also showed that when participants are unfamiliar with ontologies referred to by an interface sign, then participants interpret the meaning of that sign inaccurately and experienced high difficulty to interpret. For example, the ontology/ontologies referred to by ‘Megaliths’, ‘Lees Meer’ and ‘L²P’ signs (see Fig. 3) and the one(s) owned by them were not matched. They failed to interpret the meaning of this sign accurately and the mean intuitiveness scores for these signs were 1.71, 2.86 and 1.50 respectively. The term ‘Megaliths’ is a Greek word, meaning is large stones. Users having familiarity with this Greek word can understand the meaning of this sign. Similarly, the ‘Lees Meer’ is Dutch words; meaning is ‘Read More’. Thus, the sign ‘Megaliths’ and ‘Lees Meer’ in a museum website referred to Website Ontology and Cultural Ontology, respectively. The sign ‘L²P’ is a very specific sign to navigate users to the university’s learning and teaching portal. Thus, the sign ‘L²P’ in a university website belong to Website Ontology. The participants were not familiar with these ontologies (see Table 3), thus interpret the meaning of these signs inaccurately. One participant responded the following when interpreting the ‘Lees Meer’ sign:

“…I do not know this word; it might be written in a different language, so it is difficult for me to interpret …”

Furthermore, the qualitative analysis showed that when participants experienced ontology conflicts, they experienced comparatively (a) high difficulty to interpret the meaning of a sign with respect to the cases of ontology mapping, and (b) less difficulty to interpret the meaning of a sign with respect to the cases of unfamiliar ontology. For example, participants experienced ontology conflict for the ‘Students’, ‘Range of Courses’, ‘Other Target Groups’, ‘plus icon’ and ‘Peter Winandy’ signs in university websites (see Fig. 3). Most of the participants failed to interpret the meaning of these signs accurately and the mean intuitiveness scores for these signs were 3.14, 2.94, 3.86, 2.57, and 4.21, respectively. The sign ‘Range of Course’ actually stands for providing information of programs/degree offered (e.g., B.Sc. and M.Sc. in Psychology) in the university. But participants thought that the sign would provide the list of courses (e.g., object oriented programming language, data structure, etc.) offered in the university. They were also confused with the word ‘range’, as it could be used for different purposes like range of time or range of course fee. Here, this sign points to Website Ontology, since it is very specific to the website for this kind of information. However, participants treated the sign as built on Current Web Domain Ontology, System Ontology, and Common Sense Ontology to provide information about the list of courses offered in the university. Also, a few participants experienced ontology conflict to interpret the meaning of the ‘Students’ sign in a university website; because they thought that it could be either (1) for current students, because of the sign ‘Prospective Students’, which was placed to the left of this sign, or (2) for alumni students, because of the appended graduate hat icon. They thought this sign is referred to Current Web Domain Ontology, System Ontology, and Real World Ontology to provide information for the alumni or for students who will graduate soon. But the sign is actually referred to Website Ontology, since the ‘Students’ sign with the graduate hat is specific to this website for providing information for the current students. An ontology conflict thus occurred that led them to experience comparatively high difficulty to interpret the meaning of this sign.

Verification of structural model

In order to test the research model, we created two dummies using Ontology Conflict as the reference group. The partial least squares (PLS) analysis results are shown in Table 5.

Table 5 PLS model results

All three hypotheses are supported in this study. The results show that reported intuitiveness score is lower (β = −0.45, p < 0.001) for Unfamiliar Ontology than Conflicted Ontology (H1a supported). On the other hand, the reported intuitiveness score is higher (β = 0.55, p < 0.001) for Matched Ontology than Conflicted Ontology (H1b supported). Finally, intuitiveness (β = 0.77, p < 0.001) has significant influence on accuracy (H2 supported). The structural model explains 68 % variance of intuitiveness and 64 % variance of accuracy.

In sum, the analysis suggests that Matched Ontology cases have highest intuitiveness scores, followed by Conflicted Ontology and Unfamiliar Ontology. The analysis also suggests that intuitiveness of interface signs is positively associated with accuracy of interpreting the meaning of interface signs.

Discussions and implications

The findings clearly show the importance of matched ontology in relation to understanding the semantic of web interface signs. We provide two ontological guidelines for designing and evaluating the interface signs. First, we advise designers design interface signs based on users’ familiarity level with ontologies. For example, the word ‘Megaliths’ means large stones (referred to Website Ontology and Cultural Ontology from designers’ perspective). This sign in a museum website stands for providing its collection and exhibition information related to the large stones. This term was understandable only to a specific group of persons who have familiarity with this Greek word. One participant responded the following when he was interpreting the ‘Megaliths’ sign:

“…I do not know the meaning, without meaning how I judge it…”

The sign ‘Megaliths’ can be re-designed as ‘Large Stones’, ‘Mega Stones’, or ‘Big Stones’. In such a case the sign will point to Common Sense Ontology, Real World Ontology, and System Ontology. This in turn may improve the intuitiveness of the sign.

Second, we advise designers avoid ontology conflict when creating interface signs. For example, three participants found it difficult to interpret the meaning of the ‘Students’ sign in our experiment. They thought that it is either (1) for alumni students, because of the appended graduate hat icon, or (2) for current students, because of the sign ‘Prospective Students’, which was placed to the left of this sign. One participant responded:

“…This sign may be for alumni or those who will graduate…Oh! It may be for local students, as the neighbouring sign is for prospective students….I don’t know! It may be for alumni due to the graduation hat sign….”

Here, the sign is built on the Website Ontology, since the ‘Student’ sign with the graduate hat is specific to this website for providing information for the current student. The participants treated this sign as built on Current Web Domain Ontology, System Ontology, and Real World Ontology and assumed the sign would provide information for the alumni or for students who will graduate soon. They were familiar with this sign through the university websites, university, and real-world experience. As a result, an ontology conflict occurred that led the participants feel the sign as less intuitive.

Moreover, the study’s outcomes (i.e. two ontological guidelines) will contribute to advance practitioners’ knowledge to design and evaluate the web interface signs in order to improve the overall system usability and end user experience. The outcomes may assist practitioners: firstly, to investigate the problems and intuitiveness of interface signs based on users familiarity with ontologies; secondly, to redesign the less intuitive interface signs based on users familiarity with ontologies to make them more intuitive to end users; and finally, to apply the ontological guidelines as an integrated tool or concept with other usability evaluation process (e.g. heuristic inspection, user tests, etc.) in order to improve the usability evaluation.

This paper has a number of limitations that we plan to overcome in our future research. First, we have used a single item to measure intuitiveness in this study. Future research will focus developing a rigorous intuitiveness scale. Second, the current set of ontology and principles are derived for the desktop-based web interfaces. The ontologies and principles may not be directly applicable for mobile context. Thus, future work may focus on mobile interface, where signs have even more importance.