User experience framework that combines aspects, dimensions, and measurement methods

Abstract

Successful software products necessitate users’ satisfaction when experiencing the use of the software. This is not only determined by the software functionalities and completeness, but also with the overall user experience when using the software product. Although user experience is widely adopted by practitioners and in industry, there is no scientific consensus on a definition or a theoretical model of UX. The dynamic nature of user experience is challenging both UX design and evaluation activities. Accordingly, further research is needed to study four non-orthogonal UX issues: definition, modeling, method selection, and the interplay between evaluation and development. Moreover, UX professionals need to identify means for compromising the difficulties of evaluating UX in a holistic manner. The purpose of this research is to consolidate the findings related to UX aspects and dimensions along with the identified measurement methods into one simplified UX theoretical framework. This work is related to the aforementioned modeling issue aiming to better understand the relationship between UX dimensions, UX Aspects and UX measurement methods. The proposed framework is vital for practical application of UX, the development of UX evaluation methods and further theoretical studies of UX.

Keywords:

• human-computer interaction
• user experience
• evaluation
• framework
• systematic literature review

Public Interest Statement

Nowadays, software becomes an essential part of our lives. Successful software is determined by not only the software functionalities, but also by the overall user experience (UX) when using the software. Human-computer Interaction (HCI) has been shifted since 2000 toward measuring user experience and this formed the fifth generation of HCI domain.

Although user experience is widely adopted by practitioners and in industry, there is no scientific consensus on a definition or a theoretical model of UX. The dynamic nature of user experience is challenging both UX design and evaluation activities.

On the sense that an accepted and broadly shared understanding of UX is still lacking, this research aims to consolidate various UX concepts into one simplified UX theoretical framework. Such framework is vital for practical application of UX, the development of UX evaluation methods and for conducting further theoretical studies of UX.

1. Introduction

Software nowadays becomes an essential part of our lives; it is one of the most important delivered products or services that we use on daily bases. Software differs from any other product that we use; its intellectual nature makes it a developed, not a manufactured product and results in its having one of the most labor-intensive, complex, and error-prone products in human history (Kumaresh, 2012).

Many software engineers believe that software quality is not improving over time (Mann, 2002). Software users often experience code bloat, ugly, inefficient and poorly designed software resulting in software dysfunction (Ogheneovo, 2014). Because of its intellectual nature, users’ involvement in the software process becomes evident to achieve a better understanding of users’ needs (as in agile methodologies) and to result in successful products. Despite the importance of user involvement in the software process, successful software products necessitate users’ satisfaction when experiencing the use of the software. This is not only determined by the software functionalities and completeness, but also by the overall user experience (UX) when using the software product. UX is an emerging research area that is still immature (Law, Schaik, & Roto, 2012) and forms the fifth generation of HCI domain which has been shifted, since 2000, toward measuring user experience (Law, 2011; Yong, 2013). Although user experience is widely adopted by practitioners and in industry, there is no scientific consensus on its definition or a theoretical model of UX. This, for instance, has resulted in difficulties in classifying user requirements as pragmatic or hedonic. Even requirements engineers and UX professionals do not agree whether a user requirement focuses on pragmatic or hedonic quality (Kremer, Schlimm, & Lindemann, 2017).

User experience is a context-dependent, and subjective domain. It has been noted that users’ perception of different product qualities as well as emotions that arise before, during and after using a product is changing (Minge & Thüring, 2018) which makes UX a dynamic concept as well. Due to this fact, “the user experience is seen as something desirable without defining what something means”, and this has led to difficulty in agreeing on a user experience definition (Law, Roto, & Hassenzahl, 2009). The dynamic nature of user experience is challenging both UX design and evaluation activities (Kujala, Obrist, Vogel, & Pohlmeyer, 2013).

In literature, definitions for user experience were proposed, but those definitions were affected by the researcher’s background and interest (Law et al., 2009) and could not be used as a common definition. ISO 9241–210 defined user experience as “A person’s perceptions and responses that result from the use or anticipated use of a product, system or service” (ISO, 2010). This definition is a promising one as stated by Law et al. (2009). It is in line with the view by most researchers about the subjectivity of UX, but the used terms require more explanation to list possible objects that affect user experience (Law et al., 2009). User experience is tightly coupled with usability (Rajeshkumar, Omar, & Mahmud, 2013), and it can be conceptualized in three different ways (Bevan, 2009b). Below, we discuss and illustrate visually these three ways:^{Figure 1Figure 2Figure 3}

(a)	UX can be viewed as an elaboration of satisfaction component of usability (see Figure 1). Usability has different attributes such as efficiency, effectiveness, and satisfaction (Bevan, 2009a). Satisfaction attribute refers to the likely, pleasure, trust and other similar attributes. UX extends this usability attribute and from this perspective, the UX is considered as an elaboration of usability.
(b)	UX can be viewed as distinct from usability. Usability emphasizes objective measures, while user experience emphasizes subjective measures (see Figure 2). This differentiation is based on the usage of measurement methods. It is unnecessary that usability methods measure UX as there is a clear difference between usability characteristics and UX characteristics (Roto, Obrist, & Kaisa, 2009).
(c)	An umbrella term for users’ perceptions, whether measured objectively or subjectively (see Figure 3) follows ISO 9241–210 definition which covers perceptions from the user, either objectively or subjectively(Law et al., 2009).

Figure 1. UX is elaboration of usability.

Figure 2. Usability for objective and UX for subjective.

Figure 3. UX is an umbrella for everything.

The purpose of this research is to consolidate the findings related to UX aspects and dimensions discussed in (Zarour & Alharbi, 2017) along with the measurement methods documented in the literature, into one framework to better understand the relationship between UX dimensions, UX Aspects and UX measurement methods.

2. Literature review

UX has various concepts and terms that can be confusing for naïve readers. The use of UX terms is “partially confusing and there does not exist a clear framework that takes the different results into account” (Mahlke, 2005). This section aims to define these terms and synthesize findings of a conducted systematic literature review.

2.1. Conducted systematic literature review process

A systematic literature review to identify various UX terms has been conducted and published in (Zarour & Alharbi, 2017). Figure 4 summarizes the adopted systematic literature review process to select primary studies. A total of 114 primary studies out of 2,331 papers have been collected and analyzed, based on a defined set of inclusion/exclusion criteria. A list of primary studies that resulted from the systematic literature review is given in Appendix A.

Figure 4. SLR flow char.

The result of the analysis revealed various UX dimensions (see Table 5) and aspects (see Table 7). For more details about this phase, please refer to (Zarour & Alharbi, 2017). These findings are summarized in the subsequent sections and are used to develop the core UX framework as discussed in Section 4 of this paper.

2.2. UX perspectives

Perspective is the main element used to build UX definition (Law et al., 2009), and it will help to identify who has been affected by the changes of the UX. UX related perspectives are (Law et al., 2009):

(A)	Organization: Organizations aim to provide valuable products for the users.
(B)	User: User looks for a valuable and appropriate usage of the product.
(C)	The context of use: This has an impact on both elements (organization and user) and is the intermediate bridge between them.
(D)	Product: Focusing on how to design the product for better user experience.

Taking into consideration the main components of interaction between technology and user, namely: system or product, user, and the context of use (Jetter & Gerken, 2007; Moller, Engelbrecht, Kuhnel, Wechsung, & Weiss, 2009; Zhao & Balagué, 2014), the relationships between these components and UX perspectives is depicted in Figure 5. Note that the UX is a value-centric concept and all UX perspectives are related to the value.

Figure 5. UX Perspectives relationship.

Source: Zarour and Alharbi (2017).

2.3. UX disciplines and dimensions

Dimension is the category of a research discipline that has been followed in the primary studies to study UX. The main disciplines identified from the primary studies are (Zarour & Alharbi, 2017): Values & Experience, Brand & Experience, User Needs & Experience, Development Process & Experience and Technologies & Experience. UX dimensions suggested to cover these disciplines are (Zarour & Alharbi, 2017):

(A)	Value: This is related to the studies that have been focused on the gained value.
(B)	User Needs Experience (NX): This is related to the studies that have been focused on user needs and gained qualities.
(C)	Brand Experience (BX): This is related to the studies that have focused on the organization’s brand image.
(D)	Technology Experience (TX): This is related to the studies that have focused on the technology that has been used to deliver the product or the service.
(E)	Context: This is related to the studies that have focused on the context of use and the interaction between the previous dimensions.

Table 1, summarizes mapping between the suggested dimensions and the UX disciplines.

Table 1. Suggested dimensions for research disciplines (Zarour & Alharbi, 2017)

Discipline	Dimension
	Context
Values & Experience	Value & Overall Experience (VX)
Brand & Experience	Organization Brand Experience (BX)
User Needs & Experience	User Needs Experience (NX)
Development Process & Experience	Technology Experience (TX)
Technologies & Experience	Technology Experience (TX)

2.4. UX aspects

Aspect is the quality attribute that has an impact on UX. The aspects identified in the literature and their description is given in Table 2.

Table 2. UX aspects’ descriptions

UX aspect	Description
Branding (Jetter & Gerken, 2007)	Refers to the brand identity and the organization promises which have an impact on the trustworthiness
Everyday Operations (Jetter & Gerken, 2007)	Refers to the real business work such as selling the product or serving the customer, which has an effect on the usability of the service
Marketing (Jetter & Gerken, 2007)	Refers to communicating the brand identity and building an emotional link between the client and the organization
Business Communications (Jetter & Gerken, 2007)	Refers to communicating the business goals clearly inside the organization to construct a valuable product for the client which meets the organization values at the same time
Context of use (Moller et al., 2009)	Refers to the situation and the environment around the user when he is using the product or the service, and it is dynamic and could not be explicitly defined
Spatio-Temporal (Mercuri, 2005)	Refers to the time-space, period of usage between user and product as it has an impact on the learnability and usability
User Journey (Väänänen-Vainio-Mattila, Hassenzahl, Landau, & Fort, 2008)	Refers to the loyalty between the user and the organization as it has an impact on the emotions and trustworthiness
Cultural (Lee, Kim, & An, 2008)	Refers to the user’s culture, which has an impact on the usability and emotions as well
Emotional (Mercuri, 2005)	Refers to all emotions during the product usage, and is explicitly mentioned in some studies
Hedonic (Mahlke, 2005)	Refers to the category of all non-goals aspects, and is explicitly mentioned in some studies and use it as is
Trustworthiness (Mahlke, 2005)	Refers to the trust between the user and the organization during the usage of its products, and has an impact on different aspects such as emotions
Aesthetics (Mahlke, 2005)	Refers to the sense of beauty and how the user feels about it
Fun (Mahlke, 2005)	Refers to the enjoyment of using the product, and it has an intersection with emotional aspects
Privacy (Swallow, Blythe, & Wright, 2005)	Refers to the trust between the user and the product and how he is feeling safe and has an intersection with the trustworthiness
Sensual (Mercuri, 2005)	Refers to the experience derived from the sense when using the product
Usability (Mahlke, 2005)	Refers to the ease of use and learnability of using the product
Functionality (Mcnamara, 2005)	Refers to the state of being functional and can meet the goals of using the product
Usefulness (Mahlke, 2005)	Refers to the state of being useful
Platform Technology (Huy & Van Thanh, 2012)	Refers to the software technology that has been used to build the product, which has an impact on the usability and the emotions as well
Infrastructure (Wigelius & Väätäjä, 2009)	Refers to the hardware technology that has been used to build the product from both sides (client and Server) as it has an impact on the usability and it has interaction with Service Response Time aspect
Service Response time (Tung & Yuan, 2007)	Refers to the time that the organization takes to respond to the user by a valuable result, and it is mainly here related to the technology
Visual Attractiveness (Mahlke, 2005)	Refers to the user interface, and it has an impact on the usability and emotions as well
Development Process (Roto, Rantavuo, & Kaisa, 2009)	Refers to the development process and how it is covering the user’s needs and expectations

Taking into consideration the UX disciplines’ dimensions, the aspects that affect the UX can be categorized into one of the below types (Zarour & Alharbi, 2017):

•	User’s needs aspects: This is related to the user dimension and can be divided into two categories (Väätäjä, Koponen, & Roto, 2009) (Sproll, Peissner, & Sturm, 2010): - Pragmatic aspects: These are related to the usability and the do-goal qualities. - Hedonic aspects: These are related to the stimulation and be-goal qualities.
•	Brand aspects: Aspects related to the marketing and the business communications between the user and the organization and can be linked to the organization brand dimension.
•	Technology aspects: Aspects related to the development and production technologies that have an impact on the overall experience and can be related to the technology dimension. This category can be divided into four types: - UX designs (UXD) aspects: These are related to the UI designs. - Development technology aspects: These are related to the development technologies that are used in delivering the product. - Hardware aspects: These are related to the infrastructure of either the client side or the server side. - Operation aspects: These are related to the technologies that are used to monitor the production and response of the users.
•	The context of use aspects: Aspects related to any aspect that is not related to one of the previous aspects but has an impact on them, and this can be linked to the Context dimension.

From the previous classifications and the identified UX aspects from the primary studies, Table 3 illustrates the relationship between the UX aspects, categories and dimensions.

Table 3. Suggested categories for UX aspects and their relationship to UX dimensions (Zarour & Alharbi, 2017)

UX aspect	Aspect category	Dimension
Branding	Brand	Brand Experience (BX)
Everyday Operations	Brand	Brand Experience (BX)
Marketing	Brand	Brand Experience (BX)
Business Communications	Brand	Brand Experience (BX)
Context of use	Context	Brand Experience (BX)
Spatio-Temporal	Context	Brand Experience (BX)
User Journey	Context	Brand Experience (BX)
Cultural	Context	User Experience (UX)
Emotional	Hedonic	User Experience (UX)
Hedonic	Hedonic	User Experience (UX)
Trustworthiness	Hedonic	User Experience (UX)
Aesthetics	Hedonic	User Experience (UX)
Fun	Hedonic	User Experience (UX)
Privacy	Hedonic	User Experience (UX)
Sensual	Hedonic	User Experience (UX)
Usability	Pragmatic	User Experience (UX)
Functionality	Pragmatic	User Experience (UX)
Usefulness	Pragmatic	User Experience (UX)
Platform Technology	Development Technology	Technology Experience (TX)
Infrastructure	Hardware	Technology Experience (TX)
Service Response time	Operation	Technology Experience (TX)
Visual Attractiveness	UXD	Technology Experience (TX)

2.5. UX measurements methods

UX measurement method is a method to measure UX aspects and to get information about the fulfillment level of a certain aspect. The primary studies have been used to identify the different measurement methods used by researchers to measure UX aspects, either as a separate method or mixed with other measurement methods. In some cases, the same method is used to measure more than one aspect.

Table 4 shows the measurement methods and a number of papers that used it.

Table 4. Extracted UX measurement methods from the selected studies

Measurement method	Usage frequency	Measurement method	Usage frequency
Questionnaire	6	Experience Report (Open-ended Questions)	1
Interview	5	User’s Feedback	1
Experience report	5	Diary	1
Expert review	4	Valence Method	1
Attrakdiff	4	Expressing Experiences and Emotions (3E)	1
Electromyography responses (EMG)	3	Probe	1
Self-assessment manikin (SAM)	3	Fuzzy Cognitive Map (FCM)	1
Persona	3	UNeeQ	1
User profile	3	Survey	1
UX curve	2	System Usability Scale questionnaire (SUS)	1
Observation	2	Sentence Compaction Survey	1
PLEX	2	Emocards	1
Day reconstruction method	2	Emotion Words Priming List (EWPL)	1
Lean UX	1	PrEmo	1
Prototype	1	Group Experience Evaluation Method	1
BadIdeas	1	Subjective Usability Measurement Inventory (SUMI)	1
Electro dermal activity (EDA)	1	SUXES	1
Specification document	1	Attrak-Work	1
Think-Aloud	1	Focus Group	1
AHP	1	Experience Sampling Method (ESM)	1
Tracking real-time user experience (TRUE) method	1	Goal Question Metric (GQM)	1
DrawUX	1	Experience Evaluation Model (EEM)	1
UX cards	1	User Experience Metric(UXM)	1
UX concept testing	1	SPUX	1
iScale	1	Self-Expression Template Method	1

Note that some of the measurements methods are mentioned by the general name, such as questionnaire, interview, etc., while other methods are mentioned as part of the commercial name, such as Attrakdiff (Roto et al., 2009) and UNeeQ (Fronemann & Peissner, 2014) which are Questionnaires. The purpose of usage frequency is to understand the methods’ popularity along with the UX aspect. Table 5 summarizes the general UX measurement methods.

Table 5. UX measurement methods–general names

General measurement methods
• Questionnaire • Interview • Expert Review • Persona • Survey	• User profile • Observation • Prototype • Specification document • Focus Group	• Think-Aloud • Experience report • Users’ feedback • Diary

After reviewing the measurement methods in Table 5 and their use in the primary studies, a categorization has been identified according to the applicability of the method to the UX dimension in general. Table 6 shows the main measurement methods and the corresponding UX dimensions that can be used to measure. These measurement methods have been used recently to evaluate various UX aspects, see for example (Kremer et al., 2017).

Table 6. Suggested relationship between the UX measurement methods and the UX dimensions

Method	UX Dimensions
	NX	BX	TX
Questionnaire	√	√
Interview	√	√
Expert review	√		√
Persona	√	√
User profile	√	√
Observation	√
Prototype	√		√
Specification document			√
Think-aloud	√
Experience report	√	√
Users’ feedback	√	√
Diary		√
Survey	√	√
Focus group	√

For the Context dimension, no measurement methods are assigned, because the UX aspects in this dimension are not applicable for measurement.

2.6. UX Frameworks available in the literature

UX frameworks provide a theoretical background for a thorough understanding of what UX is. Researchers have synthesized their UX research findings in a UX framework. As can be seen from Table 7, different frameworks focused on certain UX dimensions and aspects based on selective literature; this resulted in having incomplete frameworks that provide a fractional image of the whole UX concept. For instance, Tan, Ronkko and Gencel (2013) have developed a UX framework that focuses on user needs experience dimension and its related aspects: hedonic and pragmatic. Their framework is based on a selected literature review and Goal Question Metric (GQM) paradigm. On the other hand, GAO (Changyuan, Shiying, & Chongran, 2013) have developed their framework to focus on user needs experience and Technology experience and their visual attractiveness, pragmatic and emotional aspects. Their framework is based on defining user experience under web environment.

Table 7. UX frameworks documented in the literature

Framework’s author	Year	Covered UX dimensions	UX aspects	Theoretical background
Mahlke (2005)	2005	User Needs Experience	Hedonic and Pragmatic	Selective literature
Vyas and Van Der Veer (2006)	2006	User Needs Experience	Hedonic and Pragmatic, User experience design	Domain of art is used to provide a metaphor to understand a user’s experience and design strategies
Jetter and Gerken (2007)	2007	User Needs Experience, Brand Experience	Hedonic, Pragmatic, Everyday operations, marketing, Business communication	Selective literature
Moller et al. (2009)	2009	User Needs Experience, Technology Experience	Hedonic and Pragmatic Visual attractiveness, Service Performance	Multimodal human-machine interactions used to develop a taxonomy of the most relevant QoS and QoE aspects that can be used for assessment and evaluation
Schulze and Krömker (2010)	2010	User Needs Experience	Hedonic and Pragmatic	Analyzed direct and indirect influencing factors of User Experience
Chen and Zhu (2011)	2011	User Needs Experience, Technology Experience	Hedonic, Pragmatic, User experience designs, Development technology, Hardware and Operation	Analytic Hierarchy Process (AHP) to study mobile application user experience and quantitative assessment
Gegner et al. (2011)	2011	Brand Experience Technology Experience User Needs Experience	Business communication, marketing, technology, and design, Usability	Selective literature and case study to identify UX in the context of product development in larger organizations
Fuchsberger, Moser, and Tscheligi (2012)	2012	Value Experience, User Needs Experience	Hedonic and Pragmatic	Theory of consumption values (TCV)
Gross and Bongartz (2012)	2012	User Needs Experience, Value Experience	Hedonic and Pragmatic (instrumental and non-instrumental qualities), Context	Used the components of user experience (CUE) model of Mahlke, and Thüring to hypothesize that the impact of perceived product attributes and emotional reactions on UX vary among different product types.
Anitha and Prabhu (2012)	2012	Technology Experience	Visual design	Discussions of some of the misconceptions and myths that exist in the industry regarding Requirements Engineering (RE) and User Experience Design (UXD)
Tan, Ronkko and Gencel (2013)	2013	User Needs Experience	Hedonic and Pragmatic	Selective literature review, Goal Question Metric (GQM)
GAO Changyuan et al. (2013)	2013	User Needs Experience, Technology Experience	Visual attractiveness, Pragmatic, Emotional	Defined the concept of info. platform user experience under Web environment, after reviewing some other similar frameworks
Kujala et al. (2013)	2013	Brand Experience	Temporal UX	Analyze the current state of the temporal aspect of UX, discuss definitions and theoretical models based on previous work
Kremer et al. (2017)	2017	User needs experience, technology experience	Hedonic, brand, user experience design, operation	Defined based on the ExodUX model (from Experience Oriented Disciplines to User experienced) consisting of 41 experiences from disciplines outside engineering design (e.g. sports)

Although various UX researchers have discussed various UX dimensions (see Table 3), none of them have developed a structured organization to understand UX aspects, dimensions, and measurement methods. It is stated explicitly by some researchers that their developed frameworks might not be fine-grained enough, and there can be a need to divide UX work into more unambiguous pieces, see for example (Gegner, Runonen, & Keinonen, 2011). Further aspects underlying the UX components need to be addressed to get a more holistic picture of the interaction between the different UX components and their relative importance for the overall UX, e.g. going beyond visual aesthetics for the non-instrumental qualities (Gross & Bongartz, 2012).

Accordingly, the work presented in this paper aims at handling this gap by identifying UX dimensions and aspects and link them to their corresponding measurement methods. Hence, the new framework specifications are given in Table 8.

Table 8. Proposed UX Framework

Framework’s author	Year	Covered UX dimensions	UX aspects	Theoretical background
Proposed framework	2018	Value, brand experience, user needs experience, technology experience	Hedonic, pragmatic, brand, user experience designs, development technology, hardware and operation	Systematic literature review to identify UX dimensions and aspects

3. Research methodology

The concept of user experience has been growing in the research community during the last two decades, and there is still no agreement about its nature and how to measure it (Gross & Bongartz, 2012). As stated by many studies in the literature, the “UX is dynamic, context-dependent, and subjective” (Law et al., 2009). This fact led to the need to scan this research area systematically to identify the possible dimensions that related to the UX and that have an impact on it.

Identifying these dimensions will build boundaries around the UX and will give directions on how to recognize the impact on the UX if it exists. Also, this may help to discover the methods that can be used to measure such an impact, or at least, understand its nature. Once identified, the dimensions that have an impact on the UX and the methods that have been used to measure them can be used to build s framework that covers these dimensions and methods to measure them.

Figure 6 shows the main stages of this research and the expected outcomes from each phase.

Figure 6. Research approach flow and outcomes.

(A)	Systematic Literature Review: Identify a set of selected primary studies which will be used to study UX aspects, dimensions and measurement.
(B)	Demographical Analysis: Analyse and illustrate demographical data about the topic and the selected primary papers.
(C)	Results Analysis: defines various UX concepts such as UX dimensions, aspects, and measurement methods. Moreover, in this phase, a more relational analysis will be conducted to relate findings to each other, and enhance our understanding of UX concepts.
(D)	Build a theoretical framework that synthesizes findings and explains the relation between them.
(E)	Report Results and outcomes in the form of research papers.

The first phase of this research has been completed and published in (Zarour & Alharbi, 2017) and a summary of this phase is given in Sections 2.1^2.2–2.3 as well as part of 2.4. The research work discussed in this paper focuses on developing the UX framework based on the main findings of phase one.

4. Proposed framework

In this section, the results of the extracted data from the literature review are analyzed aiming to synthesize all collected pieces of data into one theoretical framework.

Table 1 summarized the mapping between the suggested dimensions and the UX disciplines. Value dimension is the centric point for technology experience (TX), brand experience (BX) and user needs experience (NX) dimensions. Context relates all dimensions together and surrounding them. Accordingly, a relationship between these dimensions is built as shown in Figure 7.

Figure 7. The relation between value and UX dimensions.

In Section 2.4, we discussed the UX aspects and summarized the relationship between the UX aspects, their categories, and dimensions in Table 3. Adding the UX aspects to their dimensions shown in Figure 7 will give the result depicted in Figure 8.

Figure 8. UX aspects and dimensions.

The context dimension refers to the interaction with other dimensions, and it has the following aspects:

(A)	Spatio-temporal
(B)	User journey
(C)	Cultural
(D)	Context of use

Cultural aspect affects hedonic and pragmatic dimensions of UX, hence the culture can be related to User needs dimension. User Journey and Spatio-Temporal aspects are related to the period of experience that the user has with the product or the organization, hence these two aspects are related to the brand dimension. The aspect “Context of Use” is a general aspect and could not be related to a particular dimension. For the TN dimension, the interaction between the technology and context of use can be represented by the development process, hence context of use can be placed within the Technology context space as development process.

The relationship between UX aspects and their corresponding dimensions can be rebuilt again as depicted in Figure 9.

Figure 9. Relation between UX Aspects categories and UX dimensions.

Adding the remaining aspects described in Table 2 to their corresponding aspect’s category will extend Figure 9 as depicted in Figure 10.

Figure 10. Relation between UX aspects and all UX dimensions.

Figure 10 is updated by augmenting the measurement methods, given in Table 6, to each UX dimension, see Figure 11. The resulted framework covers the gathered UX dimensions, aspects and measurement methods. When compared to the various studies listed in Table 7, we can say that this framework is a comprehensive one that covered all identified UX dimensions: Value, Brand Experience, User Needs Experience, Technology Experience.

Figure 11. Proposed Framework that covers gathered UX dimensions, aspects categories, aspects, and measurement methods.

5. Conclusion & future work

UX is an emerging research area, that extends usability, and is still immature. The absence of a theoretical framework that defines and links together various UX dimensions, aspects and measurement methods resulted in disagreement between software engineers and practitioners in defining UX concepts. In this research, we analyzed UX dimensions that have an impact on the UX either during the development or awhile the product is in use. In addition to the UX dimensions, we scanned the relationship between those dimensions and UX aspects that have a direct or indirect impact on the UX. We also studied various UX measurement methods and their relation to UX dimensions.

All findings have been analyzed and consolidated in one framework that explains the relationship between UX dimensions, aspects and measurement methods. Although we cannot claim that the developed framework solves all conflicting issues related to UX concepts, but it is vital to understand UX related concepts and to develop UX evaluation methods. More work is required in the future to cover the following related issues:

(1)	Use a cumulative method to measure the UX from a long-term usage perspective, as the methods in the proposed framework are mainly for short-term usage. The UX Curve method is mentioned in literature and can be used in such cases and needs to be included in this framework (Kujala & Roto, 2011; Sahar, Varsaluoma, & Kujala, 2014).
(2)	More research is needed to link UX concepts, depicted in the proposed theoretical framework, the software quality standard known as SQuaRE (ISO25000 series) and the evaluation theory concepts to develop rigorous UX evaluation methods.
(3)	Build a relationship and integrate the proposed framework with a common maturity model such as CMMI. This will help to improve UX work level (Peres & Meira, 2015).
(4)	Identify the proper development process that brings more consistency between the proposed framework and the Development Process aspect that exists in the Context dimension. Currently, no specific process is suggested, while the Lean UX is proposed as a UX-centric development process (Liikkanen, Kilpiö, Svan, & Hiltunen, 2014) in literature.
(5)	For each UX measurement method, a template and general guidelines are required to assure that the questions and statements used in this method are covering all related aspects. This can be reached by identifying the goals that satisfy the aspect, and the Analytic Hierarchy Process (AHP) (Chen & Zhu, 2011) method can help in this case.
(6)	Empirical studies on the commercial UX methods to assure its applicability to measuring the different UX aspects. This will help to simplify the proposed framework and give directions about which method can be used.
(7)	More analysis in the Context dimension is needed to identify more UX aspects.

Funding

The authors received no direct funding for this research.

Additional information

Notes on contributors

Mohammad Zarour

Dr Zarour holds a PhD in Software Engineering (2009) from École de Technologie Supérieure (ETS)—Université du Québec (Montréal, Canada) and master degree in Computer Science (1998) from University of Jordan. He has more than 12 years of teaching and research experience in university and academic environment and also has several years of industry experience in information systems development and project management.

One of Dr Zarour’s research interests focuses on studying the effect of user expreience on the product quality. Current reasech in this domain aims to satisfy some needs in the academia and industry such as how to evalute user exeprince based SQuaRE standard and evaluation theory, how to integrate UX with CMMI, how to integrte UX with Agile development processes, and conduct more emprical studies and analysis in the UX domina to better undestand UX dimenssions and aspctes.

Appendix A

List of Primary Studies

1.	S. Mahlke, “Understanding users’ experience of interaction,” in EACE ‘05 Proceedings of the 2005 annual conference on European association of cognitive ergonomics, 2005, pp. 251–254.
2.	R. I. A. Mercuri, “Technology as Experience,” interactions - Funology, pp. 42–43, 2005.
3.	N. Mcnamara, “Functionality, Usability, and User Experience: Three Areas of Concern,” in International Journal (2005), 2005, pp. 26–28.
4.	K. Battarbee and I. Koskinen, “Co-Experience – User Experience as Interaction,” CoDesign, No. 1.1, pp. 5–18, 2005
5.	D. Swallow, M. Blythe, and P. Wright, “Grounding Experience: Relating Theory and Method to Evaluate the User Experience of Smartphones,” in EACE ‘05 Proceedings of the 2005 annual conference on European association of cognitive ergonomics, 2005, pp. 91–98
6.	Z. Pei and Z. Zhenxiang, “A Framework for Personalized Service Website based on TAM,” in Service Systems and Service Management, 2006 International Conference on, 2006, pp. 1598–1603
7.	D. Vyas and G. C. Van Der Veer, “Experience as Meaning: Some Underlying Concepts and Implications for Design,” in Proceedings of the 13th European conference on Cognitive ergonomics: trust and control in complex socio-technical systems. ACM.‏, 2006
8.	J. Schrammel, A. Geven, and M. Tscheligi, “Using Narration to Recall and Analyse User Experiences and Emotions Evoked by Today’s Technology,” in International Conference on Design and Emotion, ‏, 2006
9.	S. N. Junaini, “Towards a Framework for the Assessment of Online Store Product Catalogs,” in Information and Communication Technologies, 2006. ICTTA ‘06. 2nd, 2006, pp. 245–250.
10.	W.-F. Tung and S.-T. Yuan, “Optimization of Collaborative Service Systems Using Experience Evaluation Model,” in Services Computing, 2007. SCC 2007. IEEE International Conference on, 2007, pp. 244–247
11.	Y. S. Lee, B. M. Kleiner, P. D, And M. A. Nussbaum, “Older Adults ‘ User Experiences With Mobile Phones: Identification Of User Clusters And User Requirements,” 2007.
12.	J. Holt and S. Lock, “MARPLE Investigates: An ‘Adversarial’ Approach to Evaluating User Experience,” in BCS-HCI ‘07 Proceedings of the 21st British HCI Group Annual Conference on People and Computers, 2007, pp. 141–144.
13.	S. Mahlke and M. Thüring, “Studying antecedents of emotional experiences in interactive contexts,” in Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ‘07, 2007, p. 915
14.	H. Jetter and J. Gerken, “A Simplified Model of User Experience for Practical Application,” in 2nd International Open Workshop on COST294-MAUSE (2006), 2007, pp. 2–3.
15.	L. Patrício, R. P. Fisk, and J. F. E. Cunha, “Designing Multi-Interface Service Experiences The Service Experience Blueprint,” J. Serv. Res. No. 10.4, pp. 318–334‏, 2008
16.	Y. Lim, J. Donaldson, H. Jung, B. Kunz, D. Royer, S. Ramalingam, S. Thirumaran, and E. Stolterman, “Emotional Experience and Interaction Design,” in Affect and Emotion in Human-Computer Interface Design (2008), 2008, pp. 116 – 129
17.	M. Hassenzahl, “User experience (UX): towards an experiential perspective on product quality,” in Proceedings of the 20th International Conference of the Association Francophone d’Interaction Homme-Machine on - IHM ‘08, 2008, pp. 11–15.
18.	I. Lee, J. Kim, and Y. An “Cultural Dimensions for User Experience : Cross-Country and Cross-Product Analysis of Users’ Cultural Characteristics,” in British Computer Society Conference on Human-Computer Interaction (2008), 2008, pp. 3–12.
19.	P. Ketola and V. Roto, “Exploring User Experience Measurement Needs,” in Proc. of the 5th COST294-MAUSE Open Workshop on Valid Useful User Experience Measurement (VUUM).‏, 2008, pp. 23–26.
20.	E. Karapanos, M. Hassenzahl, and J.-B. Martens, “User experience over time,” in Proceeding of the twenty-sixth annual CHI conference extended abstracts on Human factors in computing systems - CHI ‘08, 2008, p. 3561.
21.	A. Joshi and S. Tripathi, “User Experience Metric and Index of Integration: Measuring Impact of HCI Activities on User Experience,” in I-USED‏,2008
22.	A. Ahtinen, S. Ramiah, J. Blom, and M. Isomursu, “Design of mobile wellness applications: identifying cross-cultural factors,” in OZCHI ‘08 Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat, 2008, pp. 164–171.
23.	K. Väänänen-Vainio-Mattila, M. Hassenzahl, C. Landau, and I. Fort, “Towards Practical User Experience Evaluation Methods,” in 5th COST294-MAUSE Open Workshop on Valid Useful User Experience Measurement (2008), 2008, pp. 1–4.
24.	Z. Wang, X. Zhou, H. Wang, H. Ni, and R. Wu, “A Quantitative Evaluation Model of Group User Experience,” in Computational Intelligence and Industrial Application, 2008. PACIIA ‘08. Pacific-Asia Workshop on, 2008, pp. 918–923.
25.	E. L. Law, V. Roto, and M. Hassenzahl, “Understanding, Scoping and Defining User eXperience: A Survey Approach,” in CHI ‘09 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009, pp. 719–728.
26.	V. Roto, M. Obrist, and K. Väänänen-Vainio-Mattila, “User Experience Evaluation Methods in Academic and Industrial Contexts,” in Interact 2009 conference, User Experience Evaluation Methods in Product Development (UXEM’09).‏, 2009.
27.	H. Petrie and C. Harrison, “Measuring users’ emotional reactions to websites,” in Proceedings of the 27th international conference extended abstracts on Human factors in computing systems - CHI EA ‘09, 2009, p. 3847.
28.	V. Roto, H. Rantavuo, And V. Kaisa, “Evaluating User Experience Of Early Product Concepts,” In Proc. Dppi.‏, 2009 P. 9.
29.	H. Wigelius and H. Väätäjä, “Dimensions of Context Affecting User Experience in Mobile Work,” in INTERACT ‘09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II, 2009, pp. 604–617.
30.	H. Väätäjä, T. Koponen, and V. Roto, “Developing Practical Tools for User Experience Evaluation: A Case from Mobile News Journalism,” in ECCE ‘09 European Conference on Cognitive Ergonomics: Designing beyond the Product – Understanding Activity and User Experience in Ubiquitous Environments, 2009, p. Article No. 23.
31.	A. Thayer and T. E. T. E. Dugan, “Achieving design enlightenment: Defining a new user experience measurement framework,” in Professional Communication Conference, 2009. IPCC 2009. IEEE International, 2009, pp. 1–10.
32.	K. Väänänen-Vainio-Mattila and M. Wäljas, “Developing an Expert Evaluation Method for User eXperience of Cross-platform Web Services,” in Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era, 2009, pp. 162–169.
33.	E. Karapanos, J. Zimmerman, J. Forlizzi, and J.-B. Martens, “User Experience Over Time: An Initial Framework,” in CHI ‘09 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2009, pp. 729–738.
34.	S. Wang, “Emotional design based on customer experience requirement,” in Computer-Aided Industrial Design & Conceptual Design, 2009. CAID & CD 2009. IEEE 10th International Conference on, 2009, pp. 158–161.
35.	S. Moller, K.-P. Engelbrecht, C. Kuhnel, I. Wechsung, and B. Weiss, “A taxonomy of quality of service and Quality of Experience of multimodal human-machine interaction,” in 2009 International Workshop on Quality of Multimedia Experience, 2009, pp. 7–12.
36.	N. Bevan, “What is the difference between the purpose of usability and user experience evaluation methods ?,” in INTERACT 2009, 2009.
37.	H. Petrie and N. Bevan, “The evaluation of accessibility, usability, and user experience,” in The Universal Access Handbook (2009), 2009, pp. 10–20.
38.	T. Walsh, P. Nurkka, and R. Walsh, “Cultural differences in smartphone user experience evaluation,” in Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia - MUM ‘10, 2010, pp. 1–9.
39.	K. Väänänen-Vainio-Mattila, H. Korhonen, and J. Arrasvuori, “Let Users Tell the Story: Evaluating User Experience with Experience Reports,” in CHI EA ‘10 CHI ‘10 Extended Abstracts on Human Factors in Computing Systems, 2010, pp. 4051–4056.
40.	P. Silva and J. Read, “A methodology to evaluate creative design methods: a study with the BadIdeas method,” in OZCHI ‘10 Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction, 2010, pp. 264–271.
41.	K. Schulze and H. Krömker, “A framework to measure user experience of interactive online products,” in Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research - MB ‘10, 2010, pp. 1–5.
42.	Q. Yan, “Cultural versioning of mobile user experience,” in Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems - CHI EA ‘10, 2010, p. 2943.
43.	A. P. O. S. Vermeeren, E. L. Law, and V. Roto, “User Experience Evaluation Methods: Current State and Development Needs,” in Proceedings of the 6th Nordic Conference on Human-Computer Interaction Extending Boundaries - NordiCHI ‘10 (2010), 2010, pp. 521–530
44.	Y.-L. Huang, T. Marsh, and A. D. Cheok, “Investigation of software patterns of user experience,” in Proceedings of the 7th International Conference on Advances in Computer Entertainment Technology - ACE ‘10, 2010, p. 116.
45.	J. Arrasvuori, H. Korhonen, and K. Väänänen-Vainio-Mattila, “Exploring playfulness in user experience of personal mobile products,” in Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction - OZCHI ‘10, 2010, p. 88.
46.	S. Sproll, M. Peissner, and C. Sturm, “From product concept to user experience: exploring UX potentials at early product stages,” in NordiCHI ‘10 Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, 2010, pp. 473–482.
47.	M. Burmester, M. Mast, K. Jäger, and H. Homans, “Valence method for formative evaluation of user experience,” in Proceedings of the 8th ACM Conference on Designing Interactive Systems - DIS ‘10, 2010, p. 364.
48.	K. Rodden, H. Hutchinson, and X. Fu, “Measuring the User Experience on a Large Scale : User-Centered Metrics for Web Applications,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2010), 2010.
49.	Wei Liu, Daoli Huang, and Yan Zhang, “Research on fuzzy comprehensive evaluation of user experience,” in Information Computing and Telecommunications (YC-ICT), 2010 IEEE Youth Conference on, 2010, pp. 122–125.
50.	H. Korhonen, J. Arrasvuori, and K. Väänänen-Vainio-Mattila, “Analysing User Experience of Personal Mobile Products Through Contextual Factors,” in Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia, 2010, pp. 11:1–11:10.
51.	C. Tselios, I. Politis, M. Tsagkaropoulos, and T. Dagiuklas, “Valuing quality of experience: A brave new era of user satisfaction and revenue possibilities,” in FITCE Congress (FITCE), 2011 50th, 2011, pp. 1–6
52.	J. Bargas-Avila and K. Hornbæk, “Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience,” in CHI ‘11 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011, pp. 2689–2698.
53.	C. Bach, N. Gauducheau, and P. Salembier, “Combining interviews and scales in the multidimensional evaluation of user experience: a case study in 3D games,” in ECCE ‘11 Proceedings of the 29th Annual European Conference on Cognitive Ergonomics, 2011, pp. 157–160.
54.	C. Mulwa, S. Lawless, M. Sharp, and V. Wade, “The evaluation of adaptive and personalised information retrieval systems: a review,” in International Journal of Knowledge and Web Intelligence (2011), 2011, vol. 2, pp. 138–156.
55.	L. Gegner, M. Runonen, and T. Keinonen, “Oscillating between extremes: a framework for mapping differing views on User eXperience,” in DPPI ‘11 Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces, 2011, p. Article No. 57.
56.	M. Obrist, E. Beck, D. Wurhofer, and M. Tscheligi, “Experience characters: a design tool for communicating mobile phone experiences to designers,” in MobileHCI ‘11 Proceedings of the 13th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2011, pp. 385–394.
57.	V. Roto, H. Väätäjä, S. Jumisko-Pyykkö, and K. Väänänen-Vainio-Mattila, “Best practices for capturing context in user experience studies in the wild,” in Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek ‘11, 2011, p. 91.
58.	Zhi Chen and Shangshang Zhu, “The research of mobile application user experience and assessment model,” in Computer Science and Network Technology (ICCSNT), 2011 International Conference on, 2011, pp. 2832–2835.
59.	S. Kujala and V. Roto, “Identifying hedonic factors in long-term user experience,” in DPPI ‘11 Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces, 2011, p. 17.
60.	E. L.-C. Law, “The measurability and predictability of user experience,” in Proceedings of the 3rd ACM SIGCHI symposium on Engineering interactive computing systems - EICS ‘11, 2011, p. 1.
61.	R. De Oliveira and N. Oliver, “Influence of Usability on Customer Satisfaction : A Case Study on Mobile Phone Services,” in NordiCHI 2012, 2012, No. September.
62.	V. C. Gerogiannis, S. Papadopoulou, and E. I. Papageorgiou, “A Fuzzy Cognitive Map for Identifying User Satisfaction from Smartphones,” in Informatics (PCI), 2012 16th Panhellenic Conference on, 2012, pp. 156–161.
63.	A. Gross and S. Bongartz, “Why do I like it?: investigating the product-specificity of user experience,” in NordiCHI ‘12 Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, 2012, pp. 322–330.
64.	V. Milanova, T. Mandl, and R. Kölle, “Design for emotion: a case study,” in MobileHCI ‘12 Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services companion, 2012, pp. 59–64.
65.	T. Jokela, “The early phases of UX: Why they are important (more than evaluation), and what they are?,” in NordiCHI 2012, 2012.
66.	I. Moczarny, M. Villiers, and J. Biljon, “How can usability contribute to user experience? A study in the domain of e-commerce,” in SAICSIT ‘12 Proceedings of the South African Institute for Computer Scientists and Information Technologists Conference, 2012, pp. 216–225.
67.	K. Kunze and D. Strohmeier, “Examining subjective evaluation methods used in multimedia Quality of Experience research,” in Quality of Multimedia Experience (QoMEX), 2012 Fourth International Workshop on, 2012, pp. 51–56.
68.	J. Varsaluoma and V. Kentta, “Drawux: a web-based research tool for long-term user experience evaluation,” in NordiCHI ‘12 Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, 2012, pp. 769–770.
69.	N. P. Huy and D. VanThanh, “Evaluation of mobile app paradigms,” in Proceedings of the 10th International Conference on Advances in Mobile Computing & Multimedia - MoMM ‘12, 2012, p. 25.
70.	M. Q. Abbasi, J. Weng, Y. Wang, I. Rafique, X. Wang, and P. Lew, “Modeling and Evaluating User Interface Aesthetics Employing ISO 25,010 Quality Standard,” in Quality of Information and Communications Technology (QUATIC), 2012 Eighth International Conference on, 2012, pp. 303–306.
71.	P. Anitha and B. Prabhu, “Integrating requirements engineering and user experience design in Product life cycle Management,” in 2012 First International Workshop on Usability and Accessibility Focused Requirements Engineering (UsARE), 2012, pp. 12–17.
72.	C. Jian, L. Wen-wang, Z. Wen-an, G. Hai-sheng, C. Zhang, and M. Shao-fu, “Customer experience oriented service quality management,” in Robotics and Applications (ISRA), 2012 IEEE Symposium on, 2012, pp. 298–301.
73.	S. Luojus, “Integrating Momentary and Long-term UX: A Theoretical Approach,” in OzCHI ‘12 Proceedings of the 24th Australian Computer-Human Interaction Conference, 2012, pp. 353–356.
74.	D. Buskermolen, J. Tarken, and B. Eggen, “Informing User Experience Design About Users: Insights from Practice,” in CHI EA ‘12 CHI ‘12 Extended Abstracts on Human Factors in Computing Systems, 2012, pp. 1757–1762.
75.	V. Fuchsberger, C. Moser, and M. Tscheligi, “Values in action (ViA): combining usability, user experience and user acceptance,” in CHI EA ‘12 CHI ‘12 Extended Abstracts on Human Factors in Computing Systems, 2012, pp. 1793–1798.
76.	Z. Hasan, R. Gope, and M. Nasir Uddin, “Do Aesthetics Matter in Long-Established Trust?,” in International Journal of Computer Applications, 2013, p. 77.
77.	M. Nivethika, I. Vithiya, S. Anntharshika, and S. Deegalla, “Personalized and adaptive user interface framework for mobile application,” in Advances in Computing, Communications and Informatics (ICACCI), 2013 International Conference on, 2013, pp. 1913–1918.
78.	X. Sun and A. May, “A Comparison of Field-Based and Lab-Based Experiments to Evaluate User Experience of Personalised Mobile Devices,” in Advances in Human-Computer Interaction, 2013, pp. 1–9.
79.	S. Kujala and T. Miron-Shatz, “Emotions, experiences, and usability in real-life mobile phone use,” in CHI ‘13 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2013, pp. 1061–1070.
80.	M. Salehi, M. Salimi, and A. Haque, “The Impact of Online Customer Experience (OCE) on Service Quality in Malaysia,” World Appl. Sci. J. pp. 1621–1631, 2013.
81.	M. Vogel, “Temporal evaluation of aesthetics of user interfaces as one component of user experience,” in AUIC ‘13 Proceedings of the Fourteenth Australasian User Interface Conference, 2013, pp. 131–132.
82.	S. Hedegaard and J. G. Simonsen, “Extracting usability and user experience information from online user reviews,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ‘13, 2013, p. 2089.
83.	Gao Changyuan, Wang Shiying, and Zhong Chongran, “Research on user experience evaluation system of information platform based on web environment,” in Measurement, Information and Control (ICMIC), 2013 International Conference on, 2013, pp. 558–562.
84.	L. Arhippainen, “A Tutorial of Ten User Experience Heuristics,” in MindTrek, 2013, pp. 336–337.
85.	D. Raptis, N. Tselios, J. Kjeldskov, and M. Skov, “Does size matter ? Investigating the impact of mobile phone screen size on users ‘perceived usability, effectiveness and efficiency,” in Proceedings of the international conference on Human-computer interaction with mobile devices and services (MobileHCI ‘13) (2013), 2013, pp. 127–136.
86.	C. Von Saucken, C. von Saucken, I. Michailidou, and U. Lindemann, “Emotional mental model,” in Industrial Engineering and Engineering Management (IEEM), 2013 IEEE International Conference on, 2013, pp. 802–806.
87.	L. Yong, “User experience evaluation methods for mobile devices,” in Innovative Computing Technology (INTECH), 2013 Third International Conference on, 2013, pp. 281–286.
88.	J. Tan, K. Ronkko, and C. Gencel, “A Framework for Software Usability and User Experience Measurement in Mobile Industry,” in Software Measurement and the 2013 Eighth International Conference on Software Process and Product Measurement (IWSM-MENSURA), 2013 Joint Conference of the 23rd International Workshop on, 2013, pp. 156–164.
89.	S. Kujala, M. Obrist, M. Vogel, and A. Pohlmeyer, “Lost in Time: The Meaning of Temporal Aspects in User Experience,” in CHI ‘13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA ‘13 (2013), 2013, pp. 559–564.
90.	M. Winckler, C. Bach, and R. Bernhaupt, “Identifying User Experience Dimensions for Mobile Incident Reporting in Urban Contexts,” IEEE Trans. Prof. Commun., pp. 97–119, Jun. 2013.
91.	A. Sonnleitner, M. Pawlowski, T. Kässer, and M. Peissner, “Experimentally Manipulating Positive User Experience Based on the Fulfilment of User Needs,” in Human-Computer Interaction–INTERACT 2013.‏, 2013, pp. 555–562.
92.	J. Mashapa, E. Chelule, D. Van Greunen, and A. Veldsman, “Managing User Experience – Managing Change,” in Human-Computer Interaction–INTERACT 2013.‏, 2013, pp. 660–677.
93.	S. Wirtz and E. Jakobs, “Improving User Experience for Passenger Information Systems. Prototypes and Reference Objects,” IEEE Trans. Prof. Commun., pp. 120–137, 2013.
94.	T. Keskinen, J. Hakulinen, T. Heimonen, M. Turunen, S. Sharma, T. Miettinen, and M. Luhtala, “Evaluating the experiential user experience of public display applications in the wild,” in Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia - MUM ‘13, 2013, pp. 1–10.
95.	R. Jain, J. Bose, and T. Arif, “Contextual adaptive user interface for Android devices,” in India Conference (INDICON), 2013 Annual IEEE, 2013, pp. 1–5.
96.	S. Rajeshkumar, R. Omar, and M. Mahmud, “Taxonomies of User Experience (UX) evaluation methods,” in Research and Innovation in Information Systems (ICRIIS), 2013 International Conference on, 2013, pp. 533–538.
97.	L. Arhippainen and M. Pakanen, “Utilizing Self-Expression Template Method in User Interface Design - Three Design Cases,” in Proceedings of International Conference on Making Sense of Converging Media - AcademicMindTrek ‘13, 2013, pp. 80–86.
98.	C. L. Paul and A. Komlodi, “Measuring user experience through future use and emotion,” in Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA ‘14, 2014, pp. 2503–2508.
99.	T. Yamakami, “Exploratory analysis of differences between social experience design and user experience design,” in Advanced Communication Technology (ICACT), 2014 16th International Conference on, 2014, pp. 769–773.
100.	A. Aggarwal, G. Niezen, and H. Thimbleby, “User experience evaluation through the brain’s electrical activity,” in Proceedings of the 8th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational - NordiCHI ‘14, 2014, pp. 491–500.
101.	E. L. Law and P. van Schaik, “To Measure or Not to Measure UX: An Interview Study,” in I-UxSED, 2014, pp. 58–63.
102.	N. Fronemann and M. Peissner, “User experience concept exploration: user needs as a source for innovation,” in NordiCHI ‘14 Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, 2014, pp. 727–736.
103.	C.-C. Huang and E. Stolterman, “Temporal anchors in user experience research,” in Proceedings of the 2014 conference on Designing interactive systems - DIS ‘14, 2014, pp. 271–274.
104.	J. Varsaluoma and F. Sahar, “Usefulness of long-term user experience evaluation to product development: practitioners’ views from three case studies,” in NordiCHI ‘14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, 2014, pp. 79–88.
105.	R. Alves, P. Valente, and N. J. Nunes, “The state of user experience evaluation practice,” in Proceedings of the 8th Nordic Conference on Human-Computer Interaction Fun, Fast, Foundational - NordiCHI ‘14, 2014, pp. 93–102.
106.	Y. Li and X. Wang, “Mobile interface studies about style description and influential factors,” in Management Science & Engineering (ICMSE), 2014 International Conference on, 2014, pp. 578–583
107.	C. Lallemand, V. Koenig, and G. Gronier, “How Relevant is an Expert Evaluation of User Experience based on a Psychological Needs-Driven Approach?,” in NordiCHI ‘14 Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, 2014, pp. 11–20.
108.	C. M. Macdonald and M. E. Atwood, “What Does it Mean for a System to be Useful? An Exploratory Study of Usefulness,” in Designing interactive systems (2014), 2014, pp. 885–894.
109.	N. Seyff, G. Ollmann, and M. Bortenschlager, “AppEcho: a user-driven, in situ feedback approach for mobile platforms and applications,” in Proceedings of the 1st International Conference on Mobile Software Engineering and Systems - MOBILESoft 2014, 2014, pp. 99–108.
110.	C. B. Zhenzhen Zhao, “A Design Framework of Branded Mobile Applications,” in MobileHCI ‘14 Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services, 2014, pp. 507–512.
111.	L. Liikkanen, H. Kilpiö, L. Svan, and M. Hiltunen, “Lean UX - The Next Generation of User-Centered Agile Development?,” in NordiCHI ‘14 Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, 2014, pp. 1095–1100.
112.	F. Sahar, J. Varsaluoma, and S. Kujala, “Comparing the Effectiveness of Electronic Diary and UX Curve Methods in Multi-Component Product Study,” in AcademicMindTrek ‘14: Proceedings of the 18th International Academic MindTrek Conference: Media Business, Management, Content & Services, 2014, pp. 93–100.
113.	T. Walsh, S. Kujala, H. Petrie, and C. Power, “Axe UX : Exploring Long-Term User Experience with iScale and AttrakDiff,” in AcademicMindTrek ‘14: Proceedings of the 18th International Academic MindTrek Conference: Media Business, Management, Content & Services, 2014, pp. 32–39.
114.	V. Carofiglio, G. Ricci, and F. Abbattista, “User brain-driven evaluation of an educational 3D virtual environment,” in Information Systems and Technologies (CISTI), 2015 10th Iberian Conference on, 2015, pp. 1–7.