Towards a User Experience Framework for Business Intelligence

ABSTRACT

Business intelligence (BI) systems are software applications that are used to gather and process data and to deliver the processed data in understandable way to the end users. With a younger generation of users moving into key positions in organizations and enterprises higher user experience (UX) demands are placed on BI systems interfaces. Companies developing BI systems lack standardized routines for implementing UX in their BI solutions. The purpose of this study was to develop a theoretical framework based on existing research and combine it with empirical data gathered from professionals in BI systems industry in Sweden with the intention of proposing a UX framework applicable to BI systems development. The study resulted in a framework being developed using iterative build-evaluate iterations. The framework is a scalable UX framework for BI systems interfaces covering areas from planning and strategizing to implementation, maintenance, and evaluation.

KEYWORDS:

• User experience
• business intelligence
• framework

1. Introduction

Business intelligence (BI) systems are software applications intended to assist in making informed strategic business decisions for companies and organizations. Companies that are utilizing data analysis through use of BI systems have a good chance of increased competitiveness in their industry.¹ The BI systems are used to process data that enables the end user to interpret and act on the information received.² BI systems are used for a number of purposes that assist companies in making strategic decisions and are intended for increasing productivity, lowering costs, development of the company, improving customer relations and strategic business decisions in general, assumptions are also intended to be excluded from business decisions with the use of BI systems.²

Since BI systems are used to make informed strategic business decisions it is important that the information available through the interfaces of such systems is accessible to the user and that the applications are easy to use. The functionality, presentation, performance, interactivity, and usability of a software system affects how the user perceives the user experience (UX) of the system. UX puts focus on the end users perspective while interacting with interactive software products.³ The International Organization for Standardization (ISO) defines UX as the end users perception of use when interacting with a product. UX includes aspects such as emotions, physical and mental reactions. Previous experiences, expectations, and goals for using the system also impact the users perceived experience.⁴

UX factors (e.g., usability and end user satisfaction) in BI systems are of significant importance since the systems are used for strategic business decision-making.^1,5 Jooste, Biljon, and Botha,¹ state that BI systems are an important tool for decision-making in companies and that research conducted on the area is limited, especially UX in context with BI systems. They also mention that good usability in BI systems increases the competitiveness of companies and increase the quality of decision-making.

The previously mentioned research indicates that UX is an important factor in developing BI systems, but is currently lacking theoretical basis. Hence, there is a need for a framework specified toward UX in BI systems development. This study proposes a UX framework for BI systems based on theory from previous research and empirical data gathered from Swedish companies that are developing BI solutions for their customers.

With the scope set to consultant companies in Sweden, the characteristics of the partaking companies can be considered typical for Swedish companies in the BI development industry, since the majority of them operate on a national and international level. Hence, it can be expected that similar results would be found among similar companies in the same geographic area.

The aim of this study is to identify factors for systematically incorporating UX in BI systems based on prior research and input from professionals in industry. The factors are intended to form the foundation for a framework providing BI development companies with structured guidelines for implementing UX in BI development processes. This study contributes knowledge to the topic of UX in BI systems development. The framework presented in this paper is based on existing research combined with empirical data gathered from practitioners in industry. It contributes with a foundation for future iterations of research to keep building upon the presented framework and expanding the scope of this study. This paper focuses on two research questions:

• (1)(a)What factors are important for a theoretical UX framework based on existing research?

• (b)How can the factors found in literature be specified toward BI systems interfaces?

• (2)(a)What practices and development processes are currently used for UX in BI development in industry?

• (b)How can the practices and processes be integrated into a theoretical framework based on research to fit the BI development industry?

2. Relevant literature

2.1. Business intelligence

BI systems are important tools for organizations to set quality and economic goals for their businesses. These systems are used to gather, process and analyze business data, and subsequently, assist in strategic business decision-making. For example, they enable strategic risk management, integration of company data and handle decision-making problems in the companies.^1,2,6,7 By using BI systems companies and organizations are able to interpret the true value of information.² For example, Massardi, Suharjito, and Utama,⁷ showed that BI systems in the banking industry can give important insight into historical data and anticipate changes to business indicators needed to make well-informed decisions for banks.

2.2. User experience in business intelligence

UX is a broad term that has different definitions depending on if it is used in a research or industry context as researchers and practitioners in industry often have different concepts of what UX is.^8–10 According to Lachner et al.⁸, researchers put focus on UX from a theoretical standpoint while the industry is looking for hands-on approaches for implementing UX. In industry, UX is viewed as user-centered design (UCD) while the research community emphasize the difference between UCD and UX. UCD is a multidisciplinary approach containing methods for engineering software intended to create a specific experience for the end user.¹¹ UX on the other hand puts emphasis on the users perceived experience while using a software product. With the maturation of the concept of UX, general frameworks and standards have become applicable for both the research community and the industry (Shin et al., ¹²). This has lead to development of the ISO definition of UX.⁴

In general, a good UX has been acknowledged to significantly affect the product competitiveness and influences the perceived product value to users. Given the specific role of BI systems within the decision-making process of an organization, a good UX of the BI system can play a key role in the productivity of the decision maker.¹ Hence, to be able to cater toward the specific needs of BI system users, a UX framework specified toward BI systems is needed to meet the requirements from an industry that demands real-time intelligent analytics.¹³ Such a framework could lead to improved work performance of decision makers and increased BI system competitiveness for BI development companies.

3. Method

The development of the UX framework for BI systems came in four different steps:

1. Literature study to identify important UX factors for BI systems and to develop a preliminary framework.

2. Unstructured interviews with industry that has activities in BI development to gather perceptions, opinions, and beliefs on the role of UX in BI systems and the UX factors identified in the literature.

3. Revising and enhancing the preliminary UX framework with the information gathered from the unstructured industry interviews.

4. Semi-structured interviews with industry in Sweden to evaluate the created UX framework.

For the literature study search phrases and terms were developed based on the scope of RQ1 (e.g., “user experience,” “business intelligence,” “user-product interaction”). During the searching stage of the literature study, the search terms and phrases developed during the planning stage were applied to digital databases (e.g., ACM, Scopus, IEEE). When literature was found, the article was screened and reviewed for relevance to the scope of the search. During the last stage, the eligibility of articles classified as relevant to the criteria of the extended review were reviewed more closely and synthesized. The results were used to develop the theoretical UX framework of this study.

For the unstructured and semi-structured interviews, several companies in Sweden were approached to partake in this study. The only criteria for participation were that the companies are active in BI development. Of the approached companies, four accepted to participate. The companies partaking in the study were all consultant companies. A short description of the companies can be found in Table 1.

Table 1. Short description of the companies that took part in the study

Company	Description
Company A	Company A is part of a larger group and focuses on delivering enterprise solutions in Microsoft environments. Company A is a consulting company and was started in 2000. They are providing consulting solutions, hosting, intranet solutions portal solutions and BI systems. Their focus is entirely focused on providing Microsoft solutions for their customers ranging from complete server solutions to SQL database solutions. They also provide Office365 and cloud solutions through Azure. Company A develop their BI solutions using Microsoft tools such as PowerBI and Excel. They use an industrialized approach providing a base solution for their customers and customizing it toward the specific needs of the end users and also rely heavily on training and educating the customers in usage and development.
Company B	Company B exists in all Scandinavian countries and also have departments in Ukraine and India. They deliver a wide range of IT solutions from deployment of complete server environments to application development, business systems and BI solutions. The BI department of company B are focused on delivering solutions in Microsoft environments and their BI systems often end up with presenting data in Microsoft PowerBI.
Company C	Company C is a BI consultant company that focuses entirely on providing BI consulting solutions to their customers. They are located in several countries in Scandinavia and are focused in the fields of BI and data science. Company C is not a general IT consultant company and are focused entirely on BI systems. They provide BI solutions using all leading technologies available including Microsoft, Tableau, Cognos and Qlik to be able to develop solutions for customers in any type of environment.
Company D	Company D is a general IT consultant company existing on several locations in Sweden. The main part of their business is specified toward systems development and application management. They are also offering consulting services in design and communications services and BI. In the case of BI development, Company D offer development of decision support systems to their clients. Their BI services include development, architecture, management and BI strategies.

Two different interview approaches were used: unstructured and semi-structured. The unstructured interviews were used to gather as much information as possible on the subject through in-depth conversations with the interviewees to uncover perception, opinions, and beliefs of the role of UX (and its factors) in BI systems. This information allowed us to find confirmation with UX factors found in the literature and to revise and enhance the preliminary UX framework that was created based on literature. Whereas the unstructured interviews allowed us to gather general information on UX in BI systems to further develop the framework, the semi-structured interviews were used to evaluate the created framework by having more focused and directed interview sessions.

4. Results

4.1. Findings from the literature study

The findings from the literature study were used to create a preliminary theoretical framework. The framework was compiled from relevant elements to BI systems from existing UX theories and research. The UX factors found in the literature are presented below with motivations to why they would be important in development of BI systems.

• UX strategy

• Usability

• Product-user interaction

• Context

• Agile/Lean

• Evaluation

4.1.1. UX strategy

As Jooste, Biljon, and Botha¹,mention: UX goals need to be adapted for the context of use regarding the specific software product being developed. The end user experience of a software product is dynamically dependent on the context in which it is being used to create a positive and relevant experience for the user.¹⁴ A positive experience with the system can in turn increase users’ productivity.¹ Providing a positive UX with the intention of increased productivity should be an overall goal and incorporated into the UX strategy when developing BI systems. Development companies normally have a number of different clients they provide and maintain BI systems for. Different clients might have different needs and their contexts of use might differ in a way that it can be hard to replicate a strategy for all clients. When coordinating UX over a number of different products, a UX strategy is significant in its importance.¹⁵ Currently, UX strategies are adhered to by designers, but often disregarded by software developers. Software developers should have more involvement in the practical aspects of UX in order to create a unified UX strategy.¹⁶

4.1.2. Usability

Usability is the definition of describing how easy to use and learn a product is. Usability is a tool that can be used in UX to assert quality of a software product through implementation of certain attributes making it easy and satisfying to use.¹⁷ The ISO definition of usability is that usability is designing a product that allows the user to achieve its intended goals of use with the product in an efficient and satisfactory way. Usability also takes into account the context in which the product is used.¹⁸ Usability in a BI systems context can be considered an important factor since the use of such a system is goal-oriented, the system should support the user in making strategic decisions making its ease of use crucial. End user satisfaction is an important aspect of usability and also important for BI systems.^1,5 Gaardboe, Sandalgaard, and Sudzina¹⁹,found in their study of BI systems that the UX increased if the usability of the system was compatible with the tasks performed by the users. They also found that the user had greater benefits from the systems in their work tasks if the user satisfaction was high. This shows that usability is a factor which has significant impact on UX and productivity in BI systems.

4.1.3. Product-user interaction

The product-user interaction through user interface design is a success factor when it comes to user acceptance in BI solutions.²⁰ Important information presented in the user interface of BI systems needs to be accessible to the user for effective usage.² The user interface needs to be adapted to the end user’s needs in which usability factors (e.g., efficiency, learnability, and memorability) should also be taken into account^13,21, as the user interface directly affects the productivity of the user.

A common problem within corporate development environments is that proper development of the user interface can be constrained by technological and business requirements: functionalities are often continuously added to products in order to market them the end user group.²² The continuous increase of functionalities can result in increased complexity of the product.

4.1.4. Context of use

Different applications require different implementation of UX principles depending on the purpose of the application. The UX is greatly affected by the context in which the software product is being used. The environment and specific characteristics of the end user group and the tasks intended to be performed are important aspects to take into account when planning and developing a product. UX decisions should be made taking contextual factors into account. The contextual factors can also be used for evaluation. To be able to identify the contextual factors, strong user representation is necessary throughout the identification process.²² It is important to have knowledge of the needs of the intended user group.²² Software products with different areas of intended use require their own specific UX principles to be followed. For example, a BI system differs from an e-booking system and requires a specific set of UX factors to be taken into account during development.

A generic framework for applying UX to software products in general would be disadvantageous because different systems fulfill different roles and different user groups have their own specific needs and requirements. Therefore, different systems have their own criteria to fulfill.²² A UX framework for BI systems needs its own characteristics in order to provide a UX specified toward BI systems specifically. Depending on the context of use for the specific BI system being developed, factors specific for the context and the characteristics of the end user group needs to be taken into account.

4.1.5. Agile development

15Liikkanen LA mentions in his case study that it is common that UX is disregarded in agile development processes, but can be advantageous in sprints. Agile software development methods are intended to use iterative development to uncover new user needs during the entire development process. Using agile methods, the development team works in close collaboration with the customer and the end user developing and delivering working versions of the product in short development cycles.^15,17 Maguire¹⁷ is using Scrum as the example agile method in the UX framework he proposes. Scrum is an development method focusing on delivering working versions of the product to the client in short sprints.²³ An alternative to Scrum is lean UX. Lean UX is different from the agile development method Scrum. Scrum development cycles put focus on delivering a working version of the product with each iteration while lean UX is focused on delivering MVPs (minimum viable product). The MVP can be a low fidelity prototype displaying the functionalities of the product used to validate the specific functionalities with the end user group. Even though the prototype is low fidelity it needs to be able to demonstrate the intended functionalities. Lean UX focuses on fast iterations of software development with a large emphasis on the end user.^24,25 The choice of agile development method can vary depending on the context and environment of the company applying the method. Using Lean UX development can be advantageous since a scaled down prototype is delivered and evaluated without overwhelming the end user with a large amount of functionalities.

According to de Carvalho²³, UX needs to be integrated into agile methods with regards to UX activities and developer roles in the agile teams. To efficiently include UX in agile methods a holistic systems development perspective is needed. If agile methods are applied with a focus on the coding part of the product being developed it is common that UX aspects are disregarded.

Dawar et al.²² study cognitive aspects of user needs influencing the user to choose a specific software product. They found that UX development benefits from being iterative which confirms that it is advantageous to incorporate UX in agile development methods. Also that user needs correlate to the functionalities offered by the software product. A professional software system needs attributes and functionalities to cater to a professional end user group.

4.1.6. Evaluation

According to Mulwa, Lawless, Sharp, and Wade²⁷, evaluation of UX is needed throughout the development process and contributes valuable input on changes that need to be made to achieve better results. One such evaluation method is re-framing of contexts. Re-framing contexts focus on analyzing the context of certain problem situations and reevaluating the situations from multiple perspectives with the intention of gaining new insights on user needs.²⁸ By reevaluating usage situations on a regular basis new and valuable insights can be gained that can be used to positively affect the UX of a system. Fronemann and Peissner²⁹,propose in their conceptual UX framework that user generated ideas for features of a system that are selected for implementation by UX professionals enrich the software product and increase the perceived positive UX by the user. Taking into account the wishes for specific features by the end user group in BI development can lead to improved positive effects regarding UX. With use of software applications on a regular basis, users have come to expect easy to use and pleasant user interfaces. Providing such interface experiences can give a competitive upper hand for development companies. This makes UX evaluation during development of software products an important factor.²⁶ By implementing regular UX evaluation into the development and maintenance process issues affecting the UX of the system can be identified and dealt with. This increases the possibility of providing a good UX and user involvement in the development process.

Schulze and Krömker²⁶mention that basic human factors influence the UX of software products. In their proposed evaluation framework they mention product factors influencing UX as utility, usability, visual attractiveness and hedonic quality. They describe these factors as shown in Table 2.

Table 2. Evaluation factors by Schulze and Krömiker²⁶

Factor	Description
Utility	The fulfillment of the software products intended use made possible by the functionalities of the product.
Usability	The software products ability to be used as a tool by its intended user group to achieve the goals the product was created for.
Visual attractiveness	The visual esthetics of a software product interface can act as a motivating factor for the user.
Hedonic quality	Hedonic qualities in a software product are tied to the users emotions affected by interacting with the product.

The UX factors mentioned by Schulze and Krömker²⁶ can be used during development and evaluation as KPIs (key performance indicators). Together with the re-framing contexts method of Adikari et al.²⁸, this can be combined into an evaluation tool focused on improving and maintaining a good UX (Table 3).

Table 3. Evaluation KPIs

KPI	Description
Utility	The BI interface provides the user with correct data.
Usability	The user is able to perform the intended tasks without interference from negative usability factors.
Visual attractiveness	The interface presents the data in a way that is pleasing to the user and assists the user in interpreting it.
Hedonic quality	The UX of the interface and its functionalities has a positive impact on the user emotionally. In a BI systems context, the interface is supposed to be a work tool that assists the user in decision making and getting status reports. The user needs to feel motivated by using the system interface.

4.2. Analysis and results from the first iteration of evaluation

In this section, the results from the first iteration of evaluation interviews is presented. Each of the participating companies are briefly introduced together with the most important factors found during the interviews presented in tables. Conclusions from analysis of each interview is also presented as well as an enhanced version of the UX framework.

4.3. Enhanced framework

With the analyzed data gathered from the unstructured interviews, several topics of the theoretical framework (see 4.1.1 Findings from the extended literature review) could be confirmed as relevant. This allowed the theoretical framework to be enhanced by confirming that it’s content is valid in an industry setting in Sweden. Nothing directly related to the topic of Strategy was discussed during the interviews. The reason being the unstructured and open-ended nature of the interviews.

4.3.1. Product-user interaction

Interview person C and D confirm that increased consumption of data leads to increased productivity throughout an organization. Good UX in a user interface in a BI system can act as a motivation to increase the data consumption by the users. This is in accordance with Karapanos, Hassenzahl, and Martens³⁰,who mention that the impact of software products increase if they are able to stimulate the user. It is also important to put the right interface in the hands of the right users as stated by Interview person C which is also mentioned by Agiu et al.²,stating that the content presented in a BI interface needs to be accessible to the intended users. Interview person C also confirms that it can be challenging to teach new functionalities to users, which is also mentioned by Pandey and Srivastava¹³, and Walsh et al.²¹, who say a too steep learning curve is a negative factor for user acceptance, usability, and UX. Interview person B mentions that the design of BI systems interfaces is often disregarded by the client company of the project. The design of the interface is mentioned as a key aspect in BI systems by interview person C and D which confirms what is mentioned by Brockmann et al.²⁰, who says that interface design is a key factor in BI solutions.

4.3.2. Usability

Since BI systems are used in a professional setting and are intended to deliver functionalities that act as decision support for the users, the usability of the systems is a factor with impact on the UX. Interview person B and C stated that usability and performance from the back end of the system are the largest contributing factors to positive UX in BI systems. Usability is a key factor to a positive UX in BI systems.^1,5,19

4.3.3. Context

Interview person A, B, C and D argued that it is crucial to take contextual factors into account during development of BI systems, contextual factors between different clients in different industries as well as different types of users in the same company. This is in accordance with what is mention by Bouchana and Idrissi⁵, Jooste et al.¹, Maguire¹⁷,on taking corporate and user background factors into account during UX development.

4.3.4. Agile and lean development

Interview person A, B and D mentioned the importance of delivering a slimmed down prototype with a minimum set of functionalities implemented and adding more functionalities and features successively in order to be able to adapt to possible changes in requirements and not delivering an overly complex prototype exposing the users to a steep learning curve. This confirms that a lean development methodology can be advantageous in UX development of BI systems.^24,25

4.3.5. Evaluation

None of the companies participating in the study used any standardized evaluation strategies though they did evaluate through user feedback in general but without any set procedures or methods. They did agree that it could be advantageous to use a standardized evaluation framework to measure UX. Interview person B and C in particular pointed out that there is a need for standardized routines for evaluating UX in BI systems development.

An early visualization of the UX framework was developed based on both what was found through the extended literature search and the unstructured interviews. The intention of the model of the framework was to visualize how the factors of the framework were intended to interact with each other and how they were intended to be applied during a development process. The process assumes that there is no prior system being maintained or re-developed.

1. The strategy and design components of the framework are intended to be executed in chronological order. In the strategy component UX goals, client- and user research and requirements are established.

2. After a strategy has been established, the process continues to the design component. During the design phase usability is adapted to fit the user needs and requirements. Data integrity is established and asserted through making sure the correct data is being presented through the interface. The functionalities and graphics of the user interface are adapted to the context of use and user context established in the strategy component with the intention of motivating consumption of data and productivity.

3. When the first version of the interface has been implemented, agile development sprints begin by adding functionalities and updates to the interface iteratively. Attached to the agile development component of the model are the KPI and evaluation components. With each iteration in the development process, the functionalities are evaluated and reevaluated using the KPIs as measurement. The UX goals set in the strategy component are also followed up on.

4.4. Analysis and results from second iteration of evaluation

In this section, the results from the second iteration of evaluation interviews is presented. During this evaluation round semi-structured interviews were performed. Since the interviews were structured around the topics of the theoretical UX framework, the feedback was already specified toward the specific topics. Therefore, the decisions were made not to analyze the data from the transcripts further since it was already specified toward the factors and components of the framework. The most important factors found during the evaluation along with early conclusions are presented. The final version of the UX framework is also presented in this section (Table 4).

Table 4. Findings from the second iteration of evaluations

Finding	Description
Involve software developers in the UX process	It can be advantageous to involve the software developers in the UX process to make them aware of the UX being produced.
Include performance	Performance is crucial for UX. Performance affects other aspects of the framework (usability, motivate data consumption).
Reevaluation	reevaluation of older functionalities works as long as they are well documented.
Education	Educating the end user is important. The knowledge of the end user affects their experience.
Framework can be used as a checklist	The framework can be used as a checklist during development to measure progress and to make sure nothing is being left out.
General framework/The framework is theoretical	The framework is general and would be useful in a BI development process to use as a checklist of UX factors to take into account.
Structured UX process/UX guidelines are needed	Structured UX process as presented in the framework is needed in BI development.
Structured evaluation	Structured and standardized evaluation is needed.
Content	Content is refined data intended to be interpreted by the user
Graphical design/Esthetics	Visual design is often not a priority for clients. Esthetics relies heavily on context (client, user).
Remove the increase productivity topic from model	Increase productivity sounds like something belonging in a sales pitch.
Hedonic quality related to data consumption	The hedonic quality KPI is associated with motivation to consume data.
Improve visualization	The visualization of the framework needs to be improved to better display how the parts relate
Client company goals	The goals of the client company needs to be set in the beginning of the process.
KPIs in strategy	Additional KPIs specific to the client company needs to be set in the strategy component.
Connect esthetics to usability	Esthetics are related to usability aspects of the interface.

4.5. Findings

4.6. Discussion of findings

Interview person A, B, and C mentions that the framework can be adapted and used in a BI development process and that the factors included in the framework are relevant to BI development. Interview person B and C says that the framework looks general but can be applied to a BI systems process and can be very useful as a checklist to keep track of UX factors to take into account during development. In a realistic development scenario, the framework can serve as a template or guideline for UX factors needed to be taken into account in order to produce a good UX.

According to interview person C, the visualization of the framework is correct if the intention is to give an abstract visualization of the UX development process. In a realistic development scenario, the framework can serve as a template or guideline for UX factors needed to be taken into account in order to produce a good UX. Interview person D indicated that the visualization of the framework was in need of improvements showing more clearly how the different factors included relate to each other. The client company goals need to be present in the strategy part of the development process. The client company goals with using the BI system need to be integrated and correlate with the UX goals. Interview person D also mentions that in addition to the KPIs suggested in the evaluation part of the framework, KPIs measuring the usage goals for the BI system specific to each individual client need to be set in the strategy component of the framework. Esthetics and usability are mentioned to be related to each other in a way that the graphical design of the interface affects the usability of the BI system. If the content is presented to the user in a graphically pleasing way it will assist the user in interpreting and making decisions based on the data.

Interview person B and C also mentions that standardized guidelines in the form of a framework are needed for UX in BI systems development since there is currently no standardized process being used.

Interview person A and B mentions that it can be good for the software developers to have knowledge on what the UX goals are and have communication with the people working with UX. Many times the same people work with software development and UX. If this is not the case it is even more important to have communication between the different parts of the team. Involving the software developers in the UX process to make them aware of the UX being produced. The developers are often tasked with implementing UX as well as developing the software.

Interview person A, B and C states that performance needs to be included in the framework since performance of a BI system is a key aspect regarding usability and UX. If the performance of the system is slow, the motivation to consume data is decreased along with the usability and UX of Education of the end users is mentioned as an important factor by interview person A. Since the UX of the BI system is related to the knowledge and skill of the user. The user needs to learn how to use the functionalities of the interface in a correct way, otherwise they might perceive the system as slow and difficult to use. In order to achieve user acceptance of the product, education of the end user is necessary.

Content is also a key factor belonging in the framework according to interview person A, B, and C. Content in BI systems is the refined data processed in the back-end of the system and presented through the interface. Interview person C explains that the user can not use the raw data for decision support, the data needs to be refined into content before being presented to the user. The content needs to be adapted to the specific user depending on the level of maturity regarding IT and analytics of the user.

Regarding graphical design, this is something that is often disregarded by the clients and not prioritized when requesting a BI system according to interview person B and C. Esthetics is dependant on the user and client context.

Interview person A mentions that reevaluation of older functionalities works as long as they are well documented. If undocumented it could take too much time to reevaluate. According to interview person C, the rigor of the evaluation process will be is determined by the success and amount of users of the system. The esthetics of the user interface is also an important factor to evaluate according to interview person C. Interview person C further states that the hedonic quality KPI would be used to measure the users motivation to consume data using the BI system. Interview person C further states that the hedonic quality KPI would be used to measure the users motivation to consume data using the BI system.

4.7. Revised framework

The enhanced framework (see 4.2.6) that was used during the evaluation interviews was revised based on the feedback received from the interviewees partaking in the study. The framework remained in the same state as before the evaluations with the addition of the new factors performance, content and education integrated into it. The model of the framework was also updated based on previously mentioned feedback and is presented below in (Figure 1.).

Figure 1. An early visualization of the framework

4.7.1. Performance

According to interview person A, B, and C performance is a key factor affecting the UX of a BI system. The performance of a BI system is crucial for providing good usability and UX, also for achieving user acceptance of the product. Interview person C mentions performance as the most important factor affecting UX in development of BI systems. It can be assumed that performance is affecting the UX in any type of system, but in BI it is of particular importance. Since BI systems process enormous amounts of data, the time it takes to process the data and presenting it as content in the user interface can become long. This can make functionalities in the interface slow and not working as intended, it can also affect quality of the content presented, especially if the user is requesting real-time reports from the BI system.

4.7.2. Content

Content is mentioned as another key factor to take into account during UX development for BI systems by interview person B and C. The content is the product of the data being processed in the back end of the BI system. The BI system gathers the data in raw format from a number of sources, for example, HR, CRM, and business systems. The data is processed into content, the content is the refined data presented to the user in the interface of the BI system. The content needs to be carefully planned and adapted toward the specific client and user context since the user needs to be able to make sense of the content in order to use it for support in making important decisions.

4.7.3. Education

Interview person A mentions education of the end user groups as an important factor for providing a positive UX in BI systems. When developing a new system with a new set of functionalities for a client company the users will be challenged with a level of complexity and a learning curve. If the functionalities are used in the wrong way there is a risk that the users may experience an increased level of complexity and perceive the BI system as slow and hard to use which will affect the UX negatively. On the opposite, if the users are educated in how to efficiently use the system as it is intended to be used, the UX will be affected positively.

4.7.4. Framework

After concluding the results from the evaluation interviews and revising the framework based on the feedback from the evaluations the framework remained in its original state (see 4.1.1) with the most important factors discovered during the evaluations added to it.

• UX strategy

  • Content

• Product-user interaction

• Usability

  • Performance

  • Education

• Context

• Agile/Lean

• Evaluation

The updated model of the framework shows the same process, components and factors as in (Figure 2) with the addition of a component for educating users in best practices with the intention of increasing the UX as mentioned by interview person A and a component containing client-specific KPIs mentioned by interview person D. Relations between the different components have also been added to the model to give a better semantic representation of how the factors within the components relate to each other. The relations between the factors inside the components of the model are represented by dotted lines. This shows how the content in one component relates to content in other components. For example, client goals and KPIs are established in the strategy component and followed up on and measured in the client-specific KPI and evaluation components. Usability and performance in the design component is measured in the UX KPI component with the esthetics and usability KPI. The hedonic quality KPI can be used to measure the end users motivation to consume data using the BI system interface.

Figure 2. Visualization of the framework after revision

5. Conclusions

It was confirmed that adapting BI systems interfaces to the end user is of significant importance since the users’ skill in information technology and analytics can be varying as mentioned by Jooste et al.¹ This was an important factor for providing a good UX as mentioned by the professionals participating in the study. This study showed that previously mentioned problem can be solved by adapting the complexity of BI interfaces to the skill level of the users and by educating end users in how to use the interfaces correctly. This shows the importance of taking contextual factors into account for each specific BI system being developed. The study was also able to confirm that the age of users can be an indicator on the level of UX requirements as mentioned by Pandey and Srivastava.¹³ From the evaluation interviews it became clear that younger users normally have higher requirements on graphical aspects as well as usability aspects of BI systems interfaces. It also became clear that UX in BI systems can be a factor for motivating increased data consumption by providing a positive experience for the users. The fact that good usability has a significant impact on the UX of BI systems was also confirmed during this study. It shows that key information needs to be presented in a way that assists users in interpreting information.^1,2,5 It also became clear that the partaking companies left UX decisions to the judgment of individual developers and consultants to take whatever measures they saw fit. This further shows the need for standardized guidelines in the form of a framework which could lead to better and more consistent results regarding UX aspects. In future studies it would be beneficial to include corporate processes and decision-making affecting the development processes at companies developing BI systems.