User-Related Challenges of Self-Service Business Intelligence

ABSTRACT

Self-service Business Intelligence (SSBI) allows non-technical users to use Business Intelligence in a self-reliant manner without the support of technical users. Many organizations struggle to utilize the potential of SSBI and experience implementation challenges. This study aims to explore user-related SSBI challenges by conducting 30 qualitative interviews with 2 SSBI implementation projects. Analysis revealed challenges that can help practitioners to avoid unnecessary obstacles when implementing and using SSBI, and guide researchers in simplifying the implementation process.

KEYWORDS:

• Self-Service Business Intelligence
• challenges
• ssbi
• BI
• user related

Introduction

Traditional BI has been implemented by organizations for decades. It is often defined as an umbrella concept that includes a set of technologies and software that enables users to access and analyze data in order to make better decisions (Wixom & Watson, 2010). To access and use multiple data sources for analysis and decision-making is not easy: it requires technical skills that not all users possess (Alpar & Schulz, 2016; Sulaiman et al., 2013). Decision-makers, information consumers, analysts and specialists involved in using BI for decision support are typically categorized into two types: power users and casual users. Power users have the required technical skills to build and run BI efficiently. They are able to select and use data, generate, for example, reports, and analyze content by themselves or present it to casual users so they can make decisions. Casual users are normally operational employees, mangers and users that lack technical BI skills and are thus relying on pre-defined reports or dashboards, created and provided by power users, to support their decision-making (Alpar & Schulz, 2016; Sulaiman et al., 2013). Due to this difference in technical skills, casual users and power users are forced to interact in a request and response scenario where technical BI specialists typically serve non-technical decision-makers (Alpar & Schulz, 2016). As the world continues to become more dynamic and larger volumes of data are made available, more and more casual users are in need of BI. Consequently, a bottleneck arises in the request–response relationship. BI specialists cannot cope with the increasing number of requests from a growing group of casual users (Alpar & Schulz, 2016; Imhoff & White, 2011; Yu et al., 2013). More flexible and usable BI systems are needed to cope with this development (Yu et al., 2013).

One upcoming trend suggested to address this problem is self-service BI (SSBI). A key element of SSBI is its focus on empowering casual users to become more self-reliant and less dependent on BI specialists (Imhoff & White, 2011). SSBI allows casual users to access and use data as desired, enabling them to analyze data and make decisions without support from a technical BI specialist (Alpar & Schulz, 2016; Imhoff & White, 2011). The main benefits of SSBI are to enable users to conduct their own analysis, to reduce pressure on the IT department, to eliminate guessing when making decisions, to shift from reactive to proactive analysis and to save organizational resources (Alpar & Schulz, 2016; Imhoff & White, 2011).

Even though SSBI offers benefits compared to traditional BI, many challenges need to be addressed to make SSBI work (Daradkeh & Moh’d Al-Dwairi, 2017; Lennerholt et al., 2018). There is a lot of research on challenges for traditional BI implementation (Popovič et al., 2012; Ramakrishnan et al., 2012), and it is presumable that challenges for SSBI at least partly differ from traditional BI challenges. Several studies (Eckerson, 2012; Lennerholt et al., 2018; Logi Analytics, 2015) explain on a general level how realizing SSBI is not as easy as suggested. More in-depth research is needed to understand in detail what SSBI challenges entail.

As one of the major differences between SSBI and traditional BI is the shifting role of the user (from a technical power user to a non-technical casual user) it is especially interesting to investigate whether user-related challenges for SSBI differ from user challenges for traditional BI (Dedić & Stanier, 2016; Jooste et al., 2014).

In response, this research presents a case study aiming at identifying user-related challenges of SSBI. This article starts off with an overview of user-related SSBI challenges identified in the literature. Subsequently, the research design and the empirical results of the case study are presented. The discussion section highlights how 23 user-related SSBI challenges identified in this study considerably differ from user-related SSBI challenges described in the literature.

Background: User-related SSBI challenges

One of the many benefits of SSBI is to enable all users, especially the casual users, to benefit from BI without the support of power users. They should be more self-reliant when accessing and using data and they should be able to create their own reports as desired during decision-making. In this paper, SSBI is defined as: “The facilities within the BI environment which enable BI users to become more self-reliant and less dependent on the IT organization. These facilities focus on four main objectives: easier access to source data for reporting and analysis, easier and improved support for data analysis features, faster deployment options such as appliances and cloud computing, and simpler, customizable, and collaborative end-user interfaces.” (Imhoff & White, 2011). Although SSBI offers many benefits, implementing SSBI is not as easy as expected (Alpar & Schulz, 2016; Lennerholt et al., 2018).

Previous research on SSBI is relatively limited when describing challenges of how to implement and use SSBI (Eckerson, 2012; Imhoff & White, 2011; Weber, 2013). State-of-the-art SSBI reports do only briefly mention challenges when implementing and using SSBI (Eckerson, 2012; Logi Analytics, 2015). Lennerholt et al. (2018) have performed an extensive literature review to identify known SSBI challenges. The results include six challenges related to “Access and use of Data” and four challenges to “A self-reliant user”.

SSBI challenges related to Access and Use of Data

Within the category “Access and Use of Data” Lennerholt et al. (2018) identified the following SSBI challenges:

1. Make data sources easy to access and use

2. Identify data selection criteria

3. Use correct data queries

4. Control of data integrity, security and distribution

5. Define policies for data management and data governance

6. Prepare data for visual analytics

The “Access and use of Data” challenges show that users find it difficult to access and use data as desired since data sources are not freely available (Imhoff & White, 2011; Logi Analytics, 2015; Meyers, 2014; Schlesinger & Rahman, 2016; Shakti, 2013; Spahn et al., 2008; Stodder, 2015; Weber, 2013; Zaghloul et al., 2013). The fact that users cannot use desired data when needed does not go hand in hand with SSBI since it should enable all users to use BI in a self-reliant manner without support from power users. Poor or lacking quality criteria is another challenge that makes it difficult for users to determine the right level of quality when selecting data, especially when using external data sources. Accuracy, freshness, completeness, reliability, etc., are factors that users are not aware of when selecting data, which affects decision-making negatively if used in an inappropriate manner (Abelló et al., 2013; Meyers, 2014; Schlesinger & Rahman, 2016; Weber, 2013).

If needed data sources are available and the right quality is achieved, users may still find it difficult to use correct data queries to combine and use the sources (Abelló et al., 2013; Meyers, 2014; Schlesinger & Rahman, 2016; Weber, 2013). SSBI requires more technical skills that casual users mostly do not possess. It is very important that queries are correct. Even the smallest mistake, e.g., no proper joins between data or using incorrect attributes, can lead to serious faults in analysis. Such mistakes do normally result in faulty data which in turn leads to inferior decision-making.

The control of data integrity, security and distribution is a challenge since many users are still using spreadsheets as their main BI tool (Alpar & Schulz, 2016; Logi Analytics, 2015). This becomes even more troublesome when external data are used. Therefore, organizations must ensure that data is properly incorporated into data warehouses to minimize errors. This involves deciding who can use and data and how long it should be stored until it is determined to be outdated. Inconsistent data with regard to security and quality can lead to problems when accessing and analyzing the content (Alpar & Schulz, 2016; Logi Analytics, 2015).

Policies of how to manage and govern data (Meyers, 2014; Stodder, 2015; Weber, 2013) focuses on enabling policies that define how to access data and what level of quality that is needed for the desired analysis. Users need to understand the value of data when analyzing for decision-making. Analyzes that do not make sense are mostly due to using wrong data (Alpar & Schulz, 2016; Logi Analytics, 2015). The result is inconsistent reports that users should not spend time working with.

SSBI users should be able to create their own story by using data that visualizes the content in an easy self-reliant manner (Johannessen & Fuglseth, 2016; Meyers, 2014; Stodder, 2015; Weber, 2013). SSBI should be easy to use by all users, even non-technical casual users. They are not supposed to select and use data in a way that requires programming expertise which only power users possess. Therefore, technologies are needed that enable users to create their own story in a drag and drop manner to reach conclusions when making decisions. Today, these tools are not as easy to use as would be expected from the promise of SSBI.

SSBI challenges related to A Self-Reliant User

Within the category “A self-reliant user” Lennerholt et al. (2018) identified the following SSBI challenges:

1. Make SSBI tools easy to use

2. Make SSBI results easy to consume and enhance

3. Give the right tools to the right user

4. Educate users in how to select, interpret and analyze data for decision-making

The challenges related to being a “self-reliant user” mention that SSBI tools are not easy enough to use (Alpar & Schulz, 2016; Imhoff & White, 2011; Schlesinger & Rahman, 2016; Spahn et al., 2008; Stodder, 2015). Users are unable to use SSBI in a self-reliant manner since they often need support from power users. The consequence is that users find alternative ways developing their own isolated and limited solutions, which are not always correct. Therefore, an important challenge is to make SSBI tools more user-friendly (Alpar & Schulz, 2016; Imhoff & White, 2011; Schlesinger & Rahman, 2016; Spahn et al., 2008; Stodder, 2015).

SSBI users must also be able to analyze content in a self-reliant manner when making decisions. The second challenge shows that users find it difficult to understand the information presented to them (Alpar & Schulz, 2016; Imhoff & White, 2011; Schlesinger & Rahman, 2016; Stodder, 2015). They have to spend unnecessary time when analyzing the content to interpret the overall meaning. SSBI is supposed to simplify the process to use BI, which is not always the case. SSBI does not mean that users are supposed to freely incorporate data in their own isolated spreadsheets or ghost IT system. Different SSBI users have different skills and needs compared to others, which should be reflected in the available tools. This is not always the case since tools are freely available to all users, independent on their background and technical skills (Alpar & Schulz, 2016; Eckerson, 2012; Stodder, 2015; Sulaiman et al., 2013). SSBI does not require that all users must use the same tools. Therefore, a challenge is to have a good strategy for offering the right tools to the right users, based on their individual needs and requirements.

Using SSBI is not as easy as expected, which implies that there is a need for user education as highlighted in the last identified challenge (Eckerson, 2012; Johannessen & Fuglseth, 2016; Stodder, 2015). Organizations believe that SSBI enables all users to perform their daily tasks in a much simpler way compared to using traditional BI where power users serve casual users. This is not the case since SSBI does require proper education on how to use data and how to interpret and analyze the content to make good decisions (Eckerson, 2012; Johannessen & Fuglseth, 2016; Stodder, 2015). Lack of education is a challenge that organizations must manage. They should focus on teaching users to choose the right data and to interpret its content in a way that enables them to make decisions, instead of only educating users on how to use BI tools.

Research method and analysis

Case study research is an appropriate research method to achieve in-depth understanding (Braa & Vidgen, 1999; Yin, 2013) and involves in this study a BI consulting firm and two of their customers. The customer organizations have strong ongoing relationships with the consulting firm as it supports their implementation and usage of SSBI. The consulting firm has had BI and decision-making as their core business area for over 10 years and has supported over 200 customers. The two customer organizations are included in the study since they both have a strong interest but also rather different experience with regard to implementing SSBI. One of them has implemented and used SSBI for many years and is considered an experienced SSBI adopter as they have achieved many benefits compared to other organizations. The second selected customer organization is newcomer to SSBI and started their journey toward SSBI recently. Triangulating experiences from all three involved organizations gives in-depth understanding of what kind of user-related SSBI challenges can exist.

Research process

There are many valuable contributions on how to conduct case study research (Walsham, 1995; Yin, 2013). The chosen research process is inspired by and Braa and Vidgen (1999) and Pan and Tan (2011) and has followed their proposed steps and detailed instructions. The following steps have all been conducted by the first author (Braa & Vidgen, 1999):

1. Existing theory: The case study started by conducting a review on the associated challenges existing in the current literature.

2. Collecting and organizing data: The data collection process consisted of interviews with respondents from the three organizations. It was considered done once saturation of user experiences was reached, i.e. no more user experiences were identified, and when enough data was collected to fulfil the aim of the research. All collected data was transcribed.

3. Confirming and validating data: Respondents were contacted for an additional interview to confirm previous transcripts, solve potential misinterpretations and to confirm or add missing data.

4. Coding process: Open, axial and selective coding was used iteratively to analyze the collected data (Wolfswinkel et al., 2013). During this process, the categories and challenges were identified.

5. Validate the results: a follow-up interview was used to discuss the identified categories and its challenges. No respondents rejected the results. Instead, they found the challenges plausible even though not all respondents were aware of them from the beginning.

Data collection

The data collection technique for the case study was semi-structured interviews (Pan & Tan, 2011; Yin, 2013). Data were collected during the spring and autumn in 2017 and consisted of 30 semi-structured interviews with all responsible employees and consultants who participated in the implementation of SSBI in the two customer organizations. The SSBI projects consisted of 5 BI consultants within two customer organizations of 5 employees each, resulting in 15 respondents in total. The representatives ranged from vice-presidents, consultants, analysts, architects, SSBI evangelists, BI developers, business improvements managers, strategists, business controllers, IT specialists, managers and end-users. A respondent may belong to more than one role since they share multiple responsibilities within their organizations. Based on their roles and responsibilities, they were able to speak for other employees with similar roles and backgrounds. On average, all respondents had between 5–10 years of experiences working with implementing and using BI and SSBI. Therefore, all respondents are considered experts with a strong focus toward implementing and using BI and SSBI within their organization.

Each respondent participated in an initial contact, a main interview and a follow-up interview. The initial interview was held some weeks before the main interview to inform and prepare the respondent. The focus of the main interview was to collect data that fulfills the aim of the research. The interviews from the main event were transcribed, and validated by the respondent before entering the qualitative analysis process. The follow-up interview was conducted to include details that were missed or changed during the main interview. On average, each interview lasted 1 hour. The data collection was considered complete when saturation was reached, i.e. when no more challenges were identified.

Qualitative analysis

The qualitative process applied open, axial and selective coding (Wolfswinkel et al., 2013). First, the coding process focused on identifying a set of main categories (open coding) and sub-categories (axial coding) which visualize how the collected data portrays the identified challenges. The focus was to have an open mind when identifying categories representing the collected data and not let previous research guide or affect the process. An iterative process grouped different segments of text into categories (selective coding), which represent the identified challenges within each category. The aim of the open coding was to iteratively identify a set of main categories which could incorporate all identified challenges within the collected data. It finally resulted in a set of three main categories.

The coding process and the qualitative analysis were considered complete once all transcripts had been analyzed and no new challenges, sub-categories or main-categories were identified (Wolfswinkel et al., 2013). The qualitative coding process was conducted manually using different colors for each sub and main category in a word processor. All categories and challenges were shown to the respondents for validation. No categories or challenges were changed during validation.

Results

The result of the conducted research is summarized in Figure 1. It presents the three identified main categories and seven sub-categories of user-related SSBI challenges.

Figure 1. Categories and sub-categories of user related SSBI challenges

The challenges are presented in subsections following the main categories and subcategories.

Self-reliant user

One of the key factors to use SSBI efficiently is to let all users, including non-technical casual users to be self-reliant when using BI. Casual users are supposed to access and use data for analysis and decision-making, without support from power users. The results of this study show how users have difficulties to use SSBI. The category “Self-reliant user” consists of three sub-categories and eleven challenges, which are presented in Figure 2 and described hereafter.

Figure 2. Challenges of self-reliant user

Difficult to access and use data

This section describes the challenges related to the sub-category “Access and use data”, which belongs to the main category “Self-reliant user”. Accessing data when desired and needed is important to let SSBI run efficiently. According to our findings it is not easy to fulfil this requirement.

Challenge #1: Difficult to know available data sources: The first challenge faced by casual users is the difficulty to understand which data sources are existing. Imagine that important data are needed to make good decisions. If users do not understand whether important data are available or not, it causes major problems when analyzing content for decision-making. Many respondents endorse the following quote: “I do not really know what data sources are available. I know that most of the data I need usually is located in our database”. Users have low awareness of which data sources are available.

Challenge #2: Difficult to locate data: SSBI users who know that data is available face problems to determine where it is located. They experience problems whom to contact to gain access, which is a time-consuming and troublesome process. One respondent mentions: “Who shall I contact? Who is in charge for this data? I want to calculate customers and do not know what database to access. When I manage to access the database, what fields are needed to determine if it is a customer at a given time? Does it have to be connected to a product or what is the definition?”. This shows how users lack knowledge of how to access data even though users understand that certain data is available. If users of SSBI are having problems to access and use data, it affects the benefits of SSBI negatively. Several respondents have quotes of the following character: “It is difficult to access the available data which is out there. It is possible to access and use it fully, but very few know how to”. Even more experienced SSBI users are having problems, which is shown by the following quote: “I have used the system for many years, with access to more basic data, and I am beginning to understand what data actually exists”. This challenge is considered one of the largest affecting SSBI negatively, which was mentioned by many respondents.

Challenge #3: Difficult to use data: Even though users manage to access desired and important data when making decisions, the next challenge to address is to understand how data can be used. Our findings show that there is a lack of knowledge on how to use data. Even though data is of high quality, users must understand the content of the data and to understand when to join and when to not join data. Sometimes you can filter data on specific details and sometimes you cannot. If users are not aware of how to use data, it will lead to wrong conclusions which affect decision-making negatively. A respondent mentions: “Even if people have access to data, I observe many who actually do not know how to use important data”. Another quote elaborates: “Before SSBI, there were experts who understood data. Take for instance, column names of codes. If you do not understand them, you cannot use SSBI. You need to have good names and make data more understandable. You cannot expose all data to all users because it is difficult for them to understand. Instead, it requires that different data is made available to different SSBI users.”.

Challenge #4: Different data sources: Our findings reveal the difficulty of using many different data sources at the same time. In a traditional BI system, power users have the technical skills to join data correctly. Now, casual users have to do this themselves, and if done wrong, the resulting analysis that should alert that decision-making may be based on incorrect data. A respondent explains: “It is not easy to access different data sources. There is no single main data source to use. Instead, we have around 5–10 data warehouses which have been created over a time period of 15 years”. Therefore, implementing and applying SSBI requires a lot of planning on how to access and use data efficiently. The implementation of SSBI is not as easy as expected, which confirmed by one of the respondents with a lot of experience with SSBI: “I believe SSBI is over-simplified. It sounds really nice to have and easy to implement. All users are able to do their own work without support. It is however not so simple to achieve.”

Challenge #5: Support is required to add data: Most users find the process to access and use data too difficult. They often need to contact IT support to use missing data. Even though SSBI implies letting all users be self-reliant without support from a power user, our findings show how SSBI users still need support from the IT department to add missing data when making decision. Therefore, the request-response scenario between casual and power users still exists, even though the main driver for SSBI is to eliminate this bottleneck. A respondent from the IT department mentions this challenge: “If users miss data, or the possibility to divide on, for example, car color when looking at sales; then it results often in a request back to us”. This quote is in accordance with the statements of two other users who have a lot of experience implementing and using SSBI. It is a challenge which affects even experienced SSBI organizations, not only newcomers to SSBI.

Low user skills

The second sub-category of challenges related to being a self-reliant user shows how SSBI requires skills which many users do not possess. Three challenges have been identified within this sub-category.

Challenge #6: Limited competence level: One of many mentioned and advertised benefits of SSBI is to let non-technical casual users use BI more efficiently without support from more experienced and technical power users. Bottlenecks in traditional BI systems where power users serve causal users should be removed with SSBI. Our findings indicate however that many casual users still lack the right level of competence to use SSBI. One respondent who supports organizations to implement and use SSBI mentions: “When I meet people and talk about SSBI, they want to enable all users to do more themselves. But when we sit down and try to show them how SSBI could work, I realize within 5 minutes when they push buttons and navigate systems that they will not be able to make it work. This person will never be able to use SSBI”. Such challenges are not dependent on the particular software used. A respondent within another organization who uses different software agrees: “It is pretty common that users themselves do not know what they are doing. They have no clue what they are up to.” It is important to understand which users in your organization are intended to use SSBI and what competence they need to acquire. Perhaps the ambition of SSBI to enable all users is not feasible, as illustrated by the following quote: “I work with people who have limited technical competence. They are perhaps able to use standard reports which already exist. As long as they do that it is probably okay. But if they change anything, everything crashes. So, they are able to use it as long as nothing changes. It should work for a short while. But when anything changes, it will crash and I have to fix it”. This quote brings is back to a request-response scenario where power users support casual users when using data to make decisions. So organizations implementing SSBI should ask themselves if really all users or only some selected groups should master SSBI, and what competence levels would be desirable for each user group.

Challenge #7: Difficult to interpret report content: Users find it difficult to interpret content in SSBI reports. In traditional BI systems power users have the skills to create reports which consist of relevant data which is presented in suitable way. Casual users can understand the content of those reports when analyzing and interpreting its content for decision-making. When SSBI enables all users to access and use data as desired, difficulties may arise to understand what the data actually means. One respondent exemplifies: “The biggest challenge to SSBI is to understand the data. There are so many different definitions which are named differently in different systems. Some data is ‘basic stored data’ while others are ‘created definitions based on calculations’. But what does this mean? How have they calculated things which are stored and what is not included in the calculation?” It makes a huge difference if data are needed to recognize general trends, or whether data is needed for a specific and accurate calculation. The problem accumulates when users are sharing these reports with other employees. Built-in problems that users are unaware of can cause negative outcomes when decisions are made. Users who reuse reports created by others may interpret those reports as quality proved, even though they contain faults. One respondent refers to such an incident: “We used a SSBI report and its data without knowing where the data came from. Based on what they done we created a new SSBI report and later found out this data was inaccurate and wrong. We thought the existing report was of good quality but it was not thought through well.”. Therefore, users need to increase their understanding of data, what it means and where it comes from, so they can interpret report content correctly, and to avoid decision making based on flawed reports.

Challenge #8: Limited general IT skills: Some basic IT skills are needed even though SSBI is supposed to simplify the process to use BI. Our findings show how users lack general IT skills, which affect the usage of SSBI negatively. The following quote illustrates the situation: “We work with different software to make our analyzes. They are pretty easy tools to use, but far from obvious to use. Many of our users lack general IT skills. Users need an SSBI intro, some kind of minor education, to learn basic skills for a few days and hours.” This problem may arise because SSBI is given to many casual users who were not using traditional BI earlier. Even though SSBI is advertised to simplify the process of using BI, it does not mean that anyone can use SSBI efficiently, especially not with limited general IT skills or without any basic SSBI education.

SSBI tools are difficult to use

The third and last sub-category of challenges related to being a self-reliant user shows how users experience SSBI tools as too difficult to use.

Challenge #9: Difficult to use SSBI tools: Non-technical causal users are having difficulties to understand and use SSBI tools. All respondents had some kind of quote referring to this, like: “If you use a specific software, it is not piece of cake. It took a long while before I created another SSBI report once the first was done. It was not the feeling you have when riding a bicycle.” Or: “It is pretty easy to get started creating really basics. But very difficult when you want to create good looking reports. We were in need of huge support from the software experts”. The respondents witness something more than just a threshold for learning new software, since all of them have high IT skills. Two respondents mention that SSBI tools are difficult to use and how they had to run for their colleagues to ask for support. Another respondent agrees: “I have to ask colleagues all the time. We cannot work like that. It is really difficult to understand how to use SSBI”. If users are supposed to be self-reliant it requires that SSBI tools are easy to use. Employees having to ask their colleagues for support in order to run SSBI, is not a self-reliant way of working.

Challenge #10: Users create isolated solutions: When users experience that SSBI tools are difficult to use, they may turn to building their own isolated solutions, as they need to analyze their data anyway. Excel is commonly used as traditional BI tool which users feel comfortable using. Two respondents say they have more skills with Excel and are therefore creating their own solutions when their SSBI tools are difficult to use. Yet two other respondents attest: “There are many users who believe it is no idea to contact IT for support when facing difficulties with SSBI tools. They will not prioritize it. Instead, we collect data from the SSBI tools, copy paste into our own solution, and we make sure it fulfills our requirements. I also see managers who do not want to spend a lot of time searching for data in different reports. Instead, they create their own solutions in Excel”. If SSBI tools are not used as they are supposed to, it will cause negative effects to SSBI implementation. Users who are not able to be self-reliant due to difficult SSBI tools are facing the option to ask for support or the option to create their own isolated solutions.

Challenge #11: Give the right tools to the right user: Different users have different skills and needs compared to others. This should be reflected in the SSBI tools that are offered, which is not the case according to the results of this study. Instead, organizations believe that SSBI tools are of a “one-size fits all” character due to its believed simplicity of using SSBI tools compared to traditional BI tools. However, using SSBI tools is not as easy as expected. It requires proper education on how to access and use data and how to analyze its content when making decisions. Therefore, this challenge highlights the importance to give the right tool to the right user accompanied by dedicated education. A SSBI toolset accustomed to the need of specific user groups contributes to enabling performance of SSBI tasks in a self-reliant manner.

Creating SSBI reports

Reports in many different formats are commonly used as a basis for decision support in traditional BI as well as SSBI. In SSBI all users are now able to create reports by themselves without support from a power user. Our findings show how the process to create reports is difficult and hinders the effectiveness of SSBI. Two categories and seven challenges are related to creating SSBI reports (Figure 3).

Figure 3. Challenges when creating SSBI reports

Difficult to create and change content

The first identified sub-category shows how users find it difficult to create and change content in SSBI reports. Three challenges belong to this sub-category.

Challenge #1: Difficult to create SSBI reports: Users have different background and skills, which affects the process to create SSBI reports. Many users find it difficult to create SSBI reports, especially those based on raw data from different data sources. Instead, they manage barely to create their own reports by collecting data within already created reports designed by power users. A power user respondent mentions: “Users collect data from reports that I have made. It works at this level. They will never be able to create new reports by themselves”. The more experienced users with a more technical background manage to create SSBI reports more easily while others require support. An IT department power user respondent says: “We have data scientists who are very self-reliant. They are able to work independent from us. We have a broad spectrum of users. Some users are able to create SSBI reports by themselves while others require lots of support”. Even power users are experiencing challenges when creating reports. Data is accessible but cannot easily be tailored to individual needs: “Well, I am able to access data. We have data warehouses and good information is built up in different cubes. The problem is, even though I am able to access data, I am not able to ask too many follow up questions. Then it becomes very complicated and requires an order to the IT department”. Even the technically skilled users believe that creating SSBI reports is complex.

Challenge #2: Requires lots of time and manual work: SSBI users perceive that creating SSBI reports takes a lot of time and requires lots of manual work. The software tools cannot automatically create SSBI reports but requires troublesome manual work, especially when many users are involved in the creation: “Reports which are used nationally are created by many users and cannot be automated. It requires lots of manual work even though SSBI aims to simplify the process”. Another respondent explains how creating SSBI reports sometimes is a technical and time-consuming process: “There is no list of requirements when creating SSBI reports. It is now part of my role and the time it takes depends. An example: When I create a report for our team leaders they needed lots of KPI’s from different sources. It was a mega job to fulfil everything technically”. When the process starts to become troublesome users turn to power users for support, just like in traditional BI.

Challenge #3: Difficult to change content: The third challenge shows that SSBI users often require IT support to change existing reports. Even experienced organizations working with SSBI for many years are facing this challenge on a daily basis. One quote, confirmed by many respondents from IT departments, argues: “It is common that users come back all the time to change content. When I show them what the software can do, they get new ideas and want more. Or they ask if they have interpreted things correct. SSBI is still an iterative process”. Even the smallest change is a request to the IT department: “If some data is missing or if they want to split of example car color and sales, it results often a request back to us”. The purpose of implementing SSBI may be questioned if organizations do not manage to minimize the request and response scenario which is one of the main issues SSBI tries to solve compared to traditional BI.

Difficult to assure quality

The second and last sub-category related to creating SSBI reports consists of four challenges which are described in this section.

Challenge #4: Difficult to assure quality of reports: When all users are able to create reports, the number of created reports increase. How do we know casual users have created reports which are of high quality and where data has been used properly? Many respondents put forward arguments like: “The problem with SSBI is not that users are not creating reports. Instead, too much content is created. If we are supposed to make decisions based on this, we must have extremely high confidence in the user who created the report. That the calculation of this number is correct and according to the information model”. If users have missed something while creating reports it will affect the implementation of SSBI negatively. As argued by one respondent who has been involved in many SSBI implementations: “SSBI brings many benefits in different ways. But if we cannot assure the quality within reports exists, the implementation of SSBI for many users becomes a problem”. Users are prone to believe that content is of high quality since someone else already has created the report and approved its quality.

Challenge #5: Redundant reports exist: All users with different skills and needs are allowed to create new reports. The reuse and customization of old reports is challenging as there are different levels of quality needed for different types of decisions. If users are not aware of the different levels of quality it can cause serious problems: “One unit within our organization created a report for their purposes. They wanted to look at trends and were satisfied with that level of quality so they marked the report as quality secured. Then, another unit started to look at the same report. This one looks good, so we start to create new content based on that earlier report. After a while they realized that the quality was not satisfactory for their new application. But since the report was already created and marked as quality assured, they thought it was good enough for their aims”. Such incidents can cause serious problems when making decisions, since users are more prone to believe content is good for all needs, if someone else approved its quality, even though their current needs are different. On the other side of the coin our findings reveal another challenge: reports are created even though they exist. Users are not aware of the available reports. Instead of analyzing existing reports, duplicates are created, which affect the implementation and use of SSBI negatively. Several respondents mention this and even the ones who have worked with SSBI for a long time agree: “It is common users are creating the same report over and over again. The wheel is reinvented many times”. Self-reliant users should not create redundant reports.

Challenge #6: No governance of SSBI reports: Organizations are facing a challenge to govern SSBI reports, especially when users are customizing existing reports: “Previously, I worked in a place where there were many technical users who created good reports. They were considered heroes when they released a solution. Then others started to add and change things, which started to cause problems. Later, there were ten different solutions which did not fit together. How can we know if we can use these reports? How can you give the freedom to create BI-solutions without good governance?” To implement and use SSBI efficiently requires good governance, similar to data governance which organizations have implemented for years.

Challenge #7: Unsupported tools are used: According to some respondents, it is common users are working with their own ad-hoc tools not supported by the IT department. If someone creates a report consisting of content which is created by these tools, it may cause problems. One respondent mentions: “Some users create reports with tools which are not supported. For example, when the tool hit the roof in the amount of data it could handle, this caused problems since nothing worked anymore. Then they requested help from IT. We do not support this software but tried to understand the problem anyway. One of the reasons we do not support the software is because it cannot deal with these volumes of data. It is not a stable and secure solution”. Users may start to use any software as desired when moving toward SSBI. Organizations need to be aware of these challenges. A self-reliant user is perhaps not allowed to work limitlessly. Instead, proper education is needed to achieve an efficient SSBI implementation, which leads us to the next chapter of challenges regarding SSBI and education.

SSBI education

Changes within organizations often come with education. Users need the right skills to perform their tasks and a proper understanding of how the new workflows affect their daily work. The third challenge category focuses on education and SSBI, as a lack of education affects the implementation and use of SSBI negatively. Two categories and five challenges are related to SSBI education, which are presented in Figure 4 and described hereafter.

Figure 4. Challenges of SSBI education

No formal education

The first identified sub-category related to SSBI education consists of three challenges which are described in this section.

Challenge #1: No formal educations are given: Organizations do not organize formal education as part of SSBI implementation. SSBI is supposed to simplify traditional BI. This can lead to the misunderstanding that users who master traditional BI do not require any comprehensive education when they move to SSBI. Our findings show how a lack of formal education may hinder SSBI implementation. All respondents endorse the following quote: “We do not have any formal education. But we need it since the existing education is on a very superficial level”. Instead, users meet informally to share experiences how to use SSBI. They show each other examples of how they tackled particular problems.

Challenge #2: Users forget how to use SSBI: Organizations arrange informal education focused on sharing experiences on how to use SSBI. Our findings show how users forget how to use SSBI even though they participated in such gatherings: “We have some education where many employees participate. Most of them are satisfied and want to use SSBI on their own. But not many are able to use SSBI on their own. I know a few close colleagues who took education and are probably able to use SSBI on their own, but they forget how to and ask me instead. So, they are not using SSBI in a self-reliant manner like SSBI is supposed to”. Organizations need more formal education to increase the possibility to let users be more self-reliant without support. Otherwise, there is no difference compared to traditional BI where power users create content on request.

Challenge #3: Not using SSBI after education: Respondents testify that users are not using SSBI after their education. They mention how users who participate in education are enthusiastic about SSBI, but are not using it afterward. At first, users like SSBI and see its benefits compared to traditional BI. But after a while, they have not created or used SSBI at all: “We had some educations in the beginning where you got licenses for SSBI tools. But then you do nothing, users start to lose knowledge and perhaps interest. Many users in my department are not creating their own reports. Instead, there are a few who have the competence who do. In the beginning, SSBI seems hot and attractive, but most of the users prefer standard reports where they are able to tweak and filter data. There are many users who participated in the SSBI educations and later have not touched the SSBI tools.”

It is important to plan for formal educations, especially when introducing SSBI, to let users become self-reliant and see the benefits of SSBI compared to previous ways of working. Even though SSBI is supposed to simplify the process of using BI, it requires formal education.

Low interest in SSBI

The second identified category within SSBI and education shows organizations have users who have low interest in SSBI.

Challenge #4: Users do not see the benefits of SSBI: Respondents illustrate how users fail to see how SSBI can help in their daily work: “I know many users who believe SSBI was great when first experienced during education. SSBI was so cool in the beginning, but they did not see any need to use it in their daily work.” Education needs to show users the benefits which SSBI brings compared to their previous way of working. If users do not understand the value of SSBI, they may resist the change toward SSBI: “You can easily notice users who do not care at all about SSBI. These users do not care to appropriate SSBI”. Education tailored to specific users and their daily work tasks is needed. Otherwise, users tend to reverse back to their old routines.

Challenge #5: Users have different technical backgrounds: A low interest in SSBI can follow from the fact that users have very different technical backgrounds and some thus have hard to adopt standardized SSBI tools. Users with more technical skills are able to become self-reliant more easily compared to non-technical users. Organizations need to be aware of different backgrounds of users, especially when defining formal educations. All users must be able to use SSBI in their way, even though they have different technical backgrounds: “Our data scientists are very self-reliant when using SSBI. Others are not. But we all use data differently so it matters what kind of background users have when using SSBI”. Perhaps SSBI is not a one-size fits all tool for every user in the organization. It is important to understand how SSBI can be valuable for all users within the organization when defining formal educations. Otherwise, users are probably not seeing the benefits of SSBI and may not adopt SSBI.

Discussion

This chapter compares the user-related SSBI challenges listed in literature with the user-related SSBI challenges revealed in the empirical case study.

A not yet so self-reliant user: how SSBI only shifts the support bottleneck

In traditional BI, casual users are dependent on power users who have to create reports based on their requests. In SSBI, casual users can create their own reports, but instead, other kind of requests arise. Our findings illustrate how casual users still are dependent on support from power users. Requests for support do however change in nature. They may involve help regarding getting access, locating and understanding data or assistance in how to perform certain complex operations in report creation. The traditional BI request–response relationship may be solved, but new request-response dependencies are created. The next two subsections elaborate on these new bottlenecks: data access and use challenges and the SSBI report challenges

Data access and use in SSBI: a data perspective and user perspective

Data access and use challenges have already been extensively discussed in traditional BI literature and are even recognized in literature on SSBI challenges. Whereas traditional BI and SSBI both mention data access and use challenges, the challenges may be rather different in SSBI compared to traditional BI. What is challenging for a casual user in SSBI is not the same as what is challenging for a power user in traditional BI. Power users experience data access problems despite their technical skills when data sources are poorly structured or not available at all (Olszak & Ziemba, 2012; Scholz et al., 2010). Casual users report on data access difficulties when working with relatively well-structured data sets. Thus, casual SSBI users may experience even more severe challenges when they start exploring the more complex data analysis tasks that are performed by power users in traditional BI.

Our findings illustrate how casual users in SSBI find it difficult to select desired data for their decisions to be made. Not using the right data affects decision-making negatively. However, it is difficult to make data available due to many reasons. Sometimes data is sensitive and all users are not supposed to be able to access it. Therefore, it may take a long time for users to request access to needed data. The IT department has to consider if the user is eligible for access or not, which is a time-consuming and cumbersome process. Things become even more complicated when users want to access and use data from multiple data sources in different environments. It often requires using appropriate data queries to join data correct. The simplest mistake can cause serious consequences which once again affects decision-making negatively.

To avoid many of the aforementioned challenges, organizations may increase their control of how to integrate data, of data security and of data distribution. The results show that this increased control may be counterproductive as it hinders users to access and use data as desired. Therefore, policies for data management and governance are important to achieve, which is indicated as a challenge in itself. But, even when data access works efficiently without problems, users find it difficult to interpret and analyze content when making decisions. If SSBI should work efficiently, data should be prepared for visual analytics.

It is important to realize that there are two sides of the coin with regard to data access and use challenges in SSBI. Challenges are partly data related, i.e. structuring available data sources consistently, in easy to understand overviews, with clearly understandable data labels, simple but effective data governance policies et cetera. Such challenges are well described in traditional BI literature and typically mentioned in SSBI literature. However, our findings show there is also a user-related side of the coin. Casual SSBI users do even experience severe difficulties in accessing, locating and understanding well-structured data sources. Organizations implementing SSBI should therefore not only focus on organizing the data at large, but also on guiding and educating the users. It may be wise to present data in different ways to different user groups, and develop a clear vision of required competence levels for each of them. Finally, different user groups may need their own dedicated tools.

The new quality challenge: from data quality to report quality

SSBI literature does not mention any challenges with regard to creating SSBI reports. This could easily be interpreted as if it does not require any special skills to create SSBI reports. The case study results show how users of SSBI experience severe difficulties when creating SSBI reports and how this in turn creates a quality assurance challenge. In traditional BI, a few power users took the responsibility to create content without errors. In SSBI, casual users face a dilemma when they get stuck in creating reports. Either, they need to ask help from power users (which forces them to return to the request and response waiting queue of traditional BI), or they can create a report autonomously with their limited skills (taking a risk that errors are incorporated and result in faulty reports). The latter is especially problematic if reports are heavily reused by others which makes that more and more decisions are based on erroneous reports. Data quality is a well-known challenge in traditional BI (which remains valid even for SSBI). Assuring report quality is a challenge that is more relevant for SSBI.

Self-reliant does not mean simple: the need for more education

Whereas SSBI literature already starts acknowledging that casual users need education to be able to become self-reliant SSBI users, our findings specify in much more detail what exactly this education should consist of. First and foremost organizations need to realize that SSBI is not just something simple which any user can apply. The fact that our study revealed so many education-related challenges does also imply that proper formal education is an important prerequisite to become a self-reliant user. Such education needs to accommodate for the fact that users who participate may have rather varied technical backgrounds, and thus education needs to be tailored so all these users can learn SSBI skills given their different starting points. Finally, education should address how SSBI is used in daily work. The main reason why users lose interest in SSBI is because they do not understand how to integrate it in their daily work routines. Education should therefore not be about “mastering the software” but should focus on “moving toward a new way of working” where SSBI skills become a natural element in formalized analytical daily work routines and a data-driven decision-making culture.

Conclusion and future work

Our case study has revealed 23 user-related SSBI challenges in three main categories: “becoming a self-reliant user”, “creating SSBI reports” and “SSBI education”. The identified challenges differ substantially from SSBI challenges reported in literature which implies that more research is needed to map SSBI challenges, to fully understand them, and to learn how to manage them. Practitioners need to realize that implementing SSBI is not an easy matter and that SSBI comes with different challenges rather than with less challenges. SSBI can still fulfil its purpose of engaging casual users and making organizations more data-driven, but its implementation should not be underestimated.

Contribution: more elaborated insight into user-related SSBI challenges

The self-reliant user

The key issue in SSBI is to become less dependent on power users. Whereas casual SSBI users now can create their own SSBI reports, they become dependent power users in new ways. Casual SSBI users do experience data access and use challenges that differ from the data access and use challenges commonly experienced by power users in traditional BI. Casual SSBI users may already encounter problems when accessing and using data from relatively well-structured data sets, may lack basic competence and skills to perform technical and analytical SSBI operations, and may be hampered by difficult to use SSBI tools. When casual SSBI users need to request support from power users to access and use data (rather than to create and modify reports), they are not self-reliant and the entire purpose of SSBI could be questioned.

Creating SSBI reports

Our findings reveal severe challenges with regard to creating SSBI reports, which are not mentioned in SSBI literature. Casual users struggle to create SSBI reports autonomously. Moreover, when many casual users with limited analytical skills combine data into a myriad of reports that thereafter are re-used by others for different purposes, serious quality assurance challenges may arise. The existence and use of faulty reports may impact decision-making negatively.

SSBI education

Although SSBI education already is mentioned as a challenge in SSBI literature, our case study has identified a number of education challenges. These findings help to understand in more detail what kind of SSBI education is needed, and do also emphasize the importance of SSBI education. SSBI is commonly misinterpreted to be simple: if all casual users will do it themselves, it cannot be that hard? To the contrary, organizations implementing SSBI should realize that extensive education is necessary when casual users, with limited technical and analytical skills, need to learn to perform complex data analysis tasks.

Implications for researchers and practitioners

Literature on SSBI challenges is still immature and more research on SSBI implementation challenges is needed (Imhoff & White, 2011; Logi Analytics, 2015; Svahn & Ax, 2018). Besides, future research should start to focus on how to manage the identified challenges. Recommendations for how to address each challenge would contribute to an increased adoption rate of SSBI.

Practitioners involved in introducing SSBI in organizations need to be more aware of the many new challenges of SSBI in addition to more common challenges they are familiar with from traditional BI. They should avoid picturing SSBI as an approach where “everything becomes easier” and instead plan for a more careful and thoughtful adoption process, tackling the various SSBI challenges described in literature and especially focus on proper education of those casual users that will apply SSBI.

Acknowledgments

A previous version of this article was published in the Proceedings of the 53^rd Hawaii International Conference on System Sciences (HICSS) 2020.

Additional information

Notes on contributors

Christian Lennerholt

Christian Lennerholt is a Ph.D. student in Informatics at the University of Skövde, Sweden. He holds a M.Sc. in Information Systems from University of Skövde where he teaches in courses like database systems, programming and business intelligence. Christians research focuses on self-service business intelligence. He has published at international conferences such as ECIS and HICSS, and in journals such as IJBIR and TDWI BI Journal.

Joeri van Laere is an Assistant Professor at University of Skövde, Sweden. He holds a Ph.D. in Information Systems from Delft University of Technology, the Netherlands. Joeri performs research at the interface of organization science, communication science and information systems. His research interests include business intelligence, crisis management, situation awareness, gaming-simulation, organizational change and distributed work. He has published at several international conferences such as ECIS, HICSS, and ISCRAM, and in journals including the European Journal of Information Systems and the Journal of Contingencies and Crisis Management.

Eva Söderström is an Associate Professor at the University of Skövde, Sweden. She holds a PhD in Computer Science from Stockholm University, Sweden. Eva’s research focus is primarily on digitalization and reducing the digital divide for groups of people risking digital exclusion. Other research interests include standardization and information security, with an emphasis on the administrative and organizational side. She has published in several international fora, such as ECIS, HICSS, CAiSE, EGOV and in journals including IJHCQA, GIQ, JSIT and JISS.