Open Government Data as an Innovation Process: Lessons from a Living Lab Experiment

Abstract

Open government data are claimed to contribute to transparency, citizen participation, collaboration, economic and public service development. From an innovation perspective, we explore the current gap between the promise and practice of open government data. Based on Strategic Niche Management (SNM), we identify different phases in the open data innovation process. This study uses a living lab in a province in the Netherlands to stimulate the provision and use of open data for collaborative processes and analyses the mechanisms that condition the success of this innovation process. The results based on six interventions over a period of two years show that our interventions stimulated the use of open data and raised awareness within government, but that various mechanisms inhibited the realization of the ambitions of open government data. We conclude that the challenge of open government data as an innovation lies in finding a way to scale up the provision of open data: innovation niches are established but “regime changes” do not take place. Scaling up open government data use requires strong managerial commitment and changes in the wider organizational landscape such as constructing formal and informal rules and technological developments that stimulate debate about open data practices.

Keywords:

• innovation
• open government data
• strategic niche management
• transparency

Open government initiatives have become a major worldwide administrative reform (Piotrowski, 2017). Currently, 70 countries are participating in the international Open Government Partnership (OGP). Due to initiatives such as the Open Government Declaration of the OGP (Open Government Partnership, 2011) and the EU Directive on the reuse of public sector (2013/37/EU), and fueled by internet technology, pressure is being placed on government organizations around the world to release their data (Janssen, Charalabidis, & Zuijderwijk, 2012; Linders, 2012; Sieber & Johnson, 2015). Open government data (OGD) are nonprivacy restricted and nonconfidential data, produced with public money and made available without any restrictions on its usage or distribution (Janssen et al., 2012, p. 258). OGD initiatives are expected to foster democratic and economic processes by promoting transparency, participation and collaboration, and provide opportunities for the development of new products and services (Dawes & Helbig, 2010; Halachmi & Greiling, 2013; Janssen, 2011; Lourenço, 2015; Ruijer, Grimmelikhuijsen, & Meijer, 2017). However, so far, the impact of open data is rather limited (Wang & Lo, 2016; Worthy, 2015), and this raises the question of why OGD has not yet lived up to its promise. This article uses an innovation perspective and a living lab approach to enhance our understanding of this gap between promise and practice. The aim of this study is to explore how the provision and usage of OGD can be stimulated for collaborative processes, by conceptualizing open data initiatives as an innovation process.

Recent studies have identified barriers concerning open government data usage (Barry & Bannister, 2014; Conradie & Choennie, 2014; Huijboom & Van de Broek, 2011; Janssen et al., 2012), but these analyses fail to capture the interactions and timing of the various barriers. Sieber and Johnson (2015) argue that open data is at a crossroads: governments should not only “throw data over the wall” in light of transparency, but should also contribute to participatory and collaborative democratic processes (Ruijer et al., 2017; Sieber & Johnson, 2015). The participatory and collaborative processes require different roles of government and citizens (Ruijer et al., 2017). Governments respectively become a facilitator and partner in the reuse of open data. In this study, we take collaborative processes between governments and users as our point of departure, thereby using an innovative perspective. Recent studies emphasize the complexity of open data systems (Dawes, Vidiasova, & Parkhimovich, 2016). An innovation perspective can contribute to acquiring insights into the challenges associated with OGD for collaborative processes since an innovation perspective acknowledges complex process dynamics (Wang & Lo, 2016). We combine two bodies of literature: studies on the barriers to open government data provision and usage (Janssen et al., 2012; Meijer, 2015), and innovation theories—in particular, Strategic Niche Management (SNM) (Borins, 2014; Schot & Geels, 2008; Walker, 2014).

Furthermore, we use an innovative living lab approach to explore how the provision and usage of open data can be stimulated over time. Living labs study innovation processes in a real-world setting (Ceshin, 2014). It is a user-driven, open innovation environment (Bergvall-Kareborn & Stahlbrost, 2009) that allows for collaboration between citizens, civil society and public-sector organizations (Gasco, 2017). The current empirical research into OGD has turned into a rich and diverse field of research with various research methods from, for example, case study (Dawes et al., 2016) to survey (Worthy, 2015) and design research (Zeleti, Ojo, & Curry, 2016). These methods however provide little insight in how complex open data initiatives can be strengthened through focused interventions over time. Living labs combine research and design methodology, that allow for interventions and studying the impact of these interventions (Dekker, Franco-Contreras, & Meijer, 2017). In this study, we will explore the value of the living lab approach to open data research by showing how we have used this approach to study the interactions between data providers and users, the context of use, available data, new technologies and their diffusion over time.

This study connects the literature on OGD with the literature on innovation in the public sector to develop an explanation for the gap between the open data promises and its actual impact. The use of a living lab produces a new understanding of the interrelations between the barriers to and drivers for successful OGD practices on the demand and the supply side. More specifically, our analysis puts the emphasis on the role of change agents, managerial support, and institutional pressure as key to understanding the success of OGD. For government organizations, this knowledge might help to design collaborative processes that help to realize the promise of OGD.

Theoretical framework

The challenge of finding a match between the supply and demand of open data

The underlying assumption of OGD is that there is a match between the provision and usage—supply and demand—of the development of different forms of open government data use that will contribute to society. Matching supply and demand is a complicated process influenced by varying factors (Susha, Janssen, & Verhulst, 2017). The current literature highlights that various barriers on the side of governments and users explain the current gap between the promise and practice of OGD.

From a government perspective, cultural and structural barriers can be identified (Meijer, 2015). Cultural barriers could include a closed government culture (Barry & Bannister, 2014; Huijboom & Van de Broek, 2011), a fear of misinterpretations (Barry & Bannister, 2014; Conradie & Choennie, 2014), a fear of abuse of data (Barry & Bannister, 2014), and a fear of security threats (Barry & Bannister, 2014; Huijboom & Van de Broek, 2011). Structural barriers refer to a lack of political will or leadership (Barry & Bannister, 2014), a lack of standardization of open data policy (Huijboom & Van de Broek, 2011; Janssen et al., 2012), a lack of priority within the organization, and a lack of resources to publish information (Barry & Bannister, 2014; Janssen et al., 2012). It also refers to opaque ownership, unknown data locations within and between departments (Conradie & Choennie, 2014; Zuiderwijk & Janssen, 2014), and legislation, such as privacy laws, as a barrier for releasing open data (Barry & Bannister, 2014; Conradie & Choennie, 2014; Huijboom & Van de Broek, 2011; Janssen et al., 2012) .

From the perspective of the user, cultural and structural barriers can be identified as well (Meijer, 2015). Cultural barriers refer to the digital divide and the lack of knowledge or capability to use the data (Barry & Bannister, 2014; Huijboom & Van de Broek, 2011; Janssen et al., 2012). Structural barriers refer to limited quality of the data (Barry & Bannister, 2014; Huijboom & Van de Broek, 2011; Janssen et al., 2012; Zuiderwijk & Janssen, 2014), arbitrary format of data (Janssen et al., 2012), limited user-friendliness/information overload (Huijboom & Van de Broek, 2011; Janssen et al., 2012; Zuiderwijk & Janssen, 2014), insufficient data published (Zuiderwijk & Janssen, 2014), insufficient tools for using open data (Zuiderwijk & Janssen, 2014), and lack of opportunity to provide feedback to the open data producer (Zuiderwijk & Janssen, 2014).

Hence, previous research resulted in an impressive overview of barriers but a limited understanding of their interrelations and the dynamics of the collaboration process in which these barriers and drivers feature at different stages of implementation. Hence, in order to find a match between open data provision and usage, several obstacles need to be tackled both inside and outside government organizations. In this study, we view OGD provision and usage as a complex process of public innovation (Grimmelikhuijsen & Feeney, 2016; Wang & Lo, 2016). From the innovation literature, it is known that the use of new technologies often implies overcoming resistance both inside and outside organizations (Kemp, Schot, & Hoogma, 1998). In an organization, new innovations often receive lukewarm support (Kemp et al., 1998). New technologies give rise to managerial problems and may require new user-provider relationships. OGD is a new government service provided via information technology platforms that requires a new collaborative approach and changes the relation between government and citizens (Linders, 2012; Sieber & Johnson, 2015; Wang & Lo, 2016). As such, open data can be viewed as an innovation process. The notion of adoption and diffusion of technology can provide insights in open data usage (Lee & Kwak, 2012; Wang & Lo, 2016), and can help to better understand the challenges of open data provision and usage.

Open government data as an innovation process

Innovations occur in response to changes in the external environment (e.g., user demand) and are based on internal organizational choices (Walker, 2014). Some scholars have used innovation theories in order to understand OGD adoption. For instance, Wang and Lo (2016) use the technology-organization-environment (TOE) framework. Grimmelikhuijsen and Feeney (2016) draw upon theories of policy innovation diffusion and innovation adoption to understand better the determinants of an open government. These studies are useful in that they provide insight into the role of perceived benefits and barriers of the new technology, those organizational and external environment factors that influence OGD or open government adoption, respectively. However, these studies do not provide an insight into the role of different determinants and barriers over time during the different stages of an innovation process. Most innovations do not start as a strategic activity but as a peripheral activity of a small team, protected from the pressures of the sociotechnical regime (Boon, Moors, & Meijer, 2014; Kemp et al., 1998; Mergel & Bretschneider, 2013), followed by growth and eventually possible regime transformation (Smith, 2007). In these different phases of adoption, different barriers and drivers might emerge.

To capture the process dynamics of OGD for collaborative processes and the interrelations between barriers, we use SNM. This analytical framework from science and technology studies helps to understand the successes and failures of sociotechnological innovations and can be used as a tool to manage the diffusion of innovations (Caniels & Romijn, 2006; Witkamp, Raven, & Royakkers, 2011). SNM emphasizes the role of niches as a source for sustainable innovation journeys (Smith & Raven, 2012). A niche is a space that allows for experimentation with technology and user practices. It allows new ideas and practices to develop without being exposed to the full range of selection pressures that favor the present regime (Smith, 2007; Schot & Geels, 2008). SNM has been widely used in the context of sustainability (Caniels & Romijn, 2006; Schot & Geels, 2008; Smith, 2007) but also in relation to social entrepreneurship (Witkamp et al., 2011). To our knowledge, SNM has not yet been applied to open government data initiatives.

Niche development, from niche to sustainable change, results from the interplay of three internal niche processes: the articulation of expectations, networking, and learning (Boon et al., 2014; Kemp et al., 1998; Schot & Geels, 2008). The articulation of expectations provides the direction of collaborative learning processes (Schot & Geels, 2008). Actors must translate their own expectations to other actors and engage in collaboration (Weber, Hoogma, Lane, & Schot, 1999). Networks of actors are important for niche development. By maintaining links to external user communities, government organizations maintain open channels through which external knowledge can be accessed and it provides the opportunity to respond to user needs and challenges (Roberts, 2013). This could lead to the emergence of a network of actors willing to invest in and carry the technology forward, leading to a smoother diffusion process since the technology and its environment achieve a better fit (Weber et al., 1999). Finally, learning processes are essential for niche development (Schot & Geels, 2008). If learning takes place the expectations may eventually become more articulated, specific, and stable (Schot & Geels, 2008). Niche growth could eventually lead to regime transformation (Smith, 2007) when niche innovations are linked up with ongoing processes at the sociotechnical regime and at the wider context or landscape (Schot & Geels, 2008). If we translate these processes to open data initiatives, then we can identify the importance of expressing what type of impact is expected from the data, that there should be networks present consisting of data providers and users who collaborate and interact and that learning should take place. The wider landscape consists of formal and informal rules and regulation at the macro level that influence open data practices. Changes at the landscape level create pressure on the regime (Schot & Geels, 2008).

SNM identifies several steps in the innovation process (Kemp et al., 1998): the selection of an experiment, the set-up of the experiment, scaling up the experiment, and breaking down the protection. Meijer (2014, 2015) shows that different barriers play a role in different phases of the innovation process. Combining the work of Meijer (2014, 2015) and Kemp et al. (1998), we propose the following stages:

• Building a niche refers to choosing the appropriate setting in which the new technology is to be used. Crucial aspects in this phase are keeping it simple, identify the presence of an innovation champion or agent, and identifying user needs (Caniels & Romijn, 2006). Furthermore, according to Meijer (2014), organizational and political barriers are crucial here. In the case of open data provision, this could, for instance, refer to cultural government barriers, such as selecting datasets, where security and privacy issues are not at stake, and where there is no fear for misinterpretations.

• Conducting an experiment, the phase that stimulates learning processes, involves articulating expectations and building networks (Caniels & Romijn, 2006; Kemp et al., 1998). In this phase, the innovation runs into technical and organizational barriers (Meijer, 2014). When testing an open data platform users might run into structural user barriers such as the working of the technology, limited user-friendliness, lack of and the quality of the datasets. For the provider, organizational barriers might include lack of priority.

• Scaling up the experiment refers to how innovations will diffuse from the niche to the wider sociotechnical system or embedding in the organization (Kemp et al., 1998; Lovell, 2007). From a government provider perspective, scaling up might require extra resources and even institutional transformation. Capacity barriers may prevent the innovation from moving forward (Meijer, 2014).

Meijer (2014) points out that innovation processes are chaotic, iterative processes and therefore stage models may not be regarded as a “blueprint.” However, the value of this approach is that it acknowledges that barriers in terms of the provision and use of open data may differ over time. A problem with studying innovation processes is that it is very difficult to identify when and how certain factors stimulate or inhibit an innovation process. One way to empirically study this innovation process is by using labs (Ceshin, 2014), because this method offers the opportunity to explore barriers and drivers through targeted interventions over time.

Living lab experiment

Current empirical research into OGD reflects various research methods, such as case study research (Dawes et al., 2016), Website analysis (Lourenço, 2015), survey research (Worthy, 2015), focus groups (Janssen et al., 2012), document analysis (Ohemeng & Ofosu-Adarkwa, 2015), and design research (Zeleti et al., 2016). These methods all have their strengths and weaknesses: case studies are useful for studying the interactions between multiple variables; Website analysis helps to map the variety of open data portals; survey research is important for measuring perceptions of users and providers of open data; focus groups provide insights in shared perceptions; document research helps to produce insights in official (government) considerations; and design research provides the basis for the design of open government data services. These methods, however, do not result in a systematic analysis of the effects of controlled interventions over time and this is why a living lab can be an important research strategy for open data research.

A living lab can be understood as settings or environments for user-driven innovation (Bergvall-Kareborn & Stahlbrost, 2009). Living labs offer a “collaborative platform for research, development and experimentation in a real-life settings, based on specific methodologies and tools, and implemented trough specific innovation project and community building activities” (Gasco, 2017, p. 91). They are characterized by experimentation in real-world settings and by users as coinnovators (Gasco, 2017). Users collaborate to create a desired outcome (Higgins & Klein, 2011). As Gasco (2017) shows, whereas living labs have traditionally focused on supporting companies and creating an ecosystem of innovation that benefits private companies and public organizations, they lately have also emphasized the needs of citizens. Living labs offer an alternative to public administration experiments that are often conducted in a controlled laboratory environment (Bouman & Grimmelikhuijsen, 2016). According to Manzini and Staszowski (2013) living labs provide two opportunities. First of all, they provide the possibility for bottom-up innovations to move faster from “the first ‘heroic’ stage (when social inventions are still prototypes) to the following stages when more mature enterprises are created and, if necessary, when enabling products and services are conceived and enhanced” (Manzini & Staszowski, 2013, p. vi). Second, they provide an opportunity for public agencies to meet with people and other organizations in order to experiment together (Manzini & Staszowski, 2013).

Research design

A living lab consist of five components (Bergvall-Kareborn & Stahlbrost, 2009): (1) users who represent potential end users; (2) an application environment that represents the context in which users interact and reflect on the real world’s usage scenario; (3) ICT technology that facilitates collaboration; (4) organization and methods that emerge as best practice within a living lab; and finally (5) the living lab partners who bring their own knowledge and expertise. In our study, the application environment consisted of a rural province in the Netherlands, one of the 12 provincial government agencies in the Netherlands. Public administration in the Netherlands has four tiers: central government, provinces, municipalities, and water authorities. The living lab was selected based on relevance and accessibility. The focus of the living lab was the policy area of population decline. This policy issue was identified by civil servants of the province as a top priority for both government and citizens. Civil servants indicated that OGD could help in providing insights in issues related to population decline. The users consisted of representatives of grassroots initiatives actively involved in dealing with population decline. Civil servants identified them as potential open data users. The other participants of the living lab consisted of civil servants (policy experts on population decline and open data experts), students, and researchers. The information and communications technology (ICT) used concerned an open data platform that facilitates data analysis, visualizations, discussions around open data, and the cocreation of a report based on open data.

The organization and methods of the living lab aimed at stimulating a match between open data provision and usage consisted of four interventions:

• Building a niche. We assessed the starting conditions by organizing a workshop in which we identified user needs for an open data platform, government and citizen barriers in working with data, and options to overcome these barriers. In addition, an open data champion was identified, and a project group was set up consisting of civil servants (providers of open data), representatives of grass roots initiatives (possible users) involved in population decline, and researchers. The project group was responsible for the implementation of the open data innovation project.

• Conducting experiments. Two experiments were conducted with a bottom-up approach. First, the project group identified two scenarios based on issues the two grass roots organizations were working on (see Table 1). Additionally, an experiment was conducted in which the project group tested the open data platform. Second, in a consecutive experiment, we brought in extra capacity for open data usage and asked students to participate in a population decline challenge based on the scenarios identified by the project group. The students were selected in an interview based on their motivation and their quantitative data skills. They participated for five weeks in spring 2016 on the open data platform in interaction with civil servants and representatives from the citizen’s initiatives. In order to keep the students motivated during the five weeks, a payment was promised after accomplishing the task. The group generated ideas for the scenarios based on open data.

• Scaling up the experiment. In this phase, debate was stimulated in the organization about open data provision and usage. A workshop was held in which the students presented their findings and the Director responsible for population decline was invited. A subsequent workshop was organized to prepare for an open data event around four scenarios organized by the province.

Table 1. Overview Scenarios.

Issue	Grassroots organization	Scenario
Healthcare in a small village in the province	Village cooperate	As a result of population decline, the healthcare providers and a housing cooperate are withdrawing their services from the small village, which has a huge impact on the village. Not only does it imply less accessible healthcare but also a loss of jobs and social coherence. Therefore, a village cooperate was established by citizens focused on trying to keep healthcare in the village by integrating disability and care for the elderly in one building. This citizens’ initiative wants to examine the consequences and risks of the integrated approach. There is a need for open data regarding healthcare budgets, the number of healthcare professionals working in the area, and demographics about the elderly and handicapped now and in the future.
Circular economy in a region in the province	Regional cooperation	The province is characterized by farmland. Despite the population decline, the village cooperate wants to maintain the livability in the area by focusing on an integrated approach of energy conservation, water management, food, environment, and healthcare in the region. The cooperate aims to develop sustainable innovative food production in the region. The cooperate needs assistance and data to develop ideas and solutions to strengthen the local economy.

Data collection and analysis

The data collection (see Table 2) consisted of minutes of meetings with civil servants and grassroots organizations from March 2015 to January 2017, transcripts of interviews, a usability survey when testing the technology, and logs kept by students during the population decline challenge. Furthermore, group meetings with students took place before, during, and after the challenge; minutes were recorded and used for data analysis. Finally, the online activities were monitored and analyzed, thereby using content analysis that focused on the functionalities of the platform used, datasets used, and how the data was used. The data were analyzed using a thematic analysis. Based on the empirical data collected, themes were identified that related to the research focus. The themes emerged from the codes identified in the transcripts, minutes of meetings, and logs of the students (Bryman, 2012).

Table 2. Overview of Interventions and Methods Used Between March 2015 and January 2017.

SNM stages	Interventions	Method	Participants
Building a niche	Scenario-based design workshop Setting up a project group	Meeting Interviews Workshop Project working group, meeting	2 civil servants, 2 researchers 3 stakeholders 7 civil servants, 9 stakeholders 5 civil servants, 3 stakeholders, 3 researchers
Conducting the experiment	Experiment 1: Developing scenarios and testing the technology	Project working group, meeting Project working group, usability survey	5 civil servants, 3 stakeholders, 3 researchers 2 civil servants, 4 stakeholders, 3 researchers
	Experiment 2: Population decline challenge	Two group sessions	10 students, 5 in each group
		Interviews	2 civil servants, 4 stakeholders
		Two group sessions	10 students: 5 in each group
		Logs send by mail	10 students, 1 researcher
		Monitoring the open data platform	1 researcher
		Two group sessions	9 students: group of 5 and 4
		Logs send by mail	9 students, 1 researcher
Scaling up the experiment	Stimulating debate in the organization and the community Developing a plan for a data expedition	Project working group, evaluation Interviews Workshop	2 students presenting results, 5 civil servants, 4 stakeholders, and 4 researchers 2 civil servants, 3 stakeholders 2 researchers, 4 civil servants, 4 stakeholders

Findings

Building a niche

During an initial meeting with civil servants of the province, the societal issue of population decline was identified as one of the more important policy issues. The declining population affects other areas in the province as well such as housing, education, healthcare, and employment. Two departments within the organization participated in the project: the Department of Regional Planning and Society, responsible for population decline, and the Department of Information and Communication Technology, responsible for open government data. The open data expert from the ICT Department of the province was willing to take on the role as open data champion; he was committed to coordinating the living lab within the provincial organization.

We started the living lab by assessing the initial conditions of the provider, the provincial organization, and their experience with open data. The department responsible for population decline had no prior experience working with open data. The ICT department had set up an open data portal consisting of 70 datasets. However, the use of the portal was limited and the portal was not kept updated. Furthermore, the province at the start of the project did not yet have a specific open data management policy, nor was there a specific budget available for open data. Finally, the province used to have its own research department, but due to budget cuts, research is currently conducted by a separate body, the Plan Bureau, which collects data. There is political support within the province for open data, which is expressed in the provincial program 2015–2019. Moreover, the expectation is that the new national Dutch freedom of information law, Wet Open Overheid (woo), will stimulate the release of open data at the regional and local levels, because this law makes explicit which information should be released proactively. The law is currently awaiting approval in parliament.

The first intervention consisted of a scenario-based design workshop to identify user needs concerning the design of the ICT technology—an open data platform. Civil servants from the ministry, province, and local municipality (7 in total), and external users from citizens’ initiatives and NGOs (9 in total) were invited to participate in the workshop. The province identified and invited these participants as representatives with an interest in population decline and open data. The results of this workshop were used as input by a multidisciplinary team of researchers (information technology, education, psychology, and public administration) to design the open data platform.

Furthermore, based on the workshop and additional interviews, barriers in working with data and options to overcome barriers were identified. Important barriers from the provider perspective were that open data was not yet part of the work process in the population decline domain, that there was a lack of best practices, the unfamiliarity of the value of OGD, and the lack of management support. As a civil servant indicated:

“One of the biggest challenges is getting management involved. It is the organizational layer that does not want to take risks with open data and especially wonders what it costs.” (R3)

User barriers identified at the start of the project related to the expected difficulty of building an interested community around open data, access, and technical difficulties.

“It might be difficult to pinpoint down a specific community…perhaps the community is there, but we just don’t know that it is.” (R2)

In order to anticipate these challenges, a second intervention took place. The second intervention consisted of setting up a project group as the start of community building. The project group was responsible for the implementation of the open data innovation. The project group consisted of civil servants of the province (both open data and population decline experts, including the open data champion), a civil servant of a ministry of the central government (responsible for national open data policy) as providers of OGD, and representatives of two grassroots initiatives related to population decline as potential community users of open data. The representatives were invited by a civil servant of the province to join the project group because they were actively working on population decline projects in their community. Before the start of the project, the representatives had already been in touch with government bodies regarding their project but this had not yet led to valuable results. Hence, these first interventions resulted in support for the project at the lower level of the organizations and from local citizens’ initiatives.

Conducting an experiment

In the next phase, we set up interventions to stimulate learning processes with OGD and to create a best practice. The third intervention, was aimed at experimentation and learning to work with open data technology in relation to concrete scenarios based on community issues. The project group collaborated in developing scenarios based on issues representatives of the citizens’ initiatives were facing. The project group also identified information needs related to the scenarios. For both scenarios, the project group members expressed their expectations and identified gaining insight based on open data as the main goal.

“Especially for the bio-based economy, it is important to have insight in food production. So far, it is mainly based on stories, but the data behind it is missing.” (R5)

This eventually resulted in the construction of the two scenarios related to the daily practice of the grass roots organizations (see Table 1).

Furthermore, the project group tested an early version of the technology. A short survey was conducted and a discussion took place after working with open data. In general, users expressed an interest in interactions and visualizations based on open data facilitated by the technology. In terms of learning, during this meeting an important barrier related to user skills became known. Some of the users indicated that they felt less comfortable working with data and making visualizations.

“For me, it is important to get the information in an easy way. I want to . . . type in and get it. That is what I’m looking for. How you analyze that technically, you need others for that.” (R5)

The intervention showed that from a user perspective there is a gap between the competencies necessary to work with open data and the competencies of the users. Furthermore, from the provider perspective, the lack of available data was identified as an important barrier. In order to come to insights and solutions for the scenarios, the project group concluded that extra capacity was necessary to overcome the barriers regarding the lack of skills and the time-consuming searching for relevant data. In addition, the session resulted in recommendations for the further development of the technology.

As a solution for these barriers, during the fourth intervention, extra capacity was added by involving university students who would be able to work with data and make visualizations. For five weeks civil servants, representatives of the grassroots organization, and students committed themselves to collaborating around the scenarios and to working with open data on the online platform. One group of students worked with users and civil servants on the scenario circular economy and one group worked on the scenario healthcare. The groups were asked to further define the issue, find relevant data, discuss, and generate ideas based on data, this resulted in two reports for each scenario.

Most students experienced the project as an interesting new way of working in which data was the starting point for their thinking and in finding solutions for their scenario.

“I did like searching for data, what is out there, to come up with nice ideas and to reason based on data.” (S2)

The users indicated that the outcomes in the form of student reports, based on open data, as valuable and insightful, thereby contributing to the expectations that were discussed at the start of the project.

“It was all very relevant. Cooperation, care, and energy—it all came back in your report based on numbers. It provided a nice overview of opportunities for the bio-based economy.” (R6)

One user involved in the healthcare initiative indicated that the report based on open data gave them some insight in the financial flows, but also indicated that more information is necessary. Furthermore, the representatives learned the importance of taking the time to formulate the right question. One user pointed out that too often there is a focus on solutions whereas really getting an understanding of the public problem is often lacking. Therefore, collaboration between public administration and society is important and only then new insights and transparency can be achieved. The users indicate a willingness and urgency to continue working on their scenario in collaboration with the province and researchers around open data in the future.

From a provider perspective, civil servants indicated that they had learned a lot by participating in the project. One policy maker indicated that open data is a blind spot in his daily work and that he would like to explore further possibilities. Open data experts realized that there is not always a match between the data provided by the province and the data needed by users for a specific issue. Furthermore, it was concluded that in order to effectively use open data for policy problems, more is needed than just publishing data on a portal. Communication between the provider and information user is important in order to find a match. Exchanging knowledge can result in better quality data and more efficient reuse of data. The challenge led to goodwill among the users; they appreciated the willingness of the province to collaborate with users around open data regarding public problems.

However, barriers were identified as well. Despite the fact that the students were selected because of their interest and data skills, several of them indicated that they experienced working with open data as difficult. One student noted:

“Transforming it [data] and making it usable is pretty difficult.” (S1)

In addition, it was difficult to find relevant data, select relevant data out of huge datasets on national open data portals, or to interpret data due to a lack of metadata.

“These big data files are just incomprehensible.” (S8)

“The city data are outdated. I only found data for the year 2014, and sometimes the year was completely missing.” (S7)

Furthermore, in order to be able to obtain insights for the scenarios, next to data from the province, data from municipalities, the statistics bureau, and healthcare organizations were also essential. It was time consuming to search for these datasets, and not all information was made available as open data. In addition, the technology used with the open data platform did not yet allow them to make the visualizations they wanted or retrieve the data they needed. The student challenge resulted in several suggestions for the further development of the platform.

Remarkably, despite the commitment of the civil servants and users, there was limited online interaction between students, civil servants, and users during the challenge. Civil servants had logged in and observed what happened on the platform, but did not interact or respond. When asked why they did not interact, they indicated technical issues and a lack of time. Students experienced the lack of online interaction as a barrier and contacted the users and public administrator offline in order to get feedback on their ideas.

After the fourth intervention, the project group had become a stable small community. Learning among users and providers took place and the intervention showed the potential value of open data; representatives pointed out that some elements of the reports based on data showed them new insights. At the same time, there was a lack of online commitment.

Scaling up the experiment

The fifth intervention aimed at scaling up consisted of enhancing the support within the provincial organization by presenting the results of the experiment during a meeting and inviting the Director of Urban Planning and Society of the province, responsible for population decline, to the meeting. The director indicated that the project, using open data as an instrument for policy issues, showed the value and options of open data. It provided him with the insight that the province, as a provider of open data, can play a role in this development while supporting citizens’ initiatives.

“I think it is very important to use open data for our policy issues.…I hope we can continue with this policy so that we can become more transparent, and use it to create policy.” (R7)

This resulted in the further dissemination of information about the project in the organization. The student reports were used to put open data on the political agenda. A civil servant sent the reports together with a letter to a political representative in preparation for a meeting with the political representative. In addition, news items were placed on the intranet regularly and a presentation was given to civil servants of the province, interested in the project.

Finally, in order to further build the community, the project group and the director agreed to develop a plan for and to follow up on the current scenarios, but to also develop more scenarios and invite more users. However, despite the growing interest from the top of the organization, the willingness of the grassroots organizations and the open data experts to continue with the project declined; the diffusion of the open data project came to a halt due to new barriers that emerged. First of all, the open data champion, who had intensely participated in the project for two years, switched jobs. In the short term, no replacement was organized; consequently, the project no longer had a champion inside the organization, which tempered the flow and energy of the project. In addition, it was unclear whether the ICT department or the Department of Urban Planning and Society should take ownership of the project and provide a new champion. This further slowed down the project and the responsiveness of the organization. In addition, during the project, it became clear that relevant datasets were spread out over different government organizations, and that the province itself had only a few relevant published datasets regarding population decline. Middle management realized that opening up more high-quality datasets related to public issues would require more time and resources.

In sum, these interventions as part of the living lab showed the challenges in the different stages between finding a match between provision and usage of open data and the options to overcome these challenges. This resulted in a partial match in the niche in the form of insights regarding the scenarios written down in the reports. The results also raised some awareness within the organization. However, the open data provision and usage failed to scale up to the wider organizational system because the open data champion left the organization and because there was a lack of middle management commitment. Our findings however do not imply that working with open data has come to a complete halt in the province. This study shows that the diffusion might take more time than initially anticipated. OGD needs to be incorporated in the work process. Developing and implementing an information strategy takes time, especially in hierarchical bureaucratic organizations. The findings suggest that diffusion will not take place very fast in the absence of a broader landscape, where middle management commitment is low. An overview of the finding can be found in Table 3.

Table 3. Overview of Results from Living Lab, Identified Barriers, and Developed Solutions in the Different Phases of the Living Lab.

SNM stages	Activities	Barriers	Developed solutions within living lab	Results
Building a niche	Assessment of initial conditions	Provider: no open data policy, no budget, no central research department collecting data; User: limited open data usage;	Connect data with societal issue relevant to providers and users, develop best practices that show relevance of open data and might lead to resources	Overview of barriers and options regarding working with data within the provincial agency
	Identification of user needs	Provider: lack of management support, fear of losing control, and open data not part of the work process; User: lack of community	Setting up a working group consisting of providers and users; members of different government departments to gain support and with community members (representatives of grass roots)	Committed working group and open data champion committed to coordinating the project in the provincial organization
Conducting the experiment	Experiment 1: Developing scenarios and testing the technology	Provider: lack of available data; User: lack of skills working with data among community members	Working with extra capacity: students with data skills who can search and work with data	Two scenarios of interest to government (provider) and to grassroots organizations (user)
	Experiment 2: Population decline challenge	Lack of fit between the data provided and data needed by users for a societal issue, difficulty of working with data (users), limited online interaction between provider and user	Considering the potential value, the working group would like to continue the collaboration around population decline but also explore other societal issues	Two reports written by students well received by grassroots organizations but also missed some information Recommendations to further develop the open data platform

Conclusion and discussion

Open government data platforms have sprung around the world. However, the impact of open government data is so far rather limited (Worthy, 2015; Wang & Lo, 2016). The underlying assumption of open data is that there is a match between the open data provided and the open data needed by users. This study explored how the provision and usage of open data can be stimulated for collaborative democratic processes by conceptualizing the usage of open data as an innovation process. It connects the literature on OGD with the literature on innovation (Borins, 2014; Kemp et al., 1998; Schot & Geels, 2008) in the public sector to develop an explanation for the resistance to change. This study showed that finding a match between provision and usage of open data is not simply about barriers that need to be overcome but it is a complex process of learning, interaction, and networking within government organizations, and between government organizations and the wider community over time. Based on SNM, we distinguished different stages in the OGD innovation process. We analyzed these stages by using a living lab method over a period of two years that allowed us to conduct several interventions. These interventions stimulated the use of open data and raised awareness within government. However, various mechanisms inhibited the realization of the ambitions of open government data. Our study provided some important insights that strengthen our theoretical understanding of OGD and the methods that we can use to study this innovation process.

First, this study contributes to the literature by showing the gap between the theoretical promises of OGD in contributing to transparency, citizen participation and service delivery, and the actual practice of OGD. Based on SNM, the living lab highlighted the willingness to experiment at a limited level with OGD. A partial match was realized in the niche between provision and usage, resulting in insights for the scenarios. However, the organization was reluctant to scale up the experiment. The innovation challenge lies in finding a way to proceed beyond the stage of experiments and to realize that wide organizational effort is needed to realize the promises of OGD. Scaling up OGD data may also require changes in the wider macro landscape such as constructing formal and informal rules that stimulate debate about open data practices.

Second, our results add to the increasing literature on barriers of OGD initiatives (Barry & Bannister, 2014; Huijboom & Van de Broek, 2011; Zuiderwijk, Janssen, Choenni, Meijer, & Alibaks, 2012); by taking an innovation perspective, different barriers were identified in the different stages of OGD initiatives. Cultural and structural organizational barriers played a role in building a niche, but solutions were found, and, therefore, these barriers did not hamper experimentation in the niche. This is in line with the proposition posed by Mergel and Bretschneider (2013) who argue that the first stage of ICT innovation is highly driven by individual needs and not by a dedicated formal organizational decision. In line with Mergel and Bretschneider (2013), we found that experimentation led to increased awareness inside the organization and to insights in societal issues based on open data. Unlike Mergel and Bretschneider (2013), our best practices did not lead to scaling up of the OGD innovation. Cultural and structural organizational barriers halted the project.

Third, our study provided insights in the role of different actors in the process of OGD innovation. Strong support from the top of the organization, middle management, and external pressures are crucial. Internal organizational processes can hamper the release of relevant and valuable data. In addition, the external pressure of citizens open government data usage is still in its infancy. When both internal and external antecedents of innovation are insufficient (Walker, 2014), it is complicated for open data initiatives to live up to their promise. In our study, we managed to stimulate external demand and this indeed resulted in some internal changes but, for the moment, these remain limited.

Finally, in line with Gasco (2017), we found that the living lab approach is a valuable, iterative method for analyzing the interactions between data users, available data, government objectives and new technologies and their diffusion over time in a real-life setting. It provided in-depth insights in the challenge of finding a match between supply and demand of open data (Susha et al., 2017). It provided an opportunity to study barriers but also solutions to overcome these barriers over time. Like Bergvall-Kareborn and Stahlbrost (2009), we also found limited interaction via the ICT technology between users and partners or providers within the living lab. Nevertheless, users did appreciate the personal interaction with the partners during the face-to-face project meetings. However, some methodological limitations can be pointed out as well. Because living labs concern a real-life setting, there is no end point of the study, only a virtual one. In all cases, working with OGD will continue which might lead to further diffusion in a later stage, after this study. Also, the role of the researcher as a partner in a living lab can at times be complex. There is the risk of being too partisan or “going native” (Bryman, 2012). Lastly, setting up a living lab is context specific. The application environment or context allows for a more natural setting but is also limitation due to a lack of generalizability. An important challenge for living lab experiments is the organization or finding the interventions that provide maximum information about innovation dynamics and its impact (Gasco, 2017). In that sense, this research method is still in development and not as mature as other approaches such as socio-psychological experimenting or case studies. There is a need for developing a systematic understanding of interventions in living labs. This requires more comparative work to validate the various living lab options.

Acknowledgments

This project has received funding form the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 645860. The authors would like to thank their ROUTE-TO-PA colleagues for their valuable contributions.

Additional information

Notes on contributors

Erna Ruijer

Erna Ruijer is assistant professor at the Utrecht University School of Governance. Her research interests include, open government data, government transparency, government communication and social equity.

Albert Meijer

Albert Meijer is professor of public innovation at the Utrecht University School of Governance. His research focuses on transparency, open government data, coproduction and social media.