23,621
Views
116
CrossRef citations to date
0
Altmetric
Articles

The potential impacts of digital technologies on co-production and co-creation

, ORCID Icon &

ABSTRACT

Despite growing interest in the potential of digital technologies to enhance co-production and co-creation in public services, there is a lack of hard evidence on their actual impact. Conceptual fuzziness and tech-optimism stand in the way of collecting such evidence. The article suggests an analytical framework that distinguishes between the impacts of different technologies on different elements of co-production and co-creation, and illustrates this in three different areas. It argues that there is no reason to assume that digital technologies will always encourage co-production or co-creation. In fact, they can also be used to bypass interaction with citizens.

This article is part of the following collections:
IRSPM symposium

Introduction

It is no exaggeration to say that the interest in how digital technologies shape citizen-government relationships has grown immensely. From geriatric rehabilitation robots, through social media, to citizens donating data for public service design and implementation: it is often assumed that new technologies will benefit co-production and co-creation, by making these processes more effective and more efficient and by fundamentally transforming how citizens help shape public services (Lember Citation2018). It is expected that digital technologies can empower individuals and collectives, and substantially increase the opportunities for more personalized and demand-driven public services (Noveck Citation2015; Meijer Citation2012). By extension, they are expected to increase the legitimacy of the state (Kornberger et al. Citation2017). This could be wonderful – if it happens. However, before we accept such statements for fact, we ought to acknowledge at least four major problems.

For a start, there is a considerable conflation of terms. Discussions often make no difference between various types of participation such as communication, consultation, co-production, co-creation and so forth. This can lead to misunderstandings, because there are many different ways in which technologies can empower citizens or change service relationships (Lember, Tõnurist and Surva Citation2017). For instance, whereas gathering ideas through social media can indeed give citizens a voice in public affairs, this does not necessarily mean that citizens are empowered as co-producers of public services or that it transforms public administration in the long term.

In addition, current debates on citizen co-production and digital technologies have so far mostly addressed this relationship with reference to social media (Linders Citation2012; Clark, Brudney, and Jang Citation2013), crowdsourcing (Noveck Citation2015; Clark, Zingale, and Logan Citation2017) and open data (Kornberger et al. Citation2017). However, the field as a whole is far broader. There is at this point no systematic approach that shows how (different types of) co-production and co-creation processes are affected by digital technologies. For example, although studies on crowdsourcing have indeed contributed a lot to our understanding on digital co-production, these studies often suffer from terminological conflation and a strictly instrumental perspective.

Furthermore, empirical evidence on the effects of new technologies in this area is still scarce, at best (Noveck Citation2015; Kornberger et al. Citation2017; Meijer Citation2012). There are a number of case studies and best practices, but even these focus on the implementation of specific technologies, often ignoring how they should be institutionally embedded and how the emerging practice informs the very concept of co-production. Systematic empirical evidence is still very hard to come by.

At the same time, the debate is pervaded by techno-optimism. This tends to stress the enormous benefits digital technologies could have, but tends to ignore the profound uncertainties and risks that come with technological innovation. What we do know is that even technologies that seemingly work well may have negative societal impacts (Jasanoff Citation2016; O’Neil Citation2016; Soete Citation2013; Zuboff Citation2019). Conflicting interests and diverging values among stakeholders, the inability of data and algorithms to mirror the complexity of societies, unevenly spread technological capabilities and other factors make digital co-production a fundamentally ambiguous, open-ended and contested process (Lember Citation2018). In fact, new technologies could have the opposite effect of what techno-optimists assume. The increasing capabilities of governments to gather, analyse and employ vast amounts of data through social media, sensor networks, data analytics and machine-learning solutions may, together with the tendency to prefer top-down approaches in technology implementation, actually diminish the (perceived) need for and influence of active citizens in shaping public services (see e.g. Cardullo and Kitchin Citation2018). As with co-production and co-creation (Steen, Brandsen, and Verschuere Citation2018), a more critical and balanced perspective is in order.

The question is how to get there, in the face of such brutal optimism. Public administration and management research needs to engage more seriously, not only with new technologies, but also with specialists on these technologies (and it needs no saying that the reverse is also critically important). To make this engagement productive, it is vital to be more specific. To speak about THE effects of THE new technologies on THE involvement of citizens is to mirror the errors of the techno-optimists. There are various ways of involving citizens, there are different types of effects and there are different types of technologies. Specific technologies will probably affect co-production and co-creation in different ways, in different contexts. However daunting the task, we need to start addressing these phenomena systematically, by creating a framework that captures the variety of (both positive and negative) effects digital technologies are already having on co-production practises.

First, we will state our definition of co-production and co-creation, and how we will deconstruct these concepts. Then we will provide a short overview of the main digital technological trends that need to be taken into account. We will then explore the relationship between different types of technologies and co-production/co-creation, proposing an analytical overview that shows the potential impacts with more precision. Finally, we will illustrate the framework through changes in three fields of co-production: policing, youth development and elderly care.

Deconstructing co-production and co-creation

The meaning of co-production and co-creation

The concepts of co-production and co-creation have a different history. In the context of public services, the former is most closely associated with the work of Elinor Ostrom (Ostrom Citation1996) and subsequent studies by researchers in political science and public management (Alford, Citation2002; Bovaird Citation2007; Verschuere, Brandsen, and Pestoff Citation2012). The latter has been developed most elaborately in the field of marketing research (e.g. Grönroos and Voima Citation2013), from which it has spread to public management (Osborne and Strokosch Citation2013). Generally, it is agreed that the concepts are both about involving citizens, somehow. But that is where the agreement tends to stop. There has been a variety of different interpretations in circulation. Fortunately, in recent years, there have been several studies aiming to define the terms more precisely and distinguishing them more clearly, not only from each other, but also from citizen engagement more generally (Voorberg, Bekkers, and Tummers Citation2015).

Here, we will follow Brandsen and Honingh’s interpretations of co-production (Citation2016) and co-creation (Citation2016, Citation2018). They define co-production as ‘a relationship between the employees of an organization and (groups of) individual citizens. It requires direct and active inputs from these citizens to the work of the organization. The professional is a paid employee of the organization, whereas the citizen receives compensation below market value or no compensation at all’ (Citation2016, p. 431). In the context of public services at least, co-creation is the newer and more elusive term, but there are a number of elements that the terms share in common (Brandsen and Honingh Citation2018).

The first is that they constitute a direct part of the production process. In other words, they do not include all inputs by citizens that in some way affect the overall planning, design and delivery of a service, but focus on the direct input of citizens during the production phase. ‘Direct’ here means that the input by citizens affects the service individually provided to them (as an individual, family, or community), or someone who is close to them (for instance, the involvement of parents on behalf of children). Furthermore, they both refer to collaboration between professionalized service providers in public agencies and citizens, whether individually or collectively. It does not include organizations collaborating with one another. Finally, both terms refer to active input by citizens in shaping services. This is different from passive clientelism: it is not enough simply to receive or consume a product. Osborne has argued, following the service literature, that co-production and co-creation are inevitable in the context of services (Osborne and Strokosch Citation2013). Yet it has also been argued that, in addition to such inherent participation, services can be designed to allow different degrees of active input (Porter Citation2012).

The two terms can be distinguished on the basis of the kinds of inputs citizens contribute (Brandsen and Honingh Citation2018). Co-production is generally associated with services citizens receive during the implementation phase of the production cycle, whereas co-creation concerns services at a strategic level. In other words, when citizens are involved in the general planning of a service – perhaps even initiating it – then this is co-creation, whereas if they shape the service during later phases of the cycle it is co-production. Input in the design of a service can be both individual or collective, depending on the level at which a service is addressed.

Let us illustrate these choices using the example of a digital platform:

  • If citizens actively engage in the design and delivery of their personal services through a digital platform provided by the service organisation, it is co-production. If they only passively receive services through the platform, it is not.

  • If citizens initiate the construction of the digital platform, or deliberate in a representative body discussing its maintenance and design, it is co-creation.

  • If organisations, rather than (groups of) citizens negotiate with one another over the design and maintenance of the platform, it is neither co-production nor co-creation. This has elsewhere been referred to as ‘co-management’ (Brandsen and Pestoff Citation2006).

  • If hackers invade the digital platform and wreck it, they are helping to shape the service experience of users; but they are not co-producing or co-creating.

Now that we have described the processes on which new technologies could make an impact, the question is how they would do so. For this we need to deconstruct the interaction between organisations and co-producers/-creators.

Deconstructing the interactions

There is a growing body of scholarship that de-constructs the co-production/co-creation process. Bovaird and Loeffler (Citation2013) have distinguished co-commissioning, co-design, co-delivery and co-assessment, a conceptual model that was subsequently picked up and extended by Nabatchi, Sancino, and Sicilia (Citation2017). These distinctions relate to the service or policy cycle of which co-production is part. In this article, we will focus on the co-design and co-delivery phases in these models, within which we will make a further distinction between different elements of the interaction that takes place in designing and/or delivery a service. In this, we will draw on other works that have highlighted specific aspects of the interaction.

We identify four elements of the co-production/co-creation process on which new technologies could have an impact.

1. Establishing direct interaction: To start, very simply, it is necessary for citizens and (employees within) organisations to be in contact. This does not necessarily have to be face-to-face contact (indeed, the added value of new technologies is often exactly in removing the need for it), but there should be the opportunity for direct interaction. One of the difficulties at the time when Elinor Ostrom did her research in the 1970s was to know who the potential co-producers were and how to get in touch in them. Even to this day, a major problem in service delivery is that the different parties are simply not aware of each other.

2. Motivating: The people involved must be willing to play an active role in the co-production process. This applies on both sides. Citizens must be interested, for whatever reason, to engage in such a process. But employees must also be willing to take the effort to engage with citizens, even when it may limit their own control over the outcomes. This has been one of the key issues in past co-production research. It has examined what are the key factors that make professionals and citizens engage actively in co-production or co-creation, or not (Parrado et al. Citation2013; Van Eijk and Steen Citation2016; Van Eijk Citation2017; Steen and Tuurnas Citation2018). On both sides, a sense of self-efficacy and autonomy appear important for the willingness to engage.

3. Bringing resources to the service: Both citizens and employees bring individual resources into the process. Both will at least offer time and their particular expertise. Teachers know best how to educate children generally, parents know their own children best (Honingh, Bondarouk, and Brandsen Citation2018). This is the element that past research has demonstrated most successfully, for example, how different parties contribute to the regeneration of their neighbourhoods (Vanleene, Voets, and Verschuere Citation2017) or how patients bring expertise to their treatment (Batalden, Batalden, and Margolis et al. Citation2016; McMullin and Needham Citation2018).

4. Sharing decision-making: Co-production and co-creation presumably shift decision-making power from employees to citizens. However, organisational control over the process may still be significant (if only, by limiting the range of potential decisions). Studies have shown that the role of citizens can range from providers of services, with full responsibilities, to active co-producers, to consumers, to passive beneficiaries (Pestoff Citation2018).

Using these four elements, we will in the next paragraph narrow down the effects of digital technologies.

Relevant types of digital technologies and impacts on co-production/-creation

In order to understand the profound impact of digital technologies might have on the nature and evolution of co-production, it is necessary to analytically unpack the technological processes underlying current developments. This is tricky. As the development cycles of ICT have become very rapid, any attempt to define some sort of definitive and detailed categorization of technological processes is probably doomed to fail. To stay close to the essence, we have chosen to follow a recent taxonomy offered by Aceto, Persico, and Pescapé (Citation2018), who delineate between four main instrumental characteristics of modern digital technologies: sensing, communication, processing and actuation. This comes with two caveats. First, as the authors themselves have also stressed, digital technologies usually have many parallel functionalities. Second, although this taxonomy was originally used in a study focusing on health care, we believe that this taxonomy is informative also for any other field of public service delivery. We use this approach to illustrate the characteristics of digital technologies, that is, what kind of technological opportunities come with the current ICT paradigm. Only in the later sections will we analyse how these instrumental characteristics are related to the specific elements of public service co-production.

Sensing technologies such as wearable and smart devices provide new opportunities to collect information about almost every aspect of social life. These and other sensing technologies are essential building blocks what has become known as datafication of every-day life. This has received some attention from scholars studying participation as well as co-production, as various technologies associated with the ‘smart city’, such as electronic sensors are becoming an everyday reality (Townsend Citation2013; Cardullo and Kitchin Citation2018). For example, AI-based programs to map and measure litter using visual recognition technology and accompanying mobile apps to enforce anti-littering have been developed in Japan (OECD Citation2018), directly shaping how citizens co-produce environmental protection. In the field of medication these technological solutions are widespread: various apps are already available that track medication intake and provide intake and prescription alarms (Silva et al. Citation2015); or nudge lifestyle choices, where communication between the citizen and public sector (through an incentive or nudge) is automated and tracked. Furthermore, assisted living technologies such as telecare (remote monitoring of emergencies through sensors and personal alarms) and telehealth (transmission of medical information over telecommunication) provide opportunities for elderly people to live independently at homes, while assuming a significant shift in co-production practices (Wherton et al. Citation2015). Thus, context-aware networks and pervasive monitoring applications are emerging in which the choice to actively contribute input may not be up to citizens any more or the incentives to co-produce due to pervasive monitoring change considerably.

Communication technologies from machine-to-machine communication and wireless networks to social media create new opportunities to ubiquitously interact, not only for people, but also for machines. The majority of existing studies exploring digital co-production have so far focused on social media (Meijer Citation2011; Linders Citation2012; Nam Citation2012; Mergel Citation2016; Noveck Citation2015; Paletti Citation2016), observing the relationship between citizen-state rather than citizen-machine-state. For example, many cities and municipalities have designed new online engagement tools to get feedback from citizens in an online format (this can be simply engagement, but can also involve co-production elements). There are many other emerging technologies in this grouping that have received less attention, but are becoming central to the ways citizens engage with public-service delivery – for example, blockchain, which enables peer-to-peer service delivery (Pazaitis, De Filippi, and Kostakis Citation2017; OECD Citation2018). Furthermore, the spread of Wifi networks and smart personal devices have penetrated society assuring constant connectivity, portability and computing power. This allows also interactions between the state and the citizen – and thus co-production – to overcome geographical, temporal and organisational barriers.

Processing technologies such as cloud computing, big data analytics and machine learning are the cornerstones in making sense of the vast amount of data available. Big Data, cloud capabilities, ubiquitous computing (computing is made to appear anytime and everywhere) and ambient intelligence (electronic environments that are sensitive and responsive to the presence of people) move data analysis and following interventions from descriptive to predictive to prescriptive (Chang and Choi Citation2016). These processing capabilities allow collecting data from citizens in real time and performing non-intrusive monitoring, predictions and consequently, improvements of services. In conjunction, data mining makes it possible to analyse large observational datasets to discover unsuspected relationships, possibly uncovering new avenues of co-production or making the existing ones more precise/targeted. At the same time, processing can also shift co-production towards more passive participation on the citizen side and eliminate the need for direct interaction and thus, limiting the practice of co-production itself.

Actuation technologies based on robotics, 3-D printing and other technologies capable of mechatronic actions epitomize the ability of ICT to act independently from humans. Robot-human interactions have been around for some time (Young et al. Citation2011). Governments are starting to use 3D printing to simulate and test different eco-system elements. In some cases, technology can influence co-productions patterns indirectly (e.g. in services produced based on co-created simulations). All these technological trends potentially may not only augment the existing co-production and co-creation practices, but may also change the roles and relationships between service users and professionals.

In the next sections, we bring together the core conceptual elements from previous sections on co-production (direct interaction, motivation, resources, and shared decision making) and digital technologies (sensing, communication, processing, and actuation) and show how the latter can affect co-production/-creation processes. We use examples to illustrate the emerging trends and eventually summarize the main impacts in a table.

Interaction

Interaction between citizens and professionals is a key precondition for any co-production or co-creation relationship to emerge and evolve.

Sensing technologies provide access to accurate real-time data and initiate interaction. Remote assistive health technologies that monitor heart conditions can signal the need for a patient or her doctor to intervene, thus potentially triggering a cascade of follow-up direct interactions. In other cases, sensing technologies initiate interactions that may give rise to new kinds of relationships. Mobile positioning data have been used to analyse tourism patterns and also crowd control during mass events (Shoval and Ahas Citation2016; Versichele et al. Citation2012). In the eyes of planners, these applications diminish the need for direct interaction with citizens, as they can collect masses of relevant data without it. Only where the set-up and implementation of sensor networks requires direct interaction and active contributions from the citizens and professionals does sensing technology increase direct interaction.

Communication technologies such as social media enable swifter and broader interaction between public organisations and potential co-producers/-creators, since they allow for more efficient information flows, through real-time access to and exchange of information. Importantly, digital communication technologies enable a radical increase in the scale and collectiveness of interaction. Even when interactions existed prior to the introduction of these technologies, the more user-friendly form that digital technologies allow can increase reporting from citizens substantially. However, the increasing reliance on digital communication may also mean that co-production increasingly happens only where interactions can be established digitally, to the detriment of traditional, physical interaction. An example from the history of policing is telling: while the adoption of patrol cars with radio communication technologies enabled police forces to become more effective in many different ways, this technological innovation also allowed police officers to become more reactive and detach themselves from local communities (Forst Citation2000).

Processing technologies, such as cloud computing, do not in themselves directly affect interaction in any particular direction. However, when used in combination with communication and actuation technologies, they enable decision-makers to involve targeted groups of citizens, heightening interaction. By contrast, they can also be used to crowd out deliberation and debate, or to neglect input by and, thus, potential interaction with citizens (e.g. machine-learning algorithms sorting out information for police received through official social media accounts). Thus, processing technologies can influence who, when and how gets to interact with each other.

Actuation technologies can lead to significant changes in agency. These actuation-driven developments can be summarized as a shift occurring from human-to-human interactions to human-to-machines (for instance, when people start interacting with robot assistants). This may make it possible for people in the future to co-produce with machines rather than other human beings, possibly at the cost of human interaction (for good or for bad). Actuation technologies may also allow more machine-to-machine interactions, cutting out human interaction altogether.

On the whole, digital technologies can affect interaction in a myriad of ways. It can be argued that they help to join up with people, allowing citizens get in touch with organisations more easily, or vice versa. But they may also reduce the need for direct interaction and, by implication, for co-production and co-creation; or they may empower citizens to self-organise, bypassing existing organisations.

Motivation

Motivation to be engaged in co-production can be directly shaped by the ways digital technologies are employed in the co-production process.

Sensing technologies, in combination with communication technologies, can allow for the personalisation of data through smart devices, increasing the motivation to contribute. At the same time, the increasing usage of sensing technologies can also discourage citizens from becoming involved, as fear of excessive surveillance of individual behaviour increases and as citizens become overloaded with information (e.g., in the field of personalised social services, see Tõnurist and De Tavernier Citation2017). In other words, the sense of autonomy and self-efficacy can be actually reduced.

Communication technologies potentially increase the motivation to co-produce/-create, as they lower the threshold to engage (Clark, Brudney, and Jang Citation2013). Also, the more information and knowledge becomes available through better communication and access, the more light is shed on the black box of core service delivery, and thus they may increase the motivation to participate. Finally, gamification strategies (using game-thinking or game mechanisms in non-game contexts) offer new incentives for citizens to contribute to the creation of data or other crowd-sourced resources (Mergel Citation2016).

Yet there is also the possibility that, because of gamified or nudged co-production, extrinsic motives crowd out intrinsic ones (Tõnurist and Surva Citation2017). There is a danger that co-production becomes less about quality and more about the game, which can ultimately make it easier for citizens to quit co-producing and – creating, as there is little personal loyalty (see Townsend Citation2013 for many examples). Furthermore, as with sensing technology, ubiquitous and constant communication can decrease the motivation to engage when citizens feel that their privacy is under threat.

Processing capacity, like sensing technologies, can enhance individual motivation by creating a more personalised evidence basis for co-production at the individual level. This, in combination with communication technologies, enables real-time incentives for citizens to get involved, for example, in attempts to crowdsource solutions for public sector challenges (Noveck Citation2015). But such technologies also help create personal nudges that elicit specific actions, which is not necessarily a motivation to co-produce. Coupled with Internet of Things (IoT) and Big Data analytics, nudges could be based on data regarding a person’s health, location, past preferences or any other characteristic. In essence, whenever co-production depends on digital platforms (Brown et al. Citation2017), the processing technologies enable the platform owners to influence citizens’ and professionals’ motivation to co-produce, in either direction.

Actuation can increase motivation by creating low-cost opportunities to establish or maintain co-production, for instance, through socially assistive robotics (SAR) and health tracking devices. In areas such as elderly care, behavioural therapy, mental health care, dementia care, rehabilitation, and education, people increasingly interact with robots, who encourage them to do their own work. For example, robots (‘Chili’) have been used in experiments to teach and motivate first-graders to make healthy food choices (Short et al. Citation2017). This is essentially a coproduction process. However, the effect of this technology can also be to lower engagement, as human interaction becomes less frequent and understanding of the service process less necessary, as sensors and robots take over agency and sense-making. An interesting question is whether robot-to-human interaction should be classified as co-production or not, since an implicit assumption so far has been that it concerns human-to-human interaction. Here, we will assume that it can be, if the people involved perceive it as such.

Summing up, digital technologies have the potential to create a stronger motivation for co-production and co-creation, especially by allowing more personalised solutions and by heightening the sense of entertainment. Then again, they may also make the service process less attractive for citizens to engage in, for instance, by making it less personal or less innovative.

Shared resources

Bringing resources to the service, on the part of both citizens and professionals, is another key condition for co-production and co-creation to take place.

Sensing technologies can considerably increase the scope and scale of data both governments and citizens can reach, thus potentially increasing the quality of co-production. For example, data generated by the IoT can be used to understand people’s needs and social behaviour with much more accuracy than was previously possible (e.g., in mobility, see Poslad et al. Citation2015).

Communication technologies can mobilize inputs from citizens on a far larger scale and have opened up resource-sharing in new areas. Through various digital crowdsourcing platforms, governments have been able tap into the collective wisdom of the crowds by systematically collect ideas, opinions, solutions and data from service users and citizens (Noveck Citation2015; Symons and Bass Citation2017), for example, data collection through ‘fix-my-street’ and 311-type solutions (Clark, Brudney, and Jang Citation2013). People can also be mobilized to create data in specialised interest areas. Public organizations at times raise money directly from citizens for public investments in school equipment or public walkways (The Economist Citation2013). This could be regarded as a form of hidden privatization under the label of co-production or co-creation.

Processing capabilities allow us to make sense of vast amounts of citizen- and government-generated data. The use of large-scale processing capabilities relies on the availability of datasets, as well as the right kind of input from citizens and governments. Examples include not only social media harvesting (e.g. using Twitter for sentiment analysis for getting real-time feedback), but also developing personalised solutions. This can support co-production and co-creation. For example, many European cities are creating voluntary data repositories, where citizens can donate their personal data, which are then used to co-create and co-produce new services and where the data are processed, maintained and controlled through blockchain technologies (Symons and Bass Citation2017). Likewise, opening up and processing government data can create new resources for citizens to evaluate existing services or design new services. Yet such resources can also be used to govern on behalf of citizens, without actually interacting with them.

Actuation technologies can facilitate co-production by lowering the threshold for resource contribution. When certain stages of service delivery are automated, service users may find it easier and less time-consuming to engage in co-production. However, actuation technologies can also force citizens to bring in complementary resources, for instance, when particular skills are needed to co-produce with robots or remote trackers. If technical skills become necessary to take part in co-production, this may act as a barrier, especially as this knowledge is often tacit and difficult to transfer. This may strengthen existing inequalities based on digital divide, age and other factors. For example, various geriatric rehabilitation robots have been developed that not all intended users are capable of interacting with (Sale Citation2018; Wherton et al. Citation2015).

Overall, the effects of digital technologies on resource sharing are unequivocal: they create important new resources, as well as ways to share resources, that could be used for co-production and co-creation. Whether they are actually used for that purpose is another matter.

Decision-making

In decision-making, sensing technologies can encourage an increased ‘jointness’ of service delivery. The ability to collect relevant data on, for example, pollution or the use of public spaces can provide citizens with a strong basis for co-creating new solutions with policymakers, giving them a seat at the table. As such, sensing technologies can empower citizens and give them the opportunity to become part of the decision-making process. However, more often than not, sensing technologies tend to strengthen the power of data (infrastructure) owners like private firms and public sector organizations. Data collected from or by citizens only adds public value in certain contexts. For example, data on people’s transportation and parking choices does so if joined up with mobile pollution data and if used to co-create or co-produce behavioural changes. Yet data owners are also able to build new solutions and test service quality by joining up new data sources – all of which is not usually possible for citizens by themselves. Thus, there is the risk of ‘digital capture’.

Communication technologies can give governments more control by allowing them to manipulate (centralized) communication processes and develop personalised nudges to influence decision-making on a large scale (though perhaps hidden from citizens). By contrast, citizens can use technological platforms to self-organise and to bypass public sector organisations and professionals altogether. For example, there are a number of apps that allow citizens to self-organise for disaster relief without the involvement of the state. In both cases, there is less co-production and less co-creation.

However, the wide availability of means of communication can also enhance joint decision-making and citizen empowerment, as there are more ways to organise an open decision-making process. This applies where governments face complexities in providing public services in remote areas or crises (Joshi and Moore Citation2004). Also, greater transparency leads to more effective citizen control over data underpinning co-production. Here there are numerous examples from the field of urban planning (Falco and Kleinhans Citation2018).

Processing capabilities may support shared decision-making through the resources that they bring to the process. By enabling greater interaction and motivation, they can encourage co-production (see the previous sections). However, they may also effectively allow decision-makers to cut citizens out of the process, taking decisions on the basis of aggregated data only (see actuation in this section).

Actuation through programmed solutions (prior artificial intelligence) may limit citizens’ choices in machine-human interactions. An emerging trend is the use of algorithm-based decision-making models (largely depending on processing capabilities) and the Internet of Things to exercise control, for instance, over the behaviour of crowds or public service performance. The mere presence of citizens in public spaces provides the governments potentially valuable feedback (Cardullo and Kitchin Citation2018) and makes it possible to build predictive governance models based on the actual behaviour of citizens, without actively engaging them (Athey Citation2017). Auditing machine learning algorithms, which often form the core of digital co-production, has become one of the most challenging governance issues of today. Since information is aggregated and run through complex formulae, it can be close to impossible for individuals to demand accountability from data owners about decisions shaped through algorithms, without user involvement.

As machine learning becomes more sophisticated, machines can start applying adaptive decision-making, changing behaviour and actions based on the choices that people make. Thus, this may, in principle, also decentralize decision-making power. But even in such active varieties, accountability can be an issue of contention (see our discussion of youth development).

Summing up, the effects of digital technologies on decision-making can be extremely varied. If they structure the process in ways that shift decisions to clients, or open up the process to outsiders, they can strengthen co-production and co-creation significantly. This is the future that tech optimists tend to trumpet. However, in the design of IT systems, the scope of decisions can be potentially limited to such a degree that effectively decision-making powers are reduced, both for employees and clients (Bovens and Zouridis Citation2002; Cardullo and Kitchin Citation2018). Early e-government research uncovered how ICT reinforced centralist tendencies within organisations (Snellen and van de Donk Citation1998). Now new technologies may allow decisions to be effectively taken outside of these organisations, by those who design and control the standardized (automated) decision-making processes (e.g., Lember, Tõnurist and Surva Citation2017). Both governments and citizens have quite some flexibility in the use of technology for decision-making. The choice over which communication (or any other) technologies to employ makes digitalization of co-production an inherently political issue as this choice determines also who has the control over co-production processes.

summarizes the above described relationship between the main characteristics of digital technologies, co-production and co-creation.

Table 1. Potential positive and negative impacts of digital technologies on co-production/co-creation.

Illustrations of potential effects

We will now illustrate the potential effects of digital technologies with three classic examples of public services: policing, youth development and elderly care. In each instance, we will show how digital technologies can enhance or diminish co-production or co-creation.

Policing

Policing has been central to co-production research ever since Elinor Ostrom and colleagues initiated their famous work, showing that police collaborating with citizens were far more effective than traditionally supposed. Technologies had already changed policing prior to digitalization, for instance, with the invention of radio communication technologies and cars, human contact became less frequent as police could receive messages directly and fewer walked the streets. Digital technologies are likely to give a new twist to this development.

One strand of literature has focused on how technologies have been used to engage citizens more directly, for instance, by using social media to involve citizens in detecting crimes (Meijer Citation2012). Such cases make it seem as if digital technologies will inevitably create a greater role for citizens. Twitter and other similar applications function as sensing and communication technologies for the police, which can be complemented with processing technologies that make sense of the massive data collected through social media. Clearly, these technologies enable citizens and police professionals to establish new kinds of relationships with each other. It should be noted that research has shown that it is not a broad base of citizens who are reached through such channels: it is more effective to engage with the targeted and interested few (Grimmelikhuijsen and Meijer Citation2015).

But other police practices point in a different direction, towards diminished engagement. A controversial example is predictive policing, where algorithms predict, based on citizens’ past behaviour, where the next crime will take place and correspondingly trigger preventive actions by police (Hunt, Saunders, and Hollywood Citation2014). Many cities claim that this has clearly reduced the rates of criminal offences and thus fostered change in citizens’ behaviour, though critical voices point out that the jury is still very much out (Bennett Moses and Chan Citation2018). In any case, if such practices continue to spread, it makes active co-production and co-creation less relevant. One of the potential longer term effect of this emerging practice might be that there is less motivation for citizens to directly contribute their resources to public safety through street watch groups and similar initiatives. As predictive algorithms are based on historically accumulated data, they do not rely on direct relationships. This diminishes citizens’ ability to shape the decision-making when it comes to providing public safety as algorithmic decision-making – powered by processing technologies – becomes black-boxed (O’Neil Citation2016). The use of predictive software makes it problematic for police professionals to justify their decisions (Bennett Moses and Chan Citation2018), potentially making it more difficult to form meaningful relationships with local residents.

In other words, while these may all be classed as the effects of new technologies, they are quite different in nature. In fact, the same technology may have different effects depending on context. The same communication technologies celebrated before, combined with processing and decision-making technologies, may actually diminish co-production. For instance, there are cases of police developing bots to make sense and sort the information collected from Twitter, which consequently lead to selective use of citizens input and less response, which in turn may affect motivation. Citizens as reporters of offences remain a central building block for gathering the data that feed the predictive algorithms, but their role becomes less active than when police officers and local residents actively form new relationships on the streets.

Youth development

Changing the behaviour of citizens to better fit society’s norms and expectations is one of the most complex areas where co-production is involved. As shown by Cottam (Citation2018), digital technologies can profoundly affect how behavioural change is envisioned and implemented, while fundamentally relying on co-production between citizens and professionals in social services. She describes experiments with youth and family development programs, which abandoned bespoken services, started from observing participants’ own aspirations and every-day contexts, and aimed at building participants’ capabilities through forming new relationships. Next to radically re-thinking the theoretical framework guiding how the behavioural change is to be approached, it was the use of a mix of digital technologies, from mobile phones and customer relationship management applications and data analytics tools, that enabled this new approach to be realised. It made it possible for participants to reach out to each other and programme staff, enabled programme staff to keep track of thousands of participants and their needs and progress, and connected participants with others who could provide opportunities for new experiences and relationships.

In this case, digital technologies performed the sensing, communicating and processing functions, and enabled quick feedback loops and low-cost solutions to connect people and judge their progress. Importantly, the technologies were not applied to replace human interactions, but to leverage the possibilities to form direct physical relationships, increase citizens’ and professionals’ motivation to co-produce, build and share each other’s resources, and empower citizens.

The driving force behind the use of digital technologies in these initiatives was the need to re-think the design and delivery of welfare services. It was an attempt to move beyond the mass-production paradigm that currently guide the organization of welfare services towards a genuinely co-productive one. Yet, ironically, the same functionalities of digital technologies at work in those experiments can also extend rather than end the mass-production paradigm of the welfare services. As argued in Cottam (Citation2018) and in many other accounts (e.g., OECD Citation2017), most of the youth professionals spent the majority of their time monitoring and reporting, rather than actively engaging with young people. Due to the ever-greater use of ICT, the street-level bureaucrats who engage with young people are often transformed into what has become known as ‘screen-level bureaucrats’ and eventually ‘system-level bureaucrats’, whose functions and organizational relationships are fully determined by interconnected ICT systems (Bovens and Zouridis Citation2002).

It is also increasingly common for welfare departments to experiment with self-trained models that predict which young people are likely to become trapped in problems and at what point, which can support welfare professionals in designing targeted interventions (Berk Citation2019). If the trade-offs between fairness and accuracy that arise from using these predictive technologies are handled well, this development could give a massive impetus to more personalized approaches to youth development services. But the potential limitation of these developments is that if the existing services for interventions do not rely much on co-production, then digital technologies could instead reinforce the existing service logic. The use of predictive risk assessment algorithms, and the interventions they trigger, can make citizens reluctant to co-produce, if they feel that the professionals intervene against their own will or ethical standards.

Elderly care

Finally, elderly care is an area where digital technologies could have a significant impact on co-production and co-creation. As with the police, past technological changes have already made their mark. From the 1950s onwards, with the development of the welfare state, governments increasingly took over elderly care functions from families, with the establishment of publicly funded homes for the elderly. This was assisted and to some extent made possible by advances in technology, such as improvements in medicine, and related to the rise of a new class of professionals (Litwak and Figueira Citation1968). At present there is a shift in the opposite direction, as telecare, robot assistants, real-time remote monitoring and other technology-assisted services increasingly allow elderly people to stay at home (see e.g. Wherton et al. Citation2015).

Again, there are different ways of using such technologies, which in turn shape opportunities for co-production and co-creation. For instance, technological advances already allow people to measure their own health (e.g. glucose levels). Digitalisation can extend this trend, by allowing these data to be directly transmitted to doctors and allowing remote ‘house calls’, during which citizens and doctors can discuss future treatment, much more easily than if they relied on physical encounters. Alternatively, technological nudges can be used to influence the motivation to co-produce, for instance, by using electronic dietary diaries which assess the impact of patients’ eating behaviour and provide early warnings.

However, digital technologies can also be used to diminish the involvement of clients. When sensoring technologies simply pass on data to a medical facility in another location, it diminishes the need for interaction between patients and doctors. Indeed, doctors can be cut out of the loop altogether, as computer prescribe medicine on the basis of incoming data and have them delivered to patients’ homes automatically. The functions of carers can be taken over by chat-bots and robots that combine the sensing, communicating, processing and actuation technologies, but with a potential effect of lowering the motivation of relatives to keep contributing their time and efforts (Dickinson et al. Citation2018). For example, computers are already more accurate in diagnosing some cognitive impairments (including Alzheimer’s or Parkinson’s disease) than human doctors (e.g., Fraser, Meltzer, and Rudzicz Citation2016). They can do this well in advance, before overt symptoms appear, but they need to be searching for those symptoms on the basis of personal data, which can be very intrusive. Then again, many people are afraid of doctors and being diagnosed at home can also be perceived as less intrusive. At the same time, it has been shown that co-production with medical professionals over longer periods of time brings down the stress levels associated with doctors and hospitals (Cobos, Haskard-Zolnierek, and Howard Citation2015). Relying heavily on home care may mean that the stress levels of acute incidents may put people at increased risk. Thus, the balance is somewhere in-between harnessing the new diagnostic capabilities, while not in the process alienating patients from health care workers. So far, assisted living technologies for the elderly have more often than not failed to trigger new co-production practices, nor have they had significant effects on care efficacy or cost reduction (Wherton et al. Citation2015).

Discussion: three scenarios and a fourth paradigm

There is still little systematic evidence on how digital technologies affect co-production and co-creation in practice. In this article, we have aimed to arrive at a more sophisticated understanding of the potential impacts of digital technologies on co-production and co-creation. Clearly, the impact of technologies is not straightforward. We have emphasized that the use of digital technologies does not inevitably lead to co-production and co-creation. There are cases in which service professionals and users do have choices how to design specific digital solutions for co-production. However, in others they do not have this opportunity or lack the capabilities to sense and seize it. Stakeholders may be forced to use new technologies that are perceived as innovative, but of which the potential effects may not be clear to them. Often choices about new technologies are made by third parties, rather than those who would co-produce. The centralized ICT department of a city may impose strict rules on which technologies can be used in public service provision, and how. Alternatively, a city may be held hostage by its private sector technology provider that uses legacy solutions (Dunleavy, Margetts, and Bastow et al. Citation2006) that severely limit the design options for co-production or co-creation.

One could define three potential scenarios: digital technologies can augment, diversify or substitute for co-production (see also Lember Citation2018).

A first possibility is that digital technologies enable co-production and co-creation in every possible sense, by overcoming existing geographical, temporal and organisational barriers. Provided that they enable interaction between service professionals and users, increase motivation for both sides to engage, facilitate mutual resource-contributions, and/or share perceived decision-making authority, we can expect a positive synergy between digital technologies and co-production/co-creation. On a systemic level, this could lead to the transformation of the role of government in society and the role of citizens in public governance.

A second option is that digital technologies will diversify co-production practices. As emerging opportunities are seized through the complex interaction between the characteristics of specific technologies, demand for services, organizational capabilities and individual skills (Nelson and Winter Citation1977), we can expect very diverse digital co-production practices to emerge with both positive and negative consequences. On the one hand, there are areas of co-production that adapt easily to technological change (e.g. where digitalization of interactions is easier), while the opposite is the case elsewhere (e.g. where the direct human touch is of a significant value in its own right). Consequently, we will probably see different rates of technology-related innovations in different areas. At a more fundamental level, this diversity of practices will likely differ from the existing ones, depending on how technologies affect the processes of interaction, motivation, resource sharing and decision-making.

We should also be aware of a third possibility, which is that digital technologies will substitute for co-production. This can happen in various ways. One is that it opens the way to more self-organisation by citizens. Technologies may give full control of service provision to citizens, without the need for direct or even indirect government involvement. Citizens will choose the design and implementation methods, co-create the technologies and coordinate the activities from start to finish (Niaros, Kostakis, and Drechsler Citation2017). It allows citizens to deliver peer-to-peer initiatives on a much larger scale than was possible previously. Examples are Wikipedia and community-owned public taxi services around the world. It may well result in entire eco-systems of user-driven innovators (Von Hippel Citation2016), with technology as the ‘silent partner’.

However, there are also developments in the opposite direction. The process of collecting input from citizens may be fully or partly automated, creating a machine-to-machine paradigm where everyday objects and the surrounding environment are connected and managed through a range of devices, communication networks and (cloud-based) servers (Aceto, Persico, and Pescapé Citation2018). This changes the role of citizens to a more passive one, rather than the other way round.

In past years, public management research has emphasized the shift to a more collaborative paradigm of governance, for instance, Osborne’s New Public Governance, following upon Public Administration and New Public Management (Osborne Citation2010). Digital technologies have the potential to strengthen the participatory element in this latest paradigm. However, in parallel, we may be witnessing the emergence of a ‘fourth paradigm’, one in which decision-making in public services interacts with citizens only indirectly, through nudging, or bypasses them altogether, basing decisions on complex algorithms and collective data. This is fundamentally different from the other paradigms and at another point deserves further elaboration. These radically different visions of governance may well continue to exist alongside one another, in complex and conflicting relationships.

Future research

Accordingly, there is plenty to do for future research. Currently, it is hard to predict whether the increased information exchange that follows from digitalisation will crowd out or encourage co-production and co-creation. This depends on the institutional context, organizational capabilities and personal preferences and skills. For policymakers and citizens, it is important to better understand the potential effects and the underlying variables, as it enables better the design and implementation of co-production. Prior research has been dominated by single case studies, as has been the case in co-production research more generally. The least that is necessary is to develop a more explicitly comparative approach to the development of case studies, as opposed to an emphasis on best practices.

What complicates research in this area is the fact that technologies are increasingly packaged together. In practice, different types of technological solutions (communication, sensing, processing and actuation) often overlap. IoT and robotics need to harness all core ICT pillars to be functional. Disentangling this can be approached in various ways, through experiments and vignette studies, to isolate the effects of different variables, or through theoretically more advanced, more detailed case studies that allow a thorough examination of the interactions between different functionalities and technologies.

When examining impacts, it will be important to differentiate between different social groups. The question is whether opportunities for co-production and co-creation will exist for all, or only a select few. Traditional participation has been vulnerable to capture by groups who were arguably already in control, the highly educated who are easily able to navigate such processes. Recent co-production research has put this issue more prominently on the map, to ascertain whether co-production replicates this bias or diminishes it (Verschuere et. al. Citation2018). Digital technologies may require yet new skills – then again, they may lessen the need for the ones that were traditionally required. Surveys, the analysis of secondary data and ethnographic research in communities can help show these variations.

It is equally important for the further research to understand and uncover the underlining logics that guide the development and uptake of digital technologies and thus shape the impact of these technologies on co-production. As Shoshana Zuboff has put it: ‘The digital can take many forms, depending upon the social and economic logics that bring them to life’ (Zuboff Citation2019). Applying digital technologies to augment the mass-production paradigm of the welfare state will probably bring very different impacts when this is done to foster citizen self-organization or public sector markets through platformization.

Finally, an important question is who controls the shape of digital technologies in public service delivery and, by implication, the opportunities for co-production and co-creation. For example, the platformization of economy and governments (Linders Citation2012; Brown et al. Citation2017; Teece Citation2018) potentially provides a central role for user co-production and co-creation. Using web-based interfaces, open-government and user-generated data enable citizens and other interested parties to design and implement services based on data owned and stored by the government (Kornberger et al. Citation2017; Toots et al. Citation2017). Yet at present most of the technologies underlying these platforms – if not these platforms themselves – are developed, owned and controlled by private companies who amass data – i.e., the source of control and authority – and thus potentially pave the way towards a hidden privatization of (future) services and a re-allocation of control away from citizens. Measuring this reliably may require a process of backward mapping, showing when crucial decisions on the architecture and implementation of technologies were made, underpinned by multidisciplinary expertise and underpinned by sophisticated methodological tools. But, hard as it may be, the effort is crucially important, unless we want to keep the structures of power in a digitalised society hidden.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no. 750378.

Notes on contributors

Veiko Lember

Veiko Lember, PhD, is Research Fellow at Public Governance Institute, KU Leuven, Belgium and Senior Research Fellow at Ragnar Nurkse School of Innovation and Governance, Tallinn University of Technology, Estonia. His main research interests include digital government, public sector innovation, and innovation policy ([email protected].)

Taco Brandsen

Taco Brandsen, PhD, is Professor of Public Administration, Radboud University, the Netherlands. His research focuses on public services, innovation and co-production ([email protected].)

Piret Tõnurist

Piret Tõnurist, PhD, is a policy analyst for the OECD and a Research Fellow at Ragnar Nurkse Department of Innovation and Governance. Her research topics include innovation policy, energy technologies and social innovation. She has worked for the State Audit Office of Estonia as a performance auditor in the field of innovation and entrepreneurial policy ([email protected].)

References

  • Aceto, G., V. Persico, and A. Pescapé. 2018. “The Role of Information and Communication Technologies in Healthcare: Taxonomies, Perspectives, and Challenges.” Journal of Network and Computer Applications 107: 125–154. doi:10.1016/j.jnca.2018.02.008.
  • Alford, J. 2002. “Why Do Public Sector Clients Co-Produce? Towards a Contingency Theory.” Administration & Society 34 (1): 32–56.
  • Athey, S. 2017. “Beyond Prediction: Using Big Data for Policy Problems.” Science 355 (6324): 483–485.
  • Batalden, M., P. Batalden, P. Margolis, M. Seid, G. Armstrong, L. Opipari-Arrigan, and H. Hartung 2016. “Coproduction of Healthcare Service.” BMJ Quality and Safety 25 (7): 509–517.
  • Bennett Moses, L., and J. Chan. 2018. “Algorithmic Prediction in Policing: Assumptions, Evaluation, and Accountability.” Policing and Society 28 (7): 806–822. doi:10.1080/10439463.2016.1253695.
  • Berk, R. A. 2019. “Accuracy and Fairness for Juvenile Justice Risk Assessments.” Journal of Empirical Legal Studies 16: 175–194. doi:10.1111/jels.2019.16.issue-1.
  • Bovaird, T. 2007. “Beyond Engagement & Participation: User & Community Co-Production of Public Services.” Public Administration Review 846–860. doi:10.1111/j.1540-6210.2007.00773.x.
  • Bovaird, T., and E. Loeffler. 2013. “The Role of Co-Production for Better Health and Wellbeing: Why We Need Change.” In Co-Production of Health and Wellbeing in Scotland, edited by E. Loeffler, G. Power, T. Bovaird, and F. Hine-Hughes, 20–27. Birmingham: Governance International.
  • Bovens, M., and S. Zouridis. 2002. “From Street‐Level to System‐Level Bureaucracies: How Information and Communication Technology Is Transforming Administrative Discretion and Constitutional Control.” Public Administration Review 62 (2): pp.174–184. doi:10.1111/puar.2002.62.issue-2.
  • Brandsen, T., and M. Honingh. 2016. “Distinguishing Different Types of Co-Production: A Conceptual Analysis Based on the Classical Definitions.” Public Administration Review 76 (3): pp. 427–435. doi:10.1111/puar.12465.
  • Brandsen, T., and M. Honingh. 2018. “Definitions of Co-Production and Co-Creation.” In Co-Production and Co-Creation: Engaging Citizens in Public Services, edited by T. Brandsen, T. Steen, and B. Verschuere, 2018. London: Routledge.
  • Brandsen, T., and V. Pestoff. 2006. “Co-Production, the Third Sector and the Delivery of Public Services: An Introduction.” Public Management Review 8: 493–501. doi:10.1080/14719030601022874.
  • Brown, A., J. Fishenden, M. Thompson, and W. Venters. 2017. “Appraising the Impact and Role of Platform Models and Government as a Platform (Gaap) in UK Government Public Service Reform: Towards a Platform Assessment Framework (PAF).” Government Information Quarterly 34 (2): 167–182. doi:10.1016/j.giq.2017.03.003.
  • Cardullo, P., and R. Kitchin. 2018. “Being a ‘Citizen’ in the Smart City: Up and down the Scaffold of Smart Citizen Participation in Dublin, Ireland.” GeoJournal 84 (1): 1–13. doi:10.1007/s10708-018-9845-8.
  • Chang, H., and M. Choi. 2016. “Big Data and Healthcare: Building an Augmented World.” Healthcare Informatics Research 22 (3): 153–155.
  • Clark, B., J. Brudney, and S.-G. Jang. 2013. “Coproduction of Government Services and the New Information Technology: Investigating the Distributional Biases.” Public Administration Review 73 (5): 687–701. doi:10.1111/puar.2013.73.issue-5.
  • Clark, B. Y., N. C. Zingale, and J. Logan. 2017. “Intelligence and Information Gathering through Deliberative Crowdsourcing.” Journal of Public & Non-Profit Affairs 3 (1): 55–78. doi:10.20899/jpna.3.1.
  • Cobos, B., K. Haskard-Zolnierek, and K. Howard. 2015. “White Coat Hypertension: Improving the Patient–Health Care Practitioner Relationship.” Psychology Research and Behavior Management 8: 133–141.
  • Cottam, H. 2018. Radical Help: How We Can Remake the Relationships between Us and Revolutionise the Welfare State. Hachette, UK: Virago Press.
  • Dickinson, H., C. Smith, N. Carey, and G. Carey. 2018. Robots and the Delivery of Care Services: What Is the Role for Government in Stewarding Disruptive Innovation? Melbourne: ANZSOG.
  • Dunleavy, P., H. Margetts, J. Tinkler, and S. Bastow. 2006. Digital Era Governance: IT Corporations, the State, and E-Government. Oxford: Oxford University Press.
  • Falco, E., and R. Kleinhans. 2018. “Digital Participatory Platforms for Co-Production in Urban Development: A Systematic Review.” International Journal of E-Planning Research (IJEPR) 7 (3): 1–27.
  • Forst, B. 2000. “The Privatization and Civilianization of Policing.” Criminal Justice 2 (24): 19–78.
  • Fraser, K. C., J. A. Meltzer, and F. Rudzicz. 2016. “Linguistic Features Identify Alzheimer’s Disease in Narrative Speech.” Journal of Alzheimer’s Disease 49 (2): 407–422. doi:10.3233/JAD-150417.
  • Grimmelikhuijsen, S. G., and A. J. Meijer. 2015. “Does Twitter Increase Perceived Police Legitimacy?” Public Administration Review 75 (4): 598–607. doi:10.1111/puar.12378.
  • Grönroos, C., and P. Voima. 2013. “Critical Service Logic: Making Sense of Value Creation and Co-Creation.” Journal of the Academy of Marketing Science 41 (2): 133–150. doi:10.1007/s11747-012-0308-3.
  • Honingh, M., E. Bondarouk, and T. Brandsen. 2018. “Co-Production in Primary Schools: A Systematic Literature Review.” International Review of Administrative Sciences. doi:10.1177/0020852318769143.
  • Hunt, P., J. Saunders, and J. S. Hollywood 2014. Evaluation of the Shreveport Predictive Policing Experiment. The Rand Corporation. http://www.rand.org/content/dam/rand/pubs/research_reports/RR500/RR531/RAND_RR531.pdf
  • Jasanoff, S. 2016. The Ethics of Invention. New York: W.W. Norton & Company.
  • Joshi, A., and M. Moore. 2004. “Institutionalised Co-Production: Unorthodox Public Service Delivery in Challenging Environments.” Journal of Development Studies 40 (4): 31–49. doi:10.1080/00220380410001673184.
  • Kornberger, M., R. E. Meyer, C. Brandtner, and M. A. Höllerer. 2017. “When Bureaucracy Meets the Crowd: Studying “Open Government” in the Vienna City Administration.” Organization Studies 38 (2): 179–200. doi:10.1177/0170840616655496.
  • Lember, V. 2018. “The Role of New Technologies in Co-Production and Co-Creation.” In Co-Production and Co-Creation: Engaging Citizens in Public Service Delivery, edited by T. Brandsen, T. Steen, and B. Verschuere, 115–127. London and New York: Routledge.
  • Linders, D. 2012. “From E-Government to We-Government: Defining a Typology for Citizen Coproduction in the Age of Social Media.” Government Information Quarterly 29 (4): 446–454. doi:10.1016/j.giq.2012.06.003.
  • Litwak, E., and J. Figueira. 1968. “Technological Innovation and Theoretical Functions Of Primary Groups and Bureaucratic Structures.” American Journal of Sociology 73 (4): 468–481.
  • McMullin, C., and C. Needham. 2018. “Co-Production in Healthcare.” In Co-Production and Co-Creation: Engaging Citizens in Public Services, edited by T. Brandsen, B. Verschuere, and T. Steen, 150–160. London: Routledge.
  • Meijer, A. J. 2011. “Networked Coproduction of Public Services in Virtual Communities: From a Government-Centric to a Community Approach to Public Service Support.” Public Administration Review 71 (4): 598–607. doi:10.1111/puar.2011.71.issue-4.
  • Meijer, A. J. 2012. “Co-production in an Information Age: Individual and Community Engagement Supported by New Media.” VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations 23 (4): 1156–1172.
  • Mergel, I. 2016. “Social Media in the Public Sector.” In Encyclopedia of Public Administration and Public Policy, edited by D. Bearfield and M. Dubnick, Vol. 3, 3018–3021. New York: Routledge.
  • Nabatchi, T., A. Sancino, and M. Sicilia. 2017. “Varieties of Participation in Public Services: The Who, When, and What of Coproduction.” Public Administration Review 77 (5): 766–776. doi:10.1111/puar.2017.77.issue-5.
  • Nam, T. 2012. “Suggesting Frameworks of Citizen-Sourcing via Government 2.0.” Government Information Quarterly 29 (1): 12–20. doi:10.1016/j.giq.2011.07.005.
  • Nelson, R. R., and S. G. Winter. 1977. “In Search of Useful Theory of Innovation.” Research Policy 6: 36–76. doi:10.1016/0048-7333(77)90029-4.
  • Niaros, V., V. Kostakis, and W. Drechsler. 2017. “Making (In) the Smart City: The Emergence of Makerspaces.” Telematics and Informatics 34 (7): 1143−1152. doi:10.1016/j.tele.2017.05.004.
  • Noveck, B. S. 2015. Smart Citizens, Smarter State. The Technologies of Expertise and the Future of Governing. Cambridge, MA: Harvard University Press.
  • O’Neil, C. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. UK: Allen Lane.
  • OECD. 2017. Systems Approaches to Public Sector Challenges: Working with Change. Paris: OECD Publishing.
  • OECD. 2018. Observatory of Public Sector Innovation. Global Innovation Review 2018: Case Collection. Paris: OECD.
  • Osborne, S. P., and K. Strokosch. 2013. “It Takes Two to Tango? Understanding the Co-Production of Public Services by Integrating the Services Management and Public Administration Perspectives.” British Journal of Management 24: S31–S47. doi:10.1111/bjom.2013.24.issue-s1.
  • Osborne, S. P. 2010. The New Public Governance: Emerging Perspectives on the Theory and Practice of Public Governance. London: Routledge.
  • Ostrom, E. 1996. “Crossing the Great Divide: Coproduction, Synergy, and Development.” World Development 24 (6): 1073–1087. doi:10.1016/0305-750X(96)00023-X.
  • Paletti, A. 2016. “Co-Production through ICT in the Public Sector: When Citizens Reframe the Production of Public Services.” In Digitally Supported Innovation, edited by L. Caporarello, F. Cesaroni, R. Giesecke, and M. Missikoff, 141–152. Berlin: Springer International Publishing.
  • Parrado, S., G. G. Van Ryzin, T. Bovaird, and E. Löffler. 2013. “Correlates of Co-Production: Evidence from a Five-Nation Survey of Citizens.” International Public Management Journal 16 (1): 85–112. doi:10.1080/10967494.2013.796260.
  • Pazaitis, A., P. De Filippi, and V. Kostakis 2017. Blockchain and Value Systems in the Sharing Economy: The Illustrative Case of Backfeed. Working Papers in Technology Governance and Economic Dynamics no. 73. Tallinn University of Technology and The Other Canon Foundation. http://technologygovernance.eu/files/main/2017012509590909.pdf
  • Pestoff, V. 2018. “Co-Production at the Crossroads of Public Administration Regimes.” In Co-Production and Co-Creation: Engaging Citizens in Public Services, edited by T. Brandsen, B. Verschuere, and T. Steen, 27–36. London: Routledge.
  • Porter, D. O. 2012. “Co-Production and Network Structures in Public Education.” In New Public Governance, the Third Sector and Co-Production, edited by V. Pestoff, T. Brandsen, and B. Verschuere, 145–168. London: Routledge.
  • Poslad, S., A. Ma, Z. Wang, and H. Mei. 2015. “Using a Smart City IoT to Incentivise and Target Shifts in Mobility behaviour—Is It a Piece of Pie?” Sensors 15 (6): 13069–13096. doi:10.3390/s150613069.
  • Sale, P. 2018. “Gerontechnology, Domotics, and Robotics.” In Rehabilitation Medicine for Elderly Patients, edited by S. Masiero and U. Carraro, 161–169. New York: Springer.
  • Short, E. S., E. C. Deng, D. Feil-Seifer, and M. J. Matarić. 2017. “Understanding Agency in Interactions between ChildrenWith Autism and Socially Assistive Robots.” Journal of Human-Robot Interaction 6 (3): 21–47. doi:10.5898/JHRI.6.3.Short.
  • Shoval, N., and R. Ahas. 2016. “The Use of Tracking Technologies in Tourism Research: The First Decade.” Tourism Geographies 18 (5): 587–606. doi:10.1080/14616688.2016.1214977.
  • Silva, B. M., J. J Rodrigues, I. de la Torre Díez, M. López-Coronado, and K. Saleem. 2015. “Mobile-health: A Review of Current State in 2015.” Journal of Biomedical Informatics 56: 265–272.
  • Snellen, I. Th. M., and W. B. H. J. Van De Donk, eds. 1998. Public Administration in an Information Age: A Handbook (Volume 6). Amsterdam: IOS press.
  • Soete, L. 2013. “Is Innovation Always Good?” In Innovation Studies: Evolution and Future Challenges, edited by J. Fagerberg, B. R. Martin, and E. S. Andersen, 134‒144. Oxford: Oxford University Press.
  • Steen, T., and S. Tuurnas. 2018. “The Roles of the Professional in Co-Production and Co-Creation Processes.” In Co-Production and Co-Creation: Engaging Citizens in Public Services, edited by T. Brandsen, B. Verschuere, and T. Steen, 80–92. London: Routledge.
  • Steen, T., T. Brandsen, and B. Verschuere. 2018. “The Dark Side of Co-Creation and Co-Production: Seven Evils.” In Co-Production and Co-Creation: Engaging Citizens in Public Services, 284–293. London: Routledge.
  • Symons, T., and T. Bass 2017. Me, My Data and I: The Future of the Personal Data Economy. DECODE Project Report. https://www.decodeproject.eu/file/207/download
  • Teece, D. 2018. “Profiting from Innovation in the Digital Economy: Enabling Technologies, Standards, and Licensing Models in the Wireless World.” Research Policy 47: 1367–1387. doi:10.1016/j.respol.2017.01.015.
  • The Economist. 2013. Civic Crowdfunding. Breaking Ground. https://www.economist.com/news/international/21578039- online-start-ups-are-rallying-citizens-revamp-their-neighbourhoodsbreaking-ground
  • Tõnurist, P., and L. Surva. 2017. “Is Volunteering Always Voluntary? between Compulsion and Coercion in Co-Production.” VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations 28 (1): 223–247. doi:10.1007/s11266-016-9734-z.
  • Tõnurist, P., and W. De Tavernier. 2017. “The Welfare State in Flux: Individual Responsibility and Changing Accountability Relations in Social Services.” In Routledge Handbook to Accountability and Welfare State Reforms in Europe, edited by T. Christensen and P. Lægreid, 90–104. Oxon: Routledge.
  • Toots, M., K. McBride, T. Kalvet, and R. Krimmer. 2017. “Open Data as Enabler of Public Service Co-Creation: Exploring the Drivers and Barriers.” In Proceedings of the 2017 International Conference for E-Democracy and Open Government (Cedem 2017), edited by P. Parycek and N. Edelmann, 102–112. Krems: IEEE Computer Society.
  • Townsend, A. M. 2013. Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia. New York: WW Norton & Company.
  • Van Eijk, C. 2017. “Engagement of Citizens and Public Professionals in the Co-Production of Public Services.” PhD diss., The Institute of Public Administration, Governance and Global Affairs, Leiden.
  • Van Eijk, C., and T. Steen. 2016. “Why Engage in Co-Production of Public Services? Mixing Theory and Empirical Evidence.” International Review of Administrative Sciences 82 (1): 28–46.
  • Vanleene, D., J. Voets, and B. Verschuere. 2017. “The Co-Production of a Community : Engaging Citizens in Derelict Neighbourhoods.” VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations 29 (1): 201–221.
  • Verschuere, B., D. Vanleene, T. Steen, and T. Brandsen. 2018. “Democratic Co-production: Concepts and Determinants.” In Co-production and Co-creation: Engaging Citizens in Public Services, edited by T. Brandsen, T. Steen, and B. Verschuere, 243–251. London: Routledge.
  • Verschuere, B., T. Brandsen, and V. Pestoff. 2012. “Co-Production: The State of the Art in Research and the Future Agenda.” Voluntas 23: 1083–1101. doi:10.1007/s11266-012-9307-8.
  • Versichele, M., T. Neutens, M. Delafontaine, and N. Van de Weghe. 2012. “The Use of Bluetooth for Analysing Spatiotemporal Dynamics of Human Movement at Mass Events: A Case Study of the Ghent Festivities.” Applied Geography 32 (2): 208–220. doi:10.1016/j.apgeog.2011.05.011.
  • Von Hippel, E. 2016. Free Innovation. Cambridge, MA: MIT Press.
  • Voorberg, W., V. Bekkers, and L. Tummers. 2015. “A Systematic Review of Co-Creation and Co-Production: Embarking on the Social Innovation Journey.” Public Management Review 17: 1333–1357. doi:10.1080/14719037.2014.930505.
  • Wherton, J., P. Sugarhood, R. Procter, S. Hinder, and T. Greenhalgh. 2015. “Co-Production in Practice: How People with Assisted Living Needs Can Help Design and Evolve Technologies and Services.” Implementation Science 10 (1): 1–10. doi:10.1186/s13012-015-0271-8.
  • Young, J. E., J. Sung, A. Voida, E. Sharlin, T. Igarashi, H. I. Christensen, and R. E. Grinter. 2011. “Evaluating Human-Robot Interaction.” International Journal of Social Robotics 3 (1): 53–67. doi:10.1007/s12369-010-0081-8.
  • Zuboff, S. 2019. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. London: Profile Books Ltd.