The rise of cybernetic citizenship

ABSTRACT

The global COVID-19 pandemic demonstrates how states and companies mobilise new sociotechnical systems to track, trace, evaluate, and modulate the behaviour of citizens. This development illustrates an already-existing transformation of citizenship governance, which this article captures as the move to ‘cybernetic citizenship’. First, the article explores the concept of cybernetic citizenship, providing an overview of the concepts of ‘cybernetic’ and ‘citizenship’ and synthesising these in a discussion of the cybernetic modulation of citizenship. Second, it presents the rise of cybernetic citizenship in the urban realm, zooming in on the case of the Chinese social credit system and the way it affects civic life in the urban realm. Third, it turns into the normative implications of cybernetic citizenship, arguing that it confronts the idea of citizens as equal, free, and vigilant. It challenges equality by turning rights into ends, freedom by turning status into process, and civic virtue by turning excellence into effectiveness.

KEYWORDS:

• Citizenship
• cybernetic citizenship
• COVID-19
• social credit system
• China

1. Introduction

Systemic crises like wars and natural disasters tend to propel new waves of technological innovation. This is demonstrated by the global COVID-19 pandemic, which has led to massive data collection on citizens, track- and trace applications, and automated means of intervention. A fascinating example comes from China, where the pandemic initially started, which has deployed its emerging social credit system (SCS) to fight the pandemic (Knight and Creemers 2021). The SCS uses technological innovations to rate (and often score) citizens to improve public order and trust. For instance, it has made it more socially costly to engage in behaviours that might lead to spreading the disease, such as hiding virus symptoms or evading medical treatment. At the same time, the SCS has been used to relax some restrictions on behaviours that would otherwise be more costly, for example, enforcing financial obligations that are harder to meet due to the economic downfall. The costs and benefits imposed by the SCS are not only monetary, but rather attached to one’s status, rights, and duties as a citizen.

This article investigates the emerging form of cybernetic citizenship as illustrated by the SCS. It builds on a growing body of literature that addresses the transformation brought about by technology to the concept of citizenship (digital citizenship, Mossberger, Tolbert, and McNeal 2008; e-citizenship, Dumbrava 2015; algorithmic citizenship, Cheney-Lippold 2016). At first, literature focused on the way technologies shaped existing, stable aspects of citizenship, for instance, by looking at the digitalisation of state services and online civic education. More recently, there has been a turn toward questioning the foundations of liberal citizenship, as conceptualised in the tradition that follows Marshall (1950). Most notably, Marion Fourcade has coined the term ‘ordinal citizenship’, which implies a transformation of citizenship due to the emergence of large sociotechnical systems that measure, quantify, and evaluate citizens (2021). Some utopian and dystopian scenarios related to the shift in the essence of citizenship due to cybernetic systems were already discussed decades ago by philosophers of technology like Mumford (1965) and Winner (1969, 1978). This article continues this strain of thought and draws from recent work in the humanities (Pickering 2010; Hui 2019) that signals the revival of cybernetic thinking.

The article’s central claim is that, empirically, we are witnessing the rise of cybernetic citizenship as demonstrated by the Chinese SCS and similar sociotechnical systems around the globe; normatively, this development has urgent ethical, legal, and political implications. First, cybernetics professes to be a general epistemology that takes the networked computer as its model (Winner 1969). While citizenship in the current international order of states is still primarily based on the mechanised worldview of enlightenment philosophy, cybernetics challenges this worldview and thereby the sense of citizenship based on it. Second, cybernetics accounts for how sociotechnical systems learn and adapt their behaviour according to contingent inputs (Krivý 2018). Hence, it explains how citizens’ seemingly accidental and contingent behaviours are connected and harnessed for citizenship governance. And third, initially emerging in the wake of computerised systems, cybernetics speaks to a much longer history of philosophical thinking that mobilises the notion of a living machine to understand the institution of citizenship (Mumford 1965).

The article proceeds as follows. Section 2 develops the concept of cybernetic citizenship, synthesising the outlines of ‘cybernetics’ and establishing perspectives on ‘citizenship’. Section 3 describes the rise of cybernetic citizenship through the lens of the application of the SCS in the urban realm in China. Section 4 develops a normative analysis of cybernetic citizenship, arguing that it erodes core distinctions that inform modern citizenship: between rights and ends, status and process, and excellence and effectiveness.

2. Concept

This section develops the concept of cybernetic citizenship. First, it provides an overview of the cybernetic perspective and how it is used to understand different phenomena. Second, it outlines different established perspectives on citizenship. Third, it shows how cybernetic citizenship modulates the relation between citizens and institutions in terms of space, time, and social interaction.

2.1. Cybernetics

Cybernetics can be understood in two ways: a scientific discipline and a general worldview. As a scientific discipline, cybernetics emerged at the end of World War II. A varied, interdisciplinary group of scholars, including the mathematician Norbert Wiener, psychiatrist Ross Ashby, and anthropologists Margaret Mead and Gregory Bateson, convened around a shared scientific view of the world that encompassed human and non-human elements in a communication network. This view arose in the wake of the development of early computer systems, mainly for military purposes, which gathered information from the environment and translated the data into behavioural outputs by means of feedback mechanisms. Due to the deterministic qualities of technologies at its inception, the initial focus of cybernetics was on (relatively) closed systems with negative feedback loops. This view does not fit current cybernetic systems, which are open, constitute positive feedback loops, and can control but also adapt to environmental changes (Krivý 2018).

As a general worldview, early cyberneticians successfully identified and conceptualised a way of thinking that is found in today’s biology, computer science, statistics, and political science. Cybernetics is a general understanding of the world that emerges from studying three seemingly distinct phenomena: life processes, machines, and political communities. It combines these perspectives through the single metaphor of the kybernetes, the ancient Greek word for a pilot. The kybernetes had a practical understanding of navigating the ship and adapting to unknown circumstances, like the weather. The influence of cybernetic thinking on modern technoscience can be demonstrated by the increasing use of core cybernetic concepts – such as systems, networks, feedback, and information (Françios 1999) – and their application to the world.

As a scientific discipline or a way of thinking, cybernetics is too complex to be comprehensively captured in this article. Yet, a brief explanation through three core concepts – networked system, recursive feedback, and informatisation – will suffice to make a relevant connection with citizenship.

At first, cybernetics is about the networked system. This notion refers to the way cybernetics understands space, which has horizontal and vertical dimensions. The horizontal dimension is captured by the idea of a network – a spatial arrangement of interconnected nodes connecting and exchanging information (Castells 2004, 3). Networks are relatively flat and distributed: they have no centre of command. A prime example is the Internet, the main driver of the ‘network society’. The vertical dimension is captured by the idea of a system. This idea adds control or regulation to the communication network. A system is a goal-directed assemblage of functional components. It is not simply a machine that repeats a mechanical movement, like a clock. Instead, it has a direction and can pursue an often indeterminate number of ends. For this reason, systems are ‘quasi-living’ entities (Wiener 1989); they respond to their environment in light of certain ends. Systems are open, and they interact with their environments. The system dimension introduces a hierarchy to communication networks, which have layers or components of networks that hierarchically relate to one another. An example is DNS (Domain Name System), which is nested within the Internet architecture (Galloway 2004). The hierarchy inherent to DNS is not necessarily top-down, based on a command-and-control structure. On the contrary, most systems that operate on the Internet today produce bottom-up, emergent hierarchies, such as the hierarchy of websites displayed after a Google Search query.

Next, cybernetics is about recursive feedback. This relates to the way cybernetics understands time, how networked systems operate and give rise to a particular temporality (Hui 2019). Recursive feedback allows systems to have a ‘memory’ of the past and to ‘predict’ a future. This process is often likened to forms of reflective learning, similar to how a child learns a language. The notion of recursive feedback refers to a movement between a system and its environment, whereby inputs received from the environment are processed and fed back into the system as new inputs. This is not a mechanical process of mere repetition, like the mechanical movement of a clock, but a movement of adaptation. This means that contingent inputs do not lead to noise or disruption of the system but are reflexively interpreted to anticipate future events. From a cybernetic perspective, recursive feedback can be found everywhere in our world. Animals use it to process information from their environment and adapt their behaviour to new circumstances. The human brain uses it to learn new skills. And digital systems use it to engage in processes of machine learning, in which new information is analysed based on a vast corpus of existing data. In other words, recursive feedback relies on sensing the environment, processing information, and generating behavioural change.

Finally, cybernetics relates to informatisation. It captures how systems interact. Interaction is premised on intelligibility, i.e. systems must understand each other, and they do so through information. Cybernetic systems can only understand reality once it is accessible as ‘information’. The practical notion of informatisation explains how phenomena are modulated to become informative in a cybernetic sense. This modulation shows a kinship with the much-discussed idea of datafication (Lycett 2013), which broadly means (i) eliciting the informational aspect from physical objects and events and their context, (ii) processing data in such a way that it can easily be manipulated and transferred, and (iii) recombining informational resources for particular contexts and problems. According to De Mul (1999), informatisation presupposes phenomena to be programmable (subject to calculation), synthesisable (subject to self-organisation), and manipulable (subject to artificial change of limits that govern the natural world). In short, cybernetics modulates entities so that they can be quantified, organised, and manipulated. Consider the technology of genetic modification. It captures genes as a form of code that can be programmed; it synthesises this code with a certain purpose in mind, for instance, eradicating a genetic disease; and, by doing so, artificially overcomes the limits of the natural world.

2.2. Citizenship

Like cybernetics, citizenship is a complex concept with various meanings. In the most basic sense, citizenship denotes (equal) membership in a self-governing political community (Joppke 2010, 160). This definition spells out citizenship as a basic aspect of the human condition, which considers humans as ‘political animals’ (Depew 1995) capable of co-governing communities. Considering this basic notion of citizenship, even stateless refugees could potentially be ‘citizens’ of virtual/cloud-based political refugee communities (Orgad 2018). Yet, since the world is still dominated by territorial-based nation-states, citizenship has been chiefly conceptualised by this spatial reality (Hirschl and Shachar 2019) and can be seen by using two perspectives.

The first is a human rights perspective. It focuses on the internal function of citizenship – the relation between the state and the citizens. T.H. Marshall (1950) was one of the leading proponents of the view that citizenship denotes a gradual increase in citizens’ rights, starting from civil rights, via political rights, to social rights. This historical thesis accounts for the way citizens have gained, amongst others, rights to private property, voting rights, and social rights to education and healthcare. After World War II, this idea of citizenship as giving access to rights internal to states has been extended to the idea that it could also give access to the more general category of human rights. As a citizen of France, for example, one could appeal to international or European human rights while residing in Germany. This view is paradoxical since the enjoyment of universal human rights depends on having the legal status attached to the membership of a particular nation-state. For this reason, Hannah Arendt (while not ascribing to the Marshallian view of citizenship) proclaimed that citizenship is essentially a ‘right to have rights’ (1973, 298).

The second is a sociological and anthropological perspective. Citizenship consists of different, interrelated elements, such as rights, obligations, and identities (Joppke 2010), to which might be added civic virtue and political participation. This view is more attentive to how the relationship between states and people, both as citizens and as migrants, is constituted. It considers citizenship not only as a legal status but also as a sociocultural reality of belonging. Being a German citizen does not only mean having a German passport or rights associated with it but also sharing a historical and cultural sense of being German and acting according to some German way of doing things. It is about ‘citizenship in practice’ (Barnes, Auburn, and Lea 2004), everyday social life, how people do things in a political community. This view is often used to distinguish ‘real’ or ‘true’ citizens from the ‘other’ citizens.

Citizenship, thus, can be understood both as a formal legal status and de facto membership in a political community (citizenship as everyday practice).

2.3. Cybernetic citizenship

A cybernetic understanding of citizenship is not new. It originates from what might have been the most sophisticated ‘living’ technology in ancient Greece, the trireme – a mighty warship that played a decisive role in the Peloponnesian Wars. Plato is frequently cited as the Western thinker who considered citizenship governance in the ancient polis in analogy with (or as a metaphor for) the steering of a ship (Winner 1980, 129). In The Republic (Plato [1997] 488a-489e), Plato argued that unlike the owner of a trireme, the trierarch, who lacks expertise in navigation, the kybernetes, an experienced crew member, should be in charge of the ship. This is because the kybernetes would have practical skills and technical knowledge of navigating the ship. Using this analogy, Plato advocated for governance by experts, who would use techno scientific knowledge and practical skills to govern the citizenry. While Plato initiated the debate, it was Aristotle who used the cybernetic analogy more decidedly to theorise the governance of the ancient Greek city, the polis (Mumford 1965, 275). In The Politics (Aristotle 1998] 1276b19-30), Aristotle spelt out what may be considered the first comprehensive concept of citizenship governance in cybernetic terms:

Just as a sailor is one of a number of members of a community, so, we say, is a citizen. And though sailors differ in their capacities (for one is an oarsman, another a captain, another a lookout, and others have other sorts of titles), it is clear both that the most exact account of the virtue of each sort of sailor will be peculiar to him, and similarly that there will also be some common account that fits them all. For the safety of the voyage is a task of all of them, since this is what each of the sailors strives for. In the same way, then, the citizens too, even though they are dissimilar, have the safety of the community as their task. But the community is the constitution. Hence the virtue of a citizen must be suited to his constitution.

Cybernetic thinking has migrated from the philosophical discussion of citizenship in Ancient Greece, via the Enlightenment philosophy of life and the organic (Kant, Schelling), through biology (Mendel, Darwin) and mathematical physics (Schrödinger, Gödel), to computer science (Turing, Wiener), dissipating into a variety of disciplines (cf. Hui 2019), including politics. And yet, despite its influence on political philosophy, cybernetic thinking has had limited practical impact on citizenship governance for most of history. The reason is that, as a technical notion of governance, it requires sophisticated conditions of human organisation, techniques, and technologies to be effectively applied. Early forms of cybernetic governance were applied to relatively small-scale units. A paradigmatic example is Jeremy Bentham’s Panopticon (August 2021, 16), which establishes a form of recursive feedback (the prisoners regulate their own behaviour because they might be observed) within a system that governs the network of communication (the prison architecture). Other examples applied during the Industrial Revolution, both in education through the grading system and in factory management through Frederick Taylor’s scientific management theory. But while institutions like schools and factories have relatively stable boundaries and are easy to regulate, the scale and complexity of political communities have prevented, thus far, cybernetic governance from being extended to society as a whole.

The proliferation of sociotechnical systems in every aspect of life makes the cybernetic governance of citizenship a practical possibility today. Three transformations are contributing to that. First, cybernetic systems transform the space of citizenship, promoting both connectivity (horizontal) and systematisation (vertical). Paradoxically, the more connectivity is achieved, the more room for control and regulation is made. In terms of a network, as Varnelis and Friedberg (2013, 32) argue, the space of citizenship is shaped by the merging of the ‘geospatial Web’ and ‘ubiquitous computing’. It relies on the expansion of cyberspace – the digital space through which citizens connect and their institutions, and simultaneously on a vast physical network of connected devices and sensors: phones, cameras, radio-frequency identification chips. These two aspects allow for the placing of citizens and institutions as connecting nodes in a network that functions like a geometric overlay on the physical space. In this context, citizens have to digitally check-in when they enter physical spaces so that health authorities can trace their behaviour in case a COVID-19 outbreak happens at a location they visited. Interconnectivity under such a citizenship regime is less of a free choice and more of a public imperative: being a node means being able to send and receive information. In terms of a system, cybernetic governance generates a built environment that connects people and institutions to a certain hierarchy of ends (Zandbergen and Uitermark 2019); citizens are placed into a hierarchical relation vis-à-vis the ends of political institutions, which in turn relate to a higher-order end that justifies the overall system. To be sure, citizenship has always been organised around an overarching end, at least in spirit. Yet, with the advent of cybernetic sociotechnical systems, individual behaviours can systematically be brought into a manageable relationship with the ends of the political community, similar to how the behaviours of the worker can be brought into a relationship with the ends of a company. Consider how political institutions are now able to set quantifiable metrics for individual behaviours to align with institutional aims. Such systematisation sets citizenship into a concrete rather than imagined relation to an overarching end of the political community, for the sake of which citizenship is governed.

Second, cybernetic systems transform the time of citizenship, turning it into a historicised process that draws from recorded traces and calculated predictions. Practically, this happens through the introduction of recursive feedback into the relations between citizens and institutions; more information is processed between citizens and officials through government institutions. A tax officer might scan a citizen’s tax record and send it back for corrections. This helps the tax officer ‘learn’ about faulty behaviour and the citizen to ‘learn’ about the correct way to file taxes; it is a process of mutual adaptation. Today, such recursive feedback is increasingly automated by means of algorithmic processes. An algorithm might interpret the behaviour of citizens based on historical patterns and assign them a risk score, which indicates the probability that taxes will be filed correctly (Alm and Torgler 2011). Automated processes of recursive feedback rely on vast collections of historical data to deal with scale and complexity (Fourcade and Johns 2020). They can use statistical methods to learn from the populations’ behaviour and use this data to predict and modulate individual behaviours. In this way, cybernetic systems mediate the temporality of human activities (Hui 2019). To be sure, the impact of technological change on time has been a theme of reflection since the rise of mechanisms of time and, in particular, the use of the clock in industrial operations. Yet, earlier time machines differ from contemporary time machines because they did not adapt to contingencies. The clock keeps ticking at a regular pace, regardless of whether the worker goes on strike. Cybernetic systems, in contrast, might anticipate the strike and, as a response, allocate more free time to workers to modulate their sentiments. This temporal logic penetrates now into the realm of citizenship.

Third, cybernetic systems transform the interaction between citizens and institutions by informatising it. This interaction becomes subject to calculation, which means that standards for being a ‘good’ citizen or institution are determined through quantifiable metrics. Primary instruments for this are techniques for ranking, rating, and scoring. Public officials like politicians are rated and ranked, as are public institutions like universities and hospitals. Government agencies are regularly subjected to impact assessment exercises by which they are scored on performance indicators. Similarly, citizens get scores for increasingly many activities: not only do they get grades for their educational accomplishments, but they are scored as consumers, as social media users, and even as citizens qua citizens (in the case of the SCS). Such metrics rely on standardisation and interoperability. It means that standards of measurement are brought from a local to a global understanding, like the way different standards of time have been replaced by the global Greenwich Mean Time measurement. This shows itself through global standards for qualification and certification, like ECTS (European Credit Transfer and Accumulation System) points in European education (Backer and McQuilla 2020). Interoperability, in turn, ensures that different metrics can ‘speak’ to one another. To illustrate, the metric of one’s social media activity might interact with one’s credit score. Finally, these forms of quantified assessment rely on citizens and institutions to be liable to modulation by means of incentives. Thus, metrics can only have effects when they are linked up with incentive structures that modulate behaviour. Citizens can be rewarded when they engage in public service and sanctioned when they commit traffic infractions, with the goal that they regulate themselves.

To sum up, cybernetic citizenship means that citizenship is increasingly shaped by the cybernetic understanding of space, time, and interaction. In terms of space, citizens are turned into communicating nodes that are put into a systemic relation with defined ends of their political community. In terms of time, citizenship becomes a process that draws from recorded history, leading to a prediction of future behaviour. In terms of interaction, citizens and institutions are informatised: subjected to quantifiable assessment, which is standardised and made interoperable, and premised on the practical notion that behaviour can be modulated through incentive mechanisms. This challenges the normative basis of citizenship in terms of status, rights, and identity, as discussed in Section 4. Before we turn to the normative assessment of cybernetic citizenship, we present how it already exists in the urban realm.

3. Case study

This section provides empirical evidence for the rise of cybernetic citizenship, with a focus on the urban realm. First, it gives a high-level description of how cybernetic systems are integrated into the fabric of citizenship at the urban, national, and transnational levels. Second, it discusses the case of the social credit system, emphasising how it affects citizenship in Chinese cities.

3.1. Citizenship in the digital age

With the proliferation of personal computers and the Internet, the transformation of political institutions has been mainly linked to increases in data collection and networked exchanges of digital documents. Certain rights and duties have become embedded in digital communication systems. Thus, with the advent of cloud services, large internet platforms, artificial intelligence, and advances in cryptography, new sociotechnical systems have emerged; they can collect different types of information and process them through statistical methods that enable these systems to learn from contingent inputs and execute outputs accordingly. Some social rights, for instance, are now conditioned on having a digital identity, navigating an online platform, and digitally signing or verifying information. This is particularly true in the wake of the COVID-19 pandemic, as governments around the world have implemented new digital passes (e.g. proofs of vaccination), which citizens are required to display in order to benefit from specific rights and services (e.g. free testing), enter into specific locations, and use public transportation.

These new sociotechnical systems operate both in digital environments and in the tangible world, where a large number of physical devices are increasingly linked together under the heading of the Internet of Things. Together, they constitute an emerging ecosystem that has an increasingly powerful hold on the everyday lives of citizens and the regular functioning of political institutions, be it at the municipal, national, and transnational levels.

At the municipal level, sociotechnical systems are integrated into smart cities, sensing cities, and smart urbanisation initiatives. Urban institutions deploy systems that collect data to monitor and assess citizens’ behaviours and interpret the data to enhance the responsiveness of public service infrastructures and increase the citizens’ well-being. The same systems are used to influence or modify citizens’ behaviours, nudging them to act in such a way as to improve the fit between the built environment and the public life it affords (Krivý 2018). Perhaps the clearest illustration of this trend is the multiple attempts by municipalities to encourage civic participation by rewarding citizens points whenever they engage in a particular set of desirable activities. Some cities in Europe are experimenting with a social credit system based on ‘good deeds’ to encourage civic engagement, social solidarity, and political participation. For instance, the municipality of Cascais in Portugal has established a credit system (InnoWave Citypoints) that awards citizens who achieve ‘good deeds’ included in the municipal catalogue (e.g. buying local products, adopting an animal, donating clothes, helping the elders). The points can then be used to receive a particular reward selected from the catalogue of municipal benefits (e.g. free entry to museums, books, concert tickets, animal care services, and public transport) (Orgad and Reijers 2022).

Similar developments exist at the national and transnational levels. At the national level, states are deploying systems that monitor the behaviours of citizens, evaluate them, and intervene when needed. Tracking how citizens act or interact with a governmental authority can help identify suspicious and potentially problematic behaviours, thereby triggering state interventions. Liberal democracies have already implemented such systems in a variety of contexts, ranging from predictive policing (Kaufmannn, Egbert, and Leese 2019) to tax fraud detection (Meuwese 2020) and national welfare abuse mitigation. At the transnational level, global corporations like Facebook and Uber increasingly participate providing public services, which are key to citizenship, such as maintaining a platform for public discourse or providing mobility services. The operators of these platforms act as what Frank Pasquale (2018) describes as ‘functional sovereigns’ – actors unilaterally dictating the rules of their platforms with broad discretionary powers, no checks and balances, and little to no democratic accountability. In this capacity, global corporations deploy some of the most powerful sociotechnical systems that combine vast data collection, often combined with machine learning and profiling techniques. Using these systems enables corporations to adapt to the behaviour of Internet users, possibly influencing and even manipulating their behaviour, as became clear with the Cambridge Analytica scandal (Maschewski and Nosthoff 2017).

These types of sociotechnical systems operate at different levels yet remain interconnected. Indeed, the municipal realm in which citizens live their daily lives is mediated by local, national, and transnational systems that often feed on the same information sources and potentially share information. In the not-so-distant future, it could become commonplace for the many sensors that are already embedded in our everyday environment to be connected to a variety of larger nationwide systems and transnational platforms relying on artificial intelligence to profile citizens into specific categories based on their current or future activities, moods, and behaviours.

While the trend we are witnessing is global, its most salient and alarming example is the Chinese social credit system and its implementation in the urban realm in China. The SCS is a sociotechnical system that explicitly targets citizenship (and citizens) and is arguably the most advanced system of its kind. It targets all three levels described above, involving city pilots, national government agencies, and commercial entities. We turn to it now.

3.2. Social credit in the urban realm

The SCS was initially conceived to fill the gap of credit reporting in China at the backdrop of scandals and a pervasive distrust in commercial agents during the late 1990s. Setting up an American-style credit reporting system like FICO, it was believed, would make information about commercial entities more robust and, hence, increase public trust. The initiative was soon taken up in the broad governance strategy of the Chinese Communist Party (CCP), extended beyond mere financial creditworthiness toward social and moral trustworthiness. The SCS initiative was aligned with General Secretary Jiang Zemin’s doctrine, articulated in 2001, of ‘governing by virtue’. At the same time, the SCS initiative got entangled in the broader Chinese governance strategy of ‘social management’, which finds its origins in Leninist-Maoist cybernetics and complex systems engineering (Hoffman 2017). After its initial development, the CCP launched a detailed Planning Outline of the Construction of a social credit system (2014). This document lays out the SCS as having three dimensions: economic, establishing the idea of financial creditworthiness; legal, ensuring compliance; and moral, installing a notion of trustworthiness based on following ‘professional ethics and behavioural norms’ (Orgad and Reijers 2019, 2022). While the initial emphasis of the rollout has been on the commercial sector – companies have a social credit attached to them – the SCS covers all of society, including citizens. It is arguably the first cybernetic system that targets citizens and, thus, citizenship.

The CCP characterises the overall aim of the SCS as generating an environment in which ‘keeping trust is glorious and breaking trust is disgraceful’ (State Council 2014, Para I (3)). Rather than constituting a single system with one master score for each citizen, the SCS comprises a dynamic ecosystem that connects various systems with governance experimentation initiatives. At the basic level, the SCS connects blacklists (for trust breakers) and redlists (for trust enhancers) that different state agencies maintain through information sharing and a joint sanction system. Practically, if a citizen ends up on one blacklist, s/he might also end up on another one and undergo the sanctions attached to both. Information sharing is centralised by the National Credit Information Sharing Platform (NCISP) and facilitated through memoranda of understanding between government agencies. Social credit information includes a variety of domains, from compliance with court orders, via tax payment history, up to reported behaviour in public transport.

The urban realm constitutes a particularly important nexus in the rollout of the SCS. This takes the form of an innovative and decentralised mode of experimentation with SCSs through incentivising municipal governments to compete for being ‘model cities’. This has generated a variation in SCS configurations in different urban contexts. Today, the SCS is heterogeneous and fragmented, existing not as a single, centralised system, but rather as a decentralised network of connected initiatives. Its effectiveness varies significantly across sectors and implementations. As such, it does not fit the dystopian Black Mirror characterisations it was faced in some of the Western media.¹ At the same time, all SCS initiatives share a certain overarching logic regarding their relationship with citizens’ space, time, and interactions.

First, the SCS is premised on the idea that citizens should interact with one another and their institutions in a particular manner that promotes public order. Reliance on advanced data collection, sensing technologies, and monitoring techniques enables municipalities to ensure that citizens comply with specific rules and do not misbehave in public spaces. A few smart city governments, like Shenzhen, have been experimenting with facial recognition technology in traffic situations that directly shames jaywalkers. Other cities are experimenting with different point systems, combined with specific processing methods attached to catalogues of praiseworthy and blameworthy behaviours. The catalogues include behaviours like leaving trash in hallways, which makes one lose points, or helping the elderly, which contributes to gaining points. These points are the basis for a more sophisticated system of rewards, such as preferential access to employment opportunities, and punishments, such as increased official scrutiny when partaking in an examination. This shows how the SCS reinforces the idea that citizens are integrated into a hierarchical system, where encouraged behaviours are linked to the ends of the government. For instance, donating to a charity might help a citizen enter a red list because it aligns with those behaviours designated by party officials as being in line with their overarching end. As such, the behaviours of citizens are subsumed under more general categories of ‘good’ and ‘bad’ conduct, which, in turn, are subsumed under even more general governance strategies/principles (e.g. social harmony). The Chinese government has initially followed a strategy of incentivising the development of social credits by large private companies like Alibaba and Tencent. This yielded numerous scoring systems, the most famous was Alibaba’s Sesame Credit that gathered information on one’s payment history, personal character, and social relations. Although not linked with the NCISP, these systems can affect daily lives. Low scores can restrict services, ending with reduced Internet speed or limited access to public toilets, and high scores can lead to commercial privileges like discounts on mobile phone rates (Orgad and Reijers 2019, 2022). While commercial actors were eventually denied credit reporting licenses, the CCP set up the public-private partnership Baihang Credit in 2018, which is likely to become the central scoring agency.

Second, the SCS transforms the traditional understanding of citizenship as a set of rights granted to a particular individual to a new idea of citizenship based on memory and anticipation. It creates a digital ‘memory’ of citizens’ behaviours to anticipate future events, generating a score that determines the conferring of rights or the imposition of restrictions based on whether past behaviours conform with the system’s ends. This is well illustrated by how the SCS has been used during the pandemic to provide specific affordances to virtuous individuals who have been negatively affected by the lock-down and introduce restrictions and punishments to the less virtuous individuals who did not properly comply with the rules. For instance, in the municipality of Shandong, an individual’s social credits will be reduced for refusal to wear a mask in public; participating in gatherings, parties, and crowded activities; or failing to implement the quarantine policy. All this empowers the ‘smartification’ of the urban environment where citizens live through the proliferation of sensors and information gathering systems. This goes hand in hand with China’s initiatives to enhance its surveillance capacities through installing security cameras linked to facial recognition software in many urban areas. Specifically, the SCS understands citizens in accordance with behaviours that can be measured and quantified. It relies on the presence or absence of payment data, records about types of conduct, or machine-readable recordings of infractions. This understanding can be predefined according to a list or catalogue, or emerge from big data analytics, processing information about citizens’ behaviours and using this to anticipate future events. Particularly relevant in this context is the City Brain initiative in Hangzhou, one of China’s most prominent projects in building a smart city. Originally intended to deal with the problem of heavy traffic in the city (Liu 2020), the initiative has progressively expanded in scope, engaging in a much broader range of data collection to promote a variety of public services – such as facilitating car parking, increasing public security, and providing efficient government subsidies (Zhao et al. 2020). The data gathered by the Hangzhou City Brain project comes from public departments datasets, private companies that partnered with the project, and data voluntarily provided by citizens. Today, the scope of the Hangzhou City Brain extends over nearly every aspect of city management. The system is designed to track, evaluate and provide constant feedback on social management, economic regulation, market supervision, public service, environmental protection, and government operation. Because of its focus on prediction and anticipation, the City Brain has allegedly successfully resolved the traffic problem in Hangzhou (Chen 2019) and further enhanced the efficiency of the police force (Qiu et al. 2019).

Third, when it comes to the informatisation aspect of the SCS, the system evaluates citizens based on their behaviours, which are benchmarked according to quantifiable metrics and data points. The centrality of a quantitative assessment of both citizens is well illustrated by the approach adopted by Suzhou, one of the first cities in China to explore the SCS. The municipality of Suzhou established a dual citizen scoring system, comprising the Fragrans Score and the Suzhou Civilization Code (quickly abandoned due to fierce critiques). The Fragrans Score is a scoring service operated by the Suzhou Public Credit Information Centre, which is mainly used to reward trustworthy people. While the algorithm for the score was never disclosed to the public, it seems to rely on a variety of data points, including age, marital status, educational background, professional title, volunteer service, health status, and a series of criminal and administrative records. Depending on the score, citizens can benefit from municipal rewards (i.e. travel, culture, health, education, parking, medicine). At the same time, the municipality promoted the Suzhou Civilization Code to evaluate people based on a broader scale of factors, mostly oriented toward the respect of transportation rules (e.g. one’s score would be deducted by 50 points if one drove a red light, and 100 points if one drove a car drunk). A high civilization score would provide significant benefits, such as better access to education and greater allocation of public resources. Yet, shortly after the announcement, the Suzhou Civilization Code incurred numerous critiques across China, as the treatment of what is considered a ‘civilised’ behaviour would depend on arbitrary data points. Facing the pressure, Suzhou abandoned the notion of a ‘Civilization Code’.

These three points illustrate how the implementations of the SCS, despite their divergences, contribute to creating cybernetic citizenship. The SCS changes the space of citizenship by introducing layers of connectivity and systematically incorporating parts of the digital world, where citizens interact with one another and with public institutions, into the physical space through devices, sensors, and digital cameras. The SCS also transforms the time of citizenship, which is no longer established at a particular point in time, but rather is constantly redefined in an ongoing manner from the historical tracing and evaluation of past behaviours. Finally, the SCS modifies the interaction between citizens and public institutions because of the process of datafication or informatisation. The identity of citizens is defined by metrics; and their behaviours are rated, ranked, and scored according to performance indicators.

4. Normative assessment

This section presents a normative assessment of cybernetic citizenship. It shows that the dimensions of citizenship are shifting, as conditioned by its cybernetic modulation. This happens through three tendencies: (1) the primacy of ends over rights, (2) the transformation of citizenship of status into process, and (3) the primacy of civic effectiveness over excellence. Together, these tendencies can be understood as normative consequences of what Alain Supiot (2019) calls the rise of ‘governance by numbers’. Yet, rather than seeing quantification as the basis of a new form of citizenship governance, it may be seen as an inherent aspect of a broader perspective of cybernetic governance.

4.1. From rights to ends

Modern architectures underpinning citizenship, both as a legal status and a societal practice, give rise to rights and duties that citizens might have. Those rights are categorical and not premised on the fulfilment of government ends as a prerequisite. Furthermore, the public sphere is based on the principle of equality before the law. When citizens act in the public sphere, they act as equals, for example, by voting in elections. Arendt likened the architecture of the public sphere to the space enclosed by the city wall (1958): like a wall, it sets categorical conditions for human action. Yet it does not actively interfere with the activities of free, equal citizens, and it does not introduce its agency.

Cybernetic citizenship challenges equality before the law because it links equality to human activities and the extent that they fulfil political ends, thereby introducing an architecture of ends. This notion resonates with recent debates in legal and political theory that identify alternative modes of non-legal regulation of human behaviour, such as ‘nudging’ (Sunstein 2015), market forces, social pressures, and technological codes (Lessig 2006). Yet, unlike these modes of regulation and governance, the architecture of ends, as promoted by cybernetic citizenship, is not one mode of governance out of many, but rather the basis (and sometimes a prerequisite) for realising human rights. In other words, it is not a complementary mode of regulation (alongside laws, codes, and economic forces) but often the only mode enabling access to certain rights and privileges. The realisation of rights is ends-dependent; citizens are data collectors pushed by a goal-directed system. An architecture of ends means that citizenship may still be a ‘right to have rights’ under the human rights perspective (Section 2) yet, in and of itself, it provides access to a narrower list of rights and privileges. This turns citizenship, which is already a thin legal concept in its exclusive entitled rights, to even a lighter concept.

Linking human rights to the pursuit of political ends as a prerequisite for their realisation changes the space of citizenship in three senses. First, citizens are no longer treated equally as citizens or as a subgroup of citizens (e.g. women, students, elderly), but as individuals who are constantly judged and evaluated by the extent that they align their actions with political ends. It applies a logic like that of ‘personalised pricing’, which attempts to capture the entire social surplus in a transaction for setting up the price, to the citizenry. Personalised pricing is a method that looks carefully at the customer and calculates various known factors (computer type, IP location, frequency of searches, etc.) to offer a price for a good or a service that is compatible with the customer’s profile. The method of personalised pricing, some argue, may be more equal than categorial equality because everyone is treated, to paraphrase Marx, according to their abilities and needs. Yet, applying this method to the citizenry is a radical change in the essence of equality, making it dependent on individual actions and thereby limiting its spatiality. Second, cybernetic citizenship makes the realisation of rights not only a function of actions in the physical world but also of one’s traces in the digital world. The space of citizenship is extended to cyberspace, yet this extension widens the possibilities to restrict rights due to a new set of digital activities that may negatively affect one’s equality vis-à-vis others in cyberspace. Third, organising the political community in line with an architecture of ends, citizens may find themselves in a position of having equal political rights on paper, yet without the ability to exercise them. This happens, for instance, when a low score limits access to participate in the public sphere. Cybernetic citizenship thus undermines equal citizenship since the role of networked systems generates inequalities (and hierarchies) between the people whose behaviours they modulate. These inequalities shift the idea of categorical rights toward conditional ‘rights’.

4.2. From status to process

Modern citizenship is shaped by institutions that uphold the rule of law, not just the rule by law. The rule of law is an elusive principle with various definitions. In essence, it is a legal system in which the law is the source of all political authority, no one is above the law, and all people are equally subject to the law and its enforcement. Fuller (1969) drafted eight formal principles that should exist in a rule-of-law system; the law should be general, public, understandable, coherent, enforceable, not-retroactive, similar both in books and in actions, and stable. Other definitions are broader and include the notion of just law, which respects political freedom and human dignity, since slave, apartheid, or fascist laws are also based on formal legality (Hart and Fuller 1965).²

The concept of citizenship is one of the mechanisms for securing a stable legal system (alongside others, such as the ability to amend laws and political compromises that enable peaceful co-existence). It means that, once a person is a citizen, it is a lifelong status that guarantees civil rights, including, at least in a rule-of-law system, the ability to take critical political initiatives. The lifelong status of citizenship creates stability through time, which Arendt (1958) refers to as duration. Political freedom exists through the absence of law (everything which is not forbidden is allowed) and is protected (from arbitrary domination) through its presence (Pettit 2011). Citizens can count on this legal basis unconditionally: they neither have to earn nor deserve it.

Cybernetic citizenship transforms citizenship as a legal status, whose essence derives from the law and stands despite the passing of time, into a conditioned concept, the way citizenship is implemented in practice. First, citizenship becomes contingent. The enjoyment of a complete set of human and civil rights depends on the evaluation of data about past events. Cybernetic systems learn from the behavioural histories of citizens to decide which rights and sanctions to grant or impose, when, and where. This is evident in the COVID-19 context, where some behaviours are drawn within the historicised reputation of people, like violating quarantine rules. Cybernetic citizenship is a dynamic, adjustable process that turns citizenship from an unconditional legal status to a historically lasting process, subject to fault and correction. Second, because it does not explicitly rely on laws and regulations, cybernetic citizenship, as illustrated partly by the Chinese SCS and Covid-19 policies, is not subject to the same constraints as the rule of law system. If it exists at all, the legal ground for citizenship restrictions is not always publicly formulated, clear, coherent, and retroactive; in some cases, there is no due process of law and full respect of property and liberty. Cybernetic citizenship, thus, turns citizenship from a status to a process whose entitled rights must be constantly earned, and moves the legal status outside the sphere of legality to the societal reality, the way people do things in a political community (for the sociological view of citizenship, see Section 2). We are witnessing a transition from a system governed by the rule of law (‘nomocracy’) to a system governed by numbers (‘numerocracy’). Unlike citizenship as a status, a governance by numbers (or reputation, Dai 2018) system is constantly subject to change and must be earned. It requires citizens to be constantly aware of good behaviour standards and change their behaviour accordingly. It thereby normalizes the notion of ‘citizenization’, citizenship as a process, and indirectly reinforces the idea of ‘good citizenship’ (Orgad and Reijers 2022). Third, cybernetic citizenship urges citizens to ‘labour’ for their citizenship to get access to certain rights. This ties in with Scheuerman’s (2005) argument that the ‘busyness’ generated by contemporary life makes that people have ‘no time for citizenship’ and that free political actions are relegated to laborious activities. As such, being a citizen has increasingly less to do with taking free initiatives and more to do with engaging in activities that fit the historicised expectation of the political community, as aligned with cybernetic incentives. As a result, citizens have less time to exercise their freedom since the labour to maintain rights is tied to a cyclical, never-ending process.

4.3. From excellence to effectiveness

Citizenship implies interaction, which means that citizens must be responsive to institutions and vice-versa. This idea is captured by the term civic virtue, which can be a virtue ascribed to a citizen (e.g. vigilance) and an institution (e.g. justice). These virtues are two-sided: they appeal to excellence and effectiveness. This distinction lies at the heart of Aristotle’s conception of citizenship. For free citizens, the effectiveness of being ruled coincides with the excellence of ruling; citizenship is enacted through skills (effectiveness) and virtues (excellence). Effectiveness denotes activities needed for a citizen to function according to institutional ends, like paying taxes. It requires extrinsic motivation, safeguarded by external goods such as reputational or material gains. Excellence denotes activities of democratic political life; most notably, activities in which political agents disclose themselves by acting and speaking in concert. It requires a sense of intrinsic motivation, captured by MacIntyre (2007) as the requirement to aim at goods internal to a practice.

Cybernetic citizenship challenges civic virtue as excellence by informatising the qualities of citizens and institutions. These qualities are rendered calculable, programmable, and manipulable to make interaction possible. Institutions increasingly use calculated metrics: risk scores, credit scores, and other measurements. To get to these measurements, civic virtue is not assessed directly but indirectly: it matters whether one pays one’s bills in time, not whether one has the proper disposition to do so. Cybernetic citizenship is thus premised on an implicit form of behaviourism: qualities are derived from behaviours that can be made discrete, measured, and calculated. This comes to the fore with implementing the SCS in the urban realm through a translation of atomic behaviours (e.g. paying a bill, jaywalking) into calculated measurements following a catalogue of good or bad deeds. By this process, citizens become responsive to institutions but only in an instrumental way – they engage in behaviours not primarily because these bring them internal goods but because they are extrinsically motivated.

Turning civic virtue from excellence into effectiveness limits the interaction between citizens and institutions in three senses. First, it makes civic virtue past-oriented rather than future-oriented. The metrics that establish civic virtue are derived from existing norms and standards. Citizens and institutions will strive to comply with existing norms and standards, but this, in turn, limits their ability to act innovatively and creatively, for instance, by developing new political projects. Second, it introduces the ‘tyranny of metrics’ (Muller 2018), which means that measurements become more important than what they are supposed to measure. This risks generating a false sense of ‘good citizens’ on the side of institutions, because conformist, rule-abiding citizens are valued higher than active, vigilant citizens who challenge existing structures. On the side of citizens, this same dynamic creates incentives to game the system. For, as de Filippi rightly observed, under the SCS the motto is no longer ‘I have a good credit score because I am a good citizen’, but rather becomes ‘I’m a good citizen because I have a good credit score’ (De Filippi 2019). Third, this transformation eventually instrumentalises the citizen, for civic virtue is no longer a function of the citizen’s happiness but only relates to the external well-being of the community. Cybernetic citizenship thereby falls prey to what Heidegger (1977) called the enframing of modern technology: it confers everything, including citizens themselves, into standing reserve, a resource to be used. To be clear, liberals often consider civic virtue as instrumental to the sustenance of the liberal state (McTernan 2014). Yet, liberal thinkers have generally refrained from cultivating virtue, leaving the cultivation of character almost exclusively to the private sphere, augmented with civic education. Cybernetic citizenship, on the contrary, explicitly reimagines civic virtue by subjecting it to a logic of metrified interaction. It is, therefore, ‘republican’ in reviving civic virtue in the public sphere but subverts it in turn by instrumentalising it fully.

5. Concluding remarks

Cybernetic sociotechnical systems reshape the lives of citizens in the urban realm. This article presents the first philosophical discussion of the concept of cybernetic citizenship, its implementation through the proliferation of sociotechnical systems like the SCS, and the normative challenges it raises.

The article contributes to the ongoing debate on the impacts of ‘dataism’ on citizenship. It foremost responds to the work of Fourcade (2021) on ordinal citizenship, adding to her work in three ways: by linking the discussion on citizenship and technology to the philosophy of technology, embedding the debate within the frame of cybernetics, and considering ‘ordinality’ in algorithmic governance as an enabler of a conception of citizenship that can be traced back to the birth of political philosophy. The article goes beyond ordinal citizenship and its valuable criticism (Joppke 2021) by arguing that cybernetic citizenship is not tied to any particular citizenship ideology such as neoliberalism, although it has allied itself with that in recent times. Just as cybernetics was embraced by liberal and communist states (Csizmas 1971), it can emerge in different ideological contexts. The article returns to the perplexity of philosophers of technology who consider that cybernetics is on its way to replace metaphysics and explores what this means for citizenship.

The rise of cybernetic citizenship, as discussed in the article, is limited in scope. First, emphasising the urban realm, the article is inclined toward the ‘internal’ perspective on citizenship, focusing on the relationship between citizens and political institutions rather than on between states. It would be valuable to inquire into how cybernetic systems like distributed blockchain technologies would impact the inequality between citizenships (Gstrein and Kochenov 2020). Second, it accounts for a new model of citizenship that is not yet fully applicable but exists as a loose conglomeration of sociotechnical projects. Still, we might follow John Torpey (2001) in his discussion of the invention of the passport, arguing that a technical tendency, as fractured, ineffective, and disorganised as it was at the start, tends toward effectiveness and consolidation. Hence, cybernetic citizenship represents a potent new force in our world that might transform citizenship and what it means to enact it.

Acknowledgments

We would like to acknowledge the thoughtful comments of Federico Tomasello and two persistent anonymous reviewers.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This research is funded by the European Research Council (ERC) under the European Union’s Horizon 2020 Research and Innovation Programme (Grant Agreements No. 716350 and No. 865856).

Notes

1. The episode Nosedive of Black Mirror (Netflix) is often compared to the Chinese social credit system in the popular media. In this episode, citizens use digital devices to rate each-other, which leads to a social ranking with vital benefits and sanctions attached to it.

2. For the differences between the ‘rule of law’ and the ‘rule by law,’ see Tamanaha (2004).