The Ethics of NBIC Convergence

Abstract

I. CONVERGENCE AS TECHNOLOGICAL EVOLUTION

On May 11, 2001, a planning meeting was held at the National Science Foundation to organize a workshop that brings together leaders in science, industry and government to consider how cutting edge developments in different areas of science might be integrated to advance the enhancement of human capacities. It was believed that nano-science and -technology provide an unprecedented understanding and control over the fundamental building blocks of matter, enabling an integrated knowledge of matter at all systems levels. By facilitating the convergence of nanotechnology, biomedicine, information technology, and cognitive science (NBIC), organizers hoped to harness this new knowledge for a radical augmentation of human form and function.

Mihael Roco and William Sims Bainbridge were key organizers of this event. Roco chairs the US National Science and Technology Council's subcommittee on Nanoscale Science, Engineering and Technology (NSET), and is Senior Advisor for Nanotechnology at the National Science Foundation. He has been one of the most influential architects of US federal research and funding strategies in the area of nanotechnology. Bainbridge also has an influential role at the National Science Foundation, serving as their Deputy Director of the Division of Information and Intelligent Systems.

The first NBIC workshop was held in December 2001, and was jointly sponsored by the National Science Foundation and the US Department of Commerce. Speakers at the event represented high levels of leadership in government, industry, and science. Among the politicians who spoke were Newt Gingrich, former Speaker of the House of Representatives, and Philip Bond, Undersecretary of Commerce for the Bush Administration. Industry representatives included senior managers from companies like IBM, Hewlett Packard, and Lucent Technologies. Agencies like NASA and the US Department of Energy, and several national labs were also represented, as well as leading scientists from around the country. In all, there were about 50 presentations on topics like human/machine interfaces, delayed aging, sustainable energy, and homeland security.

Annual workshops have followed this first event, and several major grants have been provided to fund research in NBIC areas (Bainbridge, 2004). Published proceedings from the first four workshops are now in print (Roco & Bainbridge, 2002; Roco & Montemagno, 2004; Bainbridge & Roco, 2006a, 2006b).

Several things about these workshops are remarkable. Perhaps most striking are the kinds of enhancements that are contemplated. They are radical. In the Overview for the first NBIC Convergence Report (Roco & Bainbridge, 2002), we are told that within 10–20 years we can implement “fast, broadband interfaces directly between the human brain and machines.” We can make the human body “more durable, healthier, more energetic, easier to repair, and more resistant to many kinds of stress, biological threats, and aging processes.” National security can be advanced by implementing “lightweight, information-rich war fighting systems,” and the promise of interstellar travel can finally be realized. Convergence will also enable “[n]ew organizational structures and management principles.” After listing 20 such enhancements, the conference organizers summarize:

Moving forward simultaneously along many of these paths could achieve a golden age that would be a turning point for human productivity and quality of life. Technological convergence could become the framework for human convergence… . The twenty-first century could end in world peace, universal prosperity, and evolution to a higher level of compassion and accomplishment. It is hard to find the right metaphor to see a century into the future, but it may be that humanity would become like a single, distributed and interconnected ‘brain’ based in new core pathways of society. (Roco & Bainbridge, 2002, p. 6)

In addition to a host of fairly radical technological projects, conference organizers and presenters also speak of a new way of doing science, even of a “new Renaissance.” They argue that nanoscience now provides us with an ability to understand and control matter at the level where its macro-level properties emerge. By filling in this last gap, we can now integrate and organize knowledge after the pattern of nature; natural systems hierarchies provide a template for a unity of science, thereby overcoming the fragmentation and disunity arising from specialization. We thus find a convergence of knowledge, and not just a convergence of technologies. Organizers believed that the appropriate coupling of these two converging processes leads to an acceleration of both developments. They propose new systems of education that are informed by this convergence paradigm.

Soon after the first US NBIC event, several other countries convened workshops or special planning and foresight committees to consider how they should respond. In Canada, activities were first convened by Defence R&D Canada (DRDC), an organization whose mission is to assure preparedness of the Canadian armed forces (Canada, 2003; MacKenzie et al., 2003). As examples of NBIC developments, they considered an “integrated helmet with tuneable hearing, night vision, communications, physical and auditory protection providing tactical awareness and cognition of ‘in-field’ activity.” They also spoke of “integrated wearable, wireless miniaturized sensors, communications and computers woven into the fabric of uniforms/body armor.” In their sponsored studies, they highlight technologies that are similar to those in the NSF/DOC reports, for example, human/machine interfaces and “intelligent systems.” They also consider more conventional medical, agricultural, and environmental applications such as new modes of out-patient treatment, genetically modified foods, and use of bionanotechnology for advancing sustainability.

II. CONVERGENCE AS A SOCIAL TECHNOLOGY FOR REALIZING THE IDEALS OF A KNOWLEDGE-BASED ECONOMY

In the European Union, a High Level Expert Group was convened to consider the prospects of convergence and make recommendations that might guide prioritization of research funding and inform ethical, social and legal deliberation (Nordmann, 2004). The EU report was drafted by its Rapporteur, Alfred Nordmann, and then revised and developed by the 25 member committee over several sessions. This committee was in many ways quite critical of the US workshops and the Canadian response (which was said to have “followed the same pattern” as the United States).

To capture the contrast between the European and American approaches, they coined the phrase “Converging Technologies for the European Knowledge Society” (CTEKS). Instead of presupposing a single agenda or goal — the “enhancement of human performance” — they sought to highlight the agenda-setting character of any convergence, and divert consideration to the many different research programs and problems that might serve as foci for concrete research initiatives (Nordmann, 2004, pp. 7–8). They also rejected any enhancement of human “hardware.” There should not be “engineering of the mind and of the body,” but rather “engineering for the mind and for the body” (Nordmann, 2004, pp. 7, 21).

The differences between the United States and EU reports are manifest in the tone and focus of the reports. The US reports basically include a written overview by conference organizers (Roco & Bainbridge, 2002), but publish free standing visionary statements by scientists, futurists, industry leaders, and politicians. Diverse essays range from fairly traditional medical or agricultural recommendations to wildly futurist speculations on the uploading of consciousness, with no attempt to realistically assess or prioritize these diverse agendas. Most contributions are science oriented, providing overviews of various kinds of research. By contrast, the EU report is a more philosophical document (Nordmann is, in fact, a philosopher of science at the University of Darmstadt, Germany), and it says relatively little about the kind of science at issue (although this is further discussed in some of the supporting documents for the report). The EU report also involves a very explicit reaction to several elements of the US report, with a strong emphasis upon social and political oversight. They provide insightful criticisms of problems of embeddedness of converging technologies, with the potential invisibility of an artificial environment, and of presumed technological fixes to all human problems. They are also very concerned about military applications that are prominently featured in the US and Canadian reports.

The EU committee attempts to contrast their agenda with that of the United States by considering two possible examples of converging technologies (CT) research (Nordmann, 2004, pp. 15, 24). As an example of US convergence, they cite the “perhaps undue attention” given to the MIT Institute for Soldier Nanotechnology (ISN), a fifty million dollar Army funded project. “Bioengineering, robotics, and nanotechnology converge to develop an exoskeleton. Like a second layer of armored skin, it supports the body's metabolic exchange with the environment while adding muscular strength and protection against incoming bullets. Researchers at the ISN are encouraged to explore civilian applications.” As an example of an EU convergence project, they cite the Artificial Hand Project at Lund University, Sweden. “The aim is to develop brain-controlled hand prostheses. This involves strategies for motor control based on electrical signals generated from multiple muscle electrodes or microchips implanted in the peripheral or central nervous system.” In contrast to the US encouragement of spin off applications, the artificial hand is a “nontradable CT service good.” “Since brain control is very difficult and highly inefficient, the artificial hand will hardly become a prototype bulk product to be sold in quantity. Instead, it serves as a highly customized solution where no alternative options exist.” They state that “[i]ncreasing emphasis on nontradable goods is a hallmark of the Lisbon Agenda's so-called ‘European knowledge society’ and one reason for the label CTEKS (Converging Technologies for the European Knowledge Society).”

III. NBIC CONVERGENCE AS A SCIENCE-BASED WORLDVIEW

This Journal issue opens with essays by two of the leaders of the US and EU initiatives. William Sims Bainbridge criticizes the precautionary stance advocated in the EU report. He believes that Western culture and its ethic are based on outmoded religious beliefs that are increasingly undermined by developments in science and technology. In the face of this decaying foundation, we have two possible futures:

1. A radical retrenchment that leads to a world fragmented among competing religious fundamentalisms — Christendom in the Americas, Islam expanding from its present territory further into Africa and Europe, and a swarm of unimaginable new cults conquering Asia. Faith would bring scientific progress to a halt and enslave technology in service of ruling elites… .

2. A transcendence of the traditional human condition — made possible by the unification of all sciences and technologies, establishing a dynamic new creed to replace religion. This would open new worlds for humanity, not only in outer space, but also in the transformation of our own nature. (2007, p. 198)

For Bainbridge, Convergence signifies a new, science-based vision of the world, and it enables the transcendence for which he hopes. In nanotechnology, we see that “the world consists of mechanisms, not mysteries.” In biotechnology we find “evolution in action.” “And cognitive technologies drag human beings themselves under the atomic force microscope, in future decades offering many means by which humans will augment and alter themselves. Once we use technologies to transform ourselves, then the technology becomes more salient for our hopes and beliefs than any ancient myth could be” (2007, p. 202).

To work out this science-based vision, Bainbridge counsels an all-out pursuit of experiments in human life. New reproductive technologies now allow alternate forms of family. In some cases, our instincts might be at odds with these experiments, since they “evolved to sustain life and reproductive success in an environment that is vastly different from the one we inhabit today.” Cognitive science and technology gives us the ability to re‐engineer ourselves, including our instincts, so they better fit with the environments that we likewise alter. Information technology shows a convergence between human and machine intelligence, and makes clear how we are “by nature dynamic patterns of information” (2007, p. 211). Even new governments may be needed.

Bainbridge believes we are currently in a dangerous transition period. We see around us the decay of older forms of life, but don't yet have the full knowledge that provides an alternative.

Humanity is crossing an abyss on a tightrope. Behind us is the old world of religious faith that compensated wretched but fertile people for the misery of their lives. On the other side, if we can only reach it, is a new land where we no longer have to live by illusions, where wisdom and procreation are compatible, where truth and life are one. Nietzsche warned that as we make this perilous crossing, we must not look down. (2007, pp. 202–203)

Bainbridge hopes that “Converging Technologies may be that tightrope, of which Nietzsche wrote, that can carry us to that other side” (2007, p. 209).

IV. EXTENDING THE SCIENTIFIC PROCESS OF DEMYSTIFICATION TO THE CONVERGENCE MYTHS

Alfred Nordmann seeks to demystify what he sees as a Convergence myth. He begins with the traditional contrast between pure sciences like physics, chemistry, and biology, and applied sciences, which are organized around specific projects. Pure sciences are cosmopolitan and involve traditions where “truth is spoken to power.” “As opposed to ‘physics,’ however, “nanotechnology” and “converging technologies” are defined by what they can do, by what problems they might solve, or what challenges they pose” (2007, p. 218). In the end, Nordmann thinks these are “entangled concepts” that bring under a single term heterogeneous research projects. He argues that we should disentangle these endeavors. By doing this, we “destroy mystical conceptions” and thereby open these projects to public scrutiny and oversight.

Nordmann advances his disentanglement as a continuation of the scientific traditions that speak truth to power; it involves an honesty he thinks is lacking in other approaches. For Nordmann, the American “credo of NBIC-convergence” is “we need technological innovation to realize human potential” (2007, pp. 218–219). This involves a “thin and ahistorical conception,” where nature is likened to an engineer and where technological innovation is seen as an extension of evolution (2007, p. 221). On this account, older ethical ideals and traditions are seen as regressive, and we need a new ethic to enable and accelerate the technological development.

Nordmann contrasts this with the European approach, which, he thinks, generally involves “a thick or substantial conception of nature,” viewed not as a nanoengineeer, but rather as “the singular process that evolved a biosphere to which we find ourselves peculiarly well-adapted.” The “credo of CTEKS is: we need social innovation to realize technological potential.” Here technological development is seen as “inherently political and open to social shaping… . Instead of producing transcendence, this political process remains ambivalent in that the expansion of power or control is accompanied by new dependencies, new kinds of ignorance, new problems even of human or ecological survival” (2007, p. 219, 222).

Nordmann recognizes how different philosophical and ethical approaches to Convergence are already entangled in these contrasting visions, but he thinks the process of disentangling is a universalizable strategy that can be embraced by all. He thinks any other approach involves a “deadly embrace,” where the integrity of the science and of the social scientific and philosophical work is called into question. For him, the challenge faced by scholars is, first, to “acknowledge how they themselves are entangled in the construction of the emerging technologies”; second, “escape the deadly embrace”; and finally, from that place of independence “find a way to engage these issues ethically” (2007, p. 226). By pursuing this, he “expresses a strategic optimism in productive disillusionment” that “warns against expecting too much from nanoscience and the convergence of technologies, warns against creating a monster in our midst or mythical entities that are amenable neither to rational debate nor to political choice” (2007, p. 230).

V. CRITICAL REFLECTION ON THE DREAMS OF REASON AND SCIENCE

Despite the obvious, stated differences between Bainbridge and Nordmann, they actually share many commitments; in fact, they are surprisingly close on what might be viewed as the deepest metaphysical issues. Both are concerned about myths that unite people into illegitimate, illusionary projects. Both see science as liberating; it destroys the myths, and in doing this, dissolves the mystical glue that holds together a hodgepodge of heterogeneous projects and people. Both also have a kind of “guarded optimism” that when the disentangling work of science is done, things will settle out ok in the end. But they are equally worried about the prospects of a “dark age” or of “monsters” that might arise if we are not vigilant, and if the illusions take hold. Neither starts with older ethical, philosophical, or religious traditions; rather, they think the work of ethics commences after the genuine work of science is done. They are both anxious to break up any ethical reflection that begins too early, and that arises upstream, entangled in the older, illusory worlds. Only after we escape the “deadly embrace” of older illusions can ethics gain a proper footing. And when ethics is eventually advanced, it will be framed by theories like evolution and ecology, and it will draw on the insights of “anti-reductionist” sciences like genetics, evolutionary anthropology, and social linguistics (for Nordmann) or nanotechnology, systems and information theory, and cognitive science (for Bainbridge).

Jean-Pierre Dupuy's essay critically reflects upon these dreams of reason and science. His target is clearly the US NBIC workshops, more than the EU version. (He was, after all, a member of the EU High Level Expert Group.) But he also considers how the more speculative, utopian interest in transcending the human condition is deeply intertwined with the more realist, demystifying strand.

In sharp contrast to Bainbridge and Nordmann, Dupuy is emphatically not “strategically optimistic” that, after science has done its thing, all will work out in the end. For him, normative reflection involves reasoning in “projected time” (Dupuy, 2004). Because humans are participant-observers in the social and technological co-evolution, their anticipations of the future can have causal efficacy. We thus have a metaphysics of temporality that involves a causal loop between future and past. Within projected time Dupuy seeks to envision a future where the very possibility of responsible human agency is lost. If this future is near and vivid enough to be real, but distant enough not to be a mere prediction of fact, then actions can be taken to avert the anticipated catastrophe. Dupuy thus strongly contrasts the quasi-scientific reasoning found in risk assessment, cost-benefit calculations, and precautionary assessments of techniques, on one hand, with normative reasoning and critical reflection on technology, on the other.

For Dupuy, NBIC Convergence and nanotechnology are not new. They express a long-standing “dream of reason,” one that involves an attempt to escape from the human condition.

Now, for Kant, as for Rousseau, what defines humanity is a ceaseless striving for perfection, a capacity to break free from the constraints of nature — in short, “man's nature is to have no nature.” An extreme form of such humanism, which makes freedom an absolute value, completely detached from any relationship to nature is Sartrian existentialism… . “Man is nothing else but what he makes of himself.” (2007, p. 245)

Dupuy sees “a direct line from this Sartrian metaphysics of the self-made man” to the hopes and dreams of NBIC Convergence. In this dream literally everything is made new: people are to be made, not born, and even death will be abolished. However, paradoxically, as the human capacity to re-engineer nature is extended, our control over what is unleashed is lost. The mechanistic logics of explanation and design involve a negation of all that is genuinely human. Dupuy thus sees in Convergence exactly those aspirations that lead to catastrophe, and he seeks to sketch the trajectory from those projects to the loss of our humanity. In closing, to make vivid the choice we face, he presents two stories, one of creation, another of love. Both confront us with questions of our limits, and of the role these limits play in making human life meaningful.

VI. THE HUMAN/MACHINE INTERFACE AND HUMAN EMBODIMENT: HUMANS AS NATURAL BORN CYBORGS

One of the most troubling features of the US Convergence initiative concerns the scope and character of the enhancements that are advanced. When Bill Bainbridge considers how cognitive science might be used to re‐engineer minds, people think of Orwell's 1984, and when he speculates about how we might become one globally interconnected brain, they think of Start Trek's Borg. These kinds of enhancements seem to be deeply destabilizing. The inhumanity is perhaps best captured by the cyborg image, with his (or do we say “its”) Matrix-like power lines feeding into the brain. Here we have a hybrid, part human, part machine.

Even if these enhancements end up taking a more humane form, they still raise concerns about new, qualitatively different powers given to individuals and groups, and how this ability might be used to manipulate or control exactly the social institutions meant to mitigate abuse. The EU Expert Group attempts to allay those fears by rejecting any enhancement of human “hardware” and shifting attention from physical to social technologies, so the crafting of specific research endeavors is subject to social and political oversight. In this way, they assume that the technological developments can be tamed, thereby made subject to our understanding and control. Instead of “engineering of the body and mind” they advance “engineering for the body and for the mind.”

Andy Clark, a prominent cognitive scientist, challenges the of/for distinction integral to the EU report, and he attempts to show that radical enhancements like the envisioned brain-machine-interfaces (BMIs) may be more continuous with the history of human development than most suppose. The EU report presupposes that we can neatly distinguish between peripheral technologies — external tools that augment function — and the underlying human “hardware.” Here humans are viewed as “‘locked-in agents’: as beings whose minds and physical abilities are fixed quantities, apt (at best) for mere support and scaffolding by their best technologies.” In contrast, Clark argues that “human minds and bodies are essentially open to episodes of deep and transformative re-structuring, in which new equipment (both physical and ‘mental’) can become quite literally incorporated into the thinking and acting systems that we identify as minds and persons” (2007, p. 264).

Human's are “natural-born cyborgs” (Clark, 2003). We continually integrate tools, like the blind person's walking stick, into our body-schema. This happens when there is an agent-world circuit, in which a subject can affect tool-mediated sensory input by means of motor control. As this human-machine interface is iteratively refined, the technology becomes invisible, an indication that it is integrated into a body-schema that is plastic, continually open to revision. Such transparency in use is a “sign of successful calibration” of “plastic neural resources” (e.g., bi-modal neurons that are responsive to both visual and somatosensory information are “tuned” by goal-directed activity within an agent-world circuit). Here we don't have some internal, human CPU that is re-engineered by integrating new peripherals into the natural hardware.

Instead, a problem-solving routine is delicately ‘grown’ so as to maximally exploit the local information field. Such a field can include biological resources, environmental structure, and cognitive artifacts such as notebooks and laptops. As we move toward an era of wearable computing and ubiquitous information access, the robust, reliable information fields to which our brains delicately adapt their routines will become increasingly dense and powerful, further blurring the distinction between cognitive agent and her best tools, props, and artifacts. (Clark, 2007, pp. 275).

By viewing the more radical enhancements as continuous with other forms of technological augmentation of human ability, Clark sees Convergence as compatible with a “sufficiently robust notion of self-control and of responsibility.” New mechanical processes are, for him, no different than other “zombie processes” that populate the brain and that “underpin the skills and capacities upon which successful behavior depends” (2007, p. 279). The challenges they pose are then not much different from those we have always faced.

Clark also thinks the cyborg image he advances will make us more attentive to the ethical issues that need to be addressed:

For once we accept that our best tools and technologies literally become us, changing who and what we are, we must surely become increasingly diligent and exigent, demanding technological prostheses better able to serve and promote human flourishing… . [T]he realization that we are soft selves, wide open to new forms of hybrid cognitive and physical being, should serve to remind us to choose our biotechnical unions very carefully, for in so doing we are choosing who and what we are. (2007, p. 279)

VII. NANOTECHNOLOGY, TECHNOLOGICAL TRANSPARENCY AND HUMAN PRIVACY

For Andy Clark, technological transparency was a sign of normalization; it indicated that an instrument was integrated into the body-schema of an agent. But there are other kinds of transparency. The EU Expert Group paid considerable attention to the ways technology might create an artificial environment that makes us increasingly inattentive and unaware of our new dependencies: “the better they work, the less we will notice our dependence on them or even their presence” (Nordmann, 2004, p. 20; see also pp. 25, 31). In contrast to Clark's “profound embodiment,” this was addressed under the category of “embeddedness.”

In their essay, Jeroen van den Hoven and Pieter Vermaas consider how such developments challenge privacy. After briefly reviewing reasons why protection of privacy is considered important, they present two core assumptions that have been integral to traditional discussions of privacy. First, privacy is about data protection. Protection of privacy thus means “appropriately constraining the acquisition, processing, use, and dissemination of personal data and information.” Second, the primary threat to privacy arises when there is centralized monitoring and control. This is referred to as panopticism, and the authors further distinguish between the continuity of monitoring and synopticism, which is the centralized integration of diverse kinds of information. They then argue that developments like those associated with nanotechnology call into question all of these assumptions.

Radio Frequency Identity Chips (RFID) provide an example of the new developments. These chips can be used to tag commercial objects, which can then provide information on those who have purchased them. Such tags are getting smaller and smaller, and they can be embedded in all commercial products and even be used to tag people. All of this will be invisible. In these cases, the primary concern is not about data protection; it is about the technology itself, and about the ways continuous monitoring is made possible. A centralized registry is no longer needed to gather and integrate diverse data because “people are transformed to carriers of their own profiles” (2007, p. 294).

An example of how the new privacy issues might be addressed can be found in the way cell phones make a noise or give off a light to indicate that a picture is being taken. By building new features into the technology, protection is provided against things like up-skirt photography.

People will have to become aware of the fact that when they buy clothing they also buy a writable memory stick. It changes the conditions under which people consent and intend things. Actions like ‘putting on a coat’, ‘carrying a gift out of a shop’, or ‘to drive from A to B’, are no longer what they appear to be. What actually happens is that one buys a gift and one lets the store know which route one followed through the shop, and ‘driving from A to B’ has become ‘driving from A to B and registering the vehicle registration system. (2007, p. 296)

To address these new privacy issues, van den Hoven and Vermaas emphasize “the need to engage information technology upstream, when it is technology, and there is not yet information” (2007, p. 295).

In many ways this is a fitting recommendation for addressing the ethical issues of Convergence more generally. But their claim also makes clear how deep the challenge of such an upstream ethic will be: to be effective, it needs to be integrated into the design process itself. For that to happen, a far deeper engagement with the emerging technology is needed. This Journal issue concludes with some open questions that still need to be addressed if we are to effectively advance such an upstream ethic of Converging Technologies.

ACKNOWLEDGEMENT

This essay is based on work supported by the National Science Foundation award 0304448–NIRT: From Laboratory to Society: Developing an Informed Approach to Nanoscale Science and Technology. The opinions expressed in this essay are not necessarily those of the National Science Foundation.