Shoshana Zuboff’s Response to Martin Schulz : The New Weapons of Mass Detection

  • -Aktualisiert am

Unternehmer mit staatlichen Aufgaben? Googles Eric Schmidt war auch schon als Minister im Gespräch Bild: AFP

The new military-informational complex: Why politics - not technology - will determine our future. A comment on Martin Schulz.

          16 Min.

          I’ve been holding my breath since the Guardian broke its first Snowden story on June 5, 2013 revealing the NSA’s bulk collection of phone data.  There is a tragic deja vu for me. In 1988, when Google founder Larry Page was fifteen years old and the word “internet” was at least ten years away from public recognition, I published In the Age of the Smart Machine.  The book was based on a decade of fieldwork inside a range of newly computerizing workplaces. During those  years I watched each group succumb to the same pattern. Computer systems that offered rich new learning opportunities at all levels of the firm were  hijacked by a narrow economic model and a managerial ideology of unilateral control.

          (Deutsche Fassung: „Die neuen Massenausforschungswaffen“ von Zshoshana Zuboff)

          Few managers could resist adapting the new systems to a surveillance agenda, as they sought detailed knowledge of workers’ behavior and performance. “The promise of automation,” I wrote seemed to exert a magnetic force, a seduction that promised to fulfill a dream of perfect control and heal egos wounded by their needs for certainty. The dream contains the image of ‘people serving a smart machine,’ but in the shadow of the dream human beings have lost the experience of critical know better than, to question, to say no.” I learned then that only new business models, purposeful leadership, clear strategies, and institutionalized values and practices could alter this course.  Now whole societies are being drawn into the same tragedy.   Technology’s promise of empowerment is under threat from those who seek to dominate through perfect information,  as they grind their way to profit and control.  

          Reading President of the European Parliament Martin Schulz’s message to the German public in his recent FAZ essay, “Warum wir jetzt kämpfen müssen”.  I find myself  exhaling –– at least a little. Schulz writes that the challenge for social democracy in the next century will be to “civilize and humanize” the new technology revolution, while standing fast for the “inviolability of human dignity” in a new world. The challenge I see is that, thanks to Edward Snowden, we know that the technology revolution has once again been hijacked by the dream of perfect control. It’s being used as a  Trojan horse for a still poorly understood convergence of public and private institutions that wield unprecedented power over information. This new power bloc operates outside our control as citizens and consumers.  I’m calling it the military-informational complex, because its  power derives from the production and deployment of what I call new weapons of mass detection composed of information and the technical apparatus required for its access, analysis, and storage.

          We can civilize technology only if we confront this new behemoth. It is this new concentration of power over information, not technology itself,  that drives the “compulsion to control” and the “anti-liberal, anti-social and anti-democratic” dynamisms of which Mr. Schulz writes.  As we have all come to understand, these effects are not limited to any one society. The poison  travels fast and infects the prospects of many societies and the wider civilizational context they share.

          The technological revolution that began with so much promise for liberation and empowerment has become a collective Faustian nightmare. Who among us wants to live without the information and connection that technology has enabled?  But who suspected that we would have to forfeit democratic principles,  personal control, and the very existence of social trust?

          We desperately need political leaders who grasp what is at stake in this epochal transition to what I call an “information civilization.”  There are many exciting new fronts that demand the energy and creativity of a new politics, and I hope to write about those in future articles. But the military-informational complex casts a long shadow over all efforts at reinvention, and therefore it’s where we must start.

          Right now we are drifting toward an iceberg  that  has all the earmarks of a “turnkey totalitarian state”, as one retired NSA official described it. President Obama’s January speech on government surveillance was a bitter  disappointment to all who see an urgent need to avert this collision.   .  Now the eyes of the world turn to Europe for a new politics that can navigate  safe passage.  .   We are at the very beginning of this effort to reclaim  what the military-informational complex has hijacked in the realms of governance, commerce, and freedom. That’s why I regard Martin Schulz’s vision as more than a call to action for the German people and the wider European community.  It is a necessary message to all humanity.

          Eisenhower’s Message in a Bottle

          The military-information complex is a convergence of public and private expertise in the control and analysis of information camouflaged by a forest of excuses. The official story is that these growing powers are a necessary response to forces beyond control: technological requirements, digital proliferation, autonomous market dynamics, and security imperatives.

          There’s been a steady flow of books and articles on the NSA over the past few years including a surreal  2012 article by NSA expert James Bamford in Wired magazine that described its vast new domestic spying operations: “including the complete contents of private emails, cell phone calls, and Google searches, as well as all sorts of personal data trails — parking receipts, travel itineraries, bookstore purchases, and other digital ‘pocket litter’...the NSA has turned its surveillance apparatus on the US and its citizens.” One NSA official boasted, “Everybody’s a target; everybody with communication is a target.” Far less has been known, however, about compliance, and/or complicity, with the NSA among the major U.S. Internet companies. Now we have more facts that paint an alarming picturing.

          Rewind for a moment to 1961 and President Eisenhower’s farewell address to the American people. He had revised the speech many times over a two year period. In each draft he insisted on retaining a crucial passage, as if he was determined to send a message in a bottle to be discovered, read, and grasped by future generations.  American society was under threat from a new “military-industrial complex,” he warned,  and only “an alert and knowledgeable citizenry” could ensure that “security and liberty” would both  prosper. Guardian readers will recognize the phrase “security and liberty” as the tag line on Glen Greenwald’s now immortal series.  Greenwald found the bottle.

          Five years later economist John Kenneth Galbraith, elaborated his concept of the “technostructure” in  The New Industrial State. “Power,” he wrote, had “passed to...a new production function... men of diverse technical knowledge, experience or other talent, which modern industrial technology and planning require.” Galbraith’s book celebrated an industrial oligarchy at the heart of the military-industrial complex, enmeshed in state functions and protected by state power, insulated from public accountability, and innocent of responsibility. Why? Because it promoted itself as the inevitable expression of technology’s indisputable “requirements.”  Galbraith had fallen under the spell of technological determinism.

          Despite the overwhelming international popularity of the book, an erudite Viennese-born American economist and university President, Walter Adams, aggressively challenged Galbraith’s defense of the status-quo. “My hypothesis,” Adams protested, “holds that industrial concentration is not the inevitable outgrowth of economic and technical forces...” Instead he cited “unwise, man-made, discriminatory, privilege-creating governmental action” as the  “keystone in an edifice of neomercantilism and industrial feudalism.”

          The military-industrial complex, Adams warned, was the most damning expression of the technostructure: “government not only permits and facilitates the entrenchment of private power but serves as its fountainhead. It creates and institutionalizes power concentrations which tend to breed on themselves and to defy public control. The scenario of events should be familiar. The ‘mad momentum’ of an international weapons race ...generates demand...for the development and production of sophisticated weaponry.” The question he posed haunts us now: How can the state regulate and oversee an industry upon which it depends for its arsenal?

          The Demilitarized Internet

          Now fast forward to the 1980s when a highly educated, energetic, and libertarian-oriented group of software designers and engineers brought us the Internet.  The new communications medium was intended to bypass entrenched institutional interests in favor of horizontal communication free from bureaucracy.  When the foundations for today’s Internet were first developed, several computer protocols competed to become the global standard.

          According to Internet historians Martin Campbell-Kelly and Daniel Garcia-Swartz, two factors enabled the TCP/IP protocol, developed by the ARPANET community, to dominate.  The first was that ARPANET operated as an ad-hoc permeable organization in which many people could participate and quickly reach consensus. The second was that the U.S. government’s military interests in ARPANET were finally disengaged from the development process and channeled into another network effort called  MILNET.  In the absence of  the defense and security agenda, ARPANET  rapidly expanded its network on a global scale and attracted many other networks to it. TCP/IP became the defacto Internet protocol precisely because it was demilitarized.

          Weapons of Mass Detection

          I once had a philosophy professor who said that the way to learn dialectical thinking was to remember that things always turn into their opposite.  The Snowden documents show just how quickly such reversals can occur. Despite its promising beginnings, the Internet and more or less all communications data, our data, have been militarized in theory and practice.This has occurred pretty much the way that Adams described. First, an arms race with  the “terrorists.” Then a frenzied effort to develop the most sophisticated weaponry.  The only difference in our century is that weapons are composed of information and the techniques of information analysis and control.  When it comes to these new weapons of mass detection, we are both the bullet and the bullseye.

          We know that in principle and in fact, US security operations, along with those of other nations (Germany, Spain, France, Britain, Australia, Canada, New Zealand, Israel, and probably more) have broad access to our data. As early as 2001, the European Parliament published a 200-page report on the “Echelon” surveillance program concluding “within Europe, all communications, such as emails, phone and fax traffic, are regularly monitored by the NSA.”

          We’re learning from Snowden the degree to which NSA operations escalated and institutionalized breaching democratically established laws and individual rights.  Regarded by the NSA and other intelligence agencies as essential to the “war on terror”, our data have been weaponized.  A key feature of the new weapons system is that we are unable to discern the extent to which we ourselves might be or become targets of the armaments to whose manufacture we have unwittingly contributed. We supply the information bullets, and then the weapons are turned on us.

          Secret reports captured by Snowden and released in January by Pro Publica, the Guardian, and the New York Times detail the extent to which the NSA and British intelligence agency GCHQ exploit a tsunami of data from smartphones. As one British agent was quoted in a 2008 report, “anyone using Google Maps on a smartphone is working in support of a GCHQ system.” In 2010 another British analyst described the development of smartphone harvesting as a “mobile surge,” and compared it to the surge of troops into Iraq and Afghanistan.

          Demilitarization had been essential for what Harvard Law Professor Jonathan Zittrain calls the “generative” capabilities of the Internet –– the ways it lends itself to trust, interaction,  invention, and creativity. Militarization is already having the opposite effect, as firms withdraw their data from cloud servers and governments explore new regulations and infrastructures that  enable nation-specific privacy controls.

            Of equal concern are the economic effects of information militarization.  It suppresses the creative adaptation to human needs that is capitalism’s greatest strength . In the annals of capitalism, the production of prosperity and well-being have depended on a steady flow of commercial “mutations” that better align business with the changing needs of people.

          The Lavabit saga illustrates this point. Its value proposition was the guarantee of secure email. Founder Ladar Levison chose to shut  down the company rather than violate his users’ trust by turning over encryption codes to the NSA. Now facing civil contempt charges, Levison is fighting for the right to operate a business model that is fundamentally different from the prevailing commercial paradigm. Over the centuries there have been times when concentrations of power crushed the creative process of economic evolution, leading to stagnation and social decline. But there is little, if anything, in the historical record that compares with the current threat of unconstrained concentrated information power, universal in scope and hidden from view.

          The big Internet companies are essential to this picture.  The New York Times reported on a 2012 NSA strategy paper that listed as one of its objectives, “to identify new access, collection and exploitation methods by leveraging global business trends in data and communications services.” This was seen as part of a larger strategy to upgrade NSA capabilities to be fully consistent with the “information age.” The NSA required the materiel to produce the world’s most sophisticated information armaments. Silicon Valley would have to become an ally, a target, or both. We know that the big tech companies comply with NSA requests, but there are many questions we need answered about the extent to which they are actively complicit in the secret militarization of our data. In either case, what’s clear is that they had already done the “heavy lifting” necessary for the military production of weapons of mass detection.

          Big Tech Goes Adversarial

          Once Google, Facebook, and others fell into the thrall of a narrow financial model dependent on advertising revenue, it became clear they had few inhibitions about violating our privacy, ignoring our claims to self-determination, and betraying our trust.  The peer-oriented dynamics of those networks gave way to a new kind of imperialism in which the companies unilaterally controlled our data and surreptitiously colonized more dimensions of our online behavior and personal identity. In order to avail ourselves of their networks, upon which we had already become dependent, we found ourselves providing the content for them to sell.  Worse still, those transactions would be invisible to us and often in violation of our interests.

          Zittrain warned in 2008 that the rise of “tethered” information devices controlled by their manufacturers, like smart phones and video game consoles, opened the door to surveillance, because they relay data to manufacturers that can be accessed by law enforcement or regulators. Tethered devices turned out to be training wheels for a new generation of audacious practices linked to mobile applications. Many apps are forging new institutional alliances across the online commercial environment without our knowledge and often at our expense.

          Barclay’s in the UK detailed its plans to sell customer data including photos, voice recordings, social media comments, and location data from mobile apps.  Other new alliances aim to shape or sanction our behavior.  Many applications designed to enrich the end user ––from health monitoring to location apps –– have formed lucrative new business partnerships.  You may find your blood pressure readings shared with your bank or your insurance company as a proxy for credit worthiness and premium adjustments. Your location information may be accessed in real time by retailers seeking a detailed profile of your shopping habits both inside and outside their stores.

          Classified documents released in January by Pro Publica and others reveal the ways in which the evolving adversarial practices of online commercial surveillance provide the opportunities and the mechanisms for information weaponization. Data streams from mobile applications and games are enthusiastically described as valuable sources of intelligence. The popular mobile video game Angry Birds (over one billion apps downloaded) was singled out as a particularly useful source because its developers had embedded code for intrusive tracking software that provides information on political affiliation, sexual orientation, and other personal behavioral data along with the more common demographics.

          Based on the same cache of reports The Washington Post describes NSA and British intelligence practices related to harvesting “tracking cookies” from personal computers and mobil phones ––especially those assigned by Google anytime a user connects to its properties or services. Such cookies,  an already contested mechanism for tracking user browsing patterns, are the basis for pop-up ads and other forms of online advertising. One British report notes that cookies  “are gathered in bulk, and are currently our single largest type of events.”  While the documents don’t indicate whether Internet companies, including Google, cooperated with these programs or simply complied with  government data requests, the convergence between commercial and military aims illustrates a shared surveillance paradigm.

          The New Complex Takes Shape

          Snowden brought us more. The PRISM program documents show the NSA accesses user data through servers belonging to Apple, Google, and other companies.  The NSA “paid millions of dollars to cover the costs of major internet companies” involved in this program. According to these disclosures, Microsoft closely collaborated with the NSA, providing access to pre-encrypted email streams and Skype calls. Microsoft, like the other companies, insisted that they only complied with legal requests. 

          Reporting in early June  by the New York Times indicates that while some Silicon Valley companies “bristled” at NSA requests, others were “more compliant,”  including Google, Microsoft, Yahoo, Facebook, AOL, and Apple. According to the New York Times account, the documents “illustrate how intricately the government and tech companies work together, and the depth of their behind -the- scenes transactions...In at least two cases, at Google and Facebook, one of the plans discussed was to build separate secure some instances on company servers.”   In some cases, the New York Times reported, tech company employees have national security clearance. In others, NSA agents installed their own software on company servers, and even hung out at the company for days or weeks of system monitoring.

          By September 5, The Guardian exposed documents on the NSA’s SIGINT program stating that the NSA “actively engages US and foreign IT industries to covertly influence and/or overtly leverage their commercial products’ designs.”   The document assures its readers that, “To the consumer and other adversaries, however, the systems’ security remains intact...successfully exploiting systems of interest within the ever-more integrated and security-focused global communications investing in corporate partnerships and providing new access to intelligence sources...”

          Linger for a moment on the NSA’s word choice and sentence construction.  In NSA-speak “consumers” are cast as a subset of a larger category called “adversaries.”  As set and subset, the two groups share a single  preposition.  The clarity is bracing. As consumers we had come full circle from an involuntary  role as providers of materiel for the new weapons system to  now also being its adversaries and thus its targets. The idea of consumers as adversaries was a routine aspect of the NSA’s prevailing logic.  Bulk data collection assumed that because anyone might be the enemy, everyone was a suspect.

          Among the big tech companies the treating the consumer as bullet and bullseye, was established practice.  In this context perhaps the prospect of engaging with the NSA seemed like a reasonable extension of  “best practice.” Already in 2009 Google CEO Eric Schmidt had displayed an arrogant indifference toward privacy concerns when he commented, “If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place...the reality is that search engines including Google do retain this information for some time...we are all subject in the United States to the Patriot Act. It is possible that that information could be made available to the authorities.”

          In retrospect one finds news in Schmidt’s words:  the suggestion that the intermeshing of private and state power was already well underway, that even search data were candidates for scrutiny, and, of greatest concern,  that all of this had come to be regarded, in Schmidt’s mind at least,  as a matter of course.

          After release of the SIGINT documents, the tech companies denied all knowledge of their contents. But the Guardian’s reporting  quoted a former White House official who alluded to a long productive working relationship between the companies and the NSA. Had Google and Facebook learned surveillance tactics from the NSA in the first place? Or was it the other way around? The identity of the military-informational complex was taking shape along with its assumptions, attitudes, interests, and perspectives. Armaments production was well underway. As Walter Adams lamented a half century ago, under these conditions it is ludicrous to  expect our governments to impose the people’s interests on a technology sector that is the critical source of its weapons. It is equally ridiculous to imagine that big tech can represent  the people’s  interests to the U.S. or any or government. Their  posturing as our champions is simply brilliant public relations.

          Technology is the Trojan Horse

          There are many unknowns and more questions than answers. We need far more information about the interface between the tech companies, telecoms, and state intelligence agencies. We need to know more about the variations among the tech companies as they developed policies, practices, and patterns of engagement with the NSA. Still, a picture emerges from these documents of the real-time trial-and-error invention of  a new military-informational complex who’s power and scope, at once universal and deeply intimate, extend beyond anything that President Eisenhower could have imagined.

          Like its 20th century counterpart, the new complex claims it is a necessary response to implacable technological, market, and security “requirements.”  It’s the technology that makes us vulnerable and forces these measures upon us, we are told.  Its the behavior of our enemies that compels us to respond this way, the intelligence agencies claim.  It’s the only way we can generate enough revenue to provide you with service, says big tech. Eric Schmidt hides behind the smokescreen of  a fictional autonomous technology: “the reality is that search engines including Google do retain this information for some time...”  NSA talking points  blame the complexity of digital networks and the terrorists who use them, “A major challenge in this digital age is that terrorists and citizens use the same communications networks. Because terrorists and key targets use the full spectrum of global communication tools...NSA maintains a robust set of tools and capabilities...” Tech executives and NSA operatives share a strangely helpless innocence.

          Ignorance for Us is Bliss for Them

          I say no! We are in the realm of politics here, not necessity.  The growing concentration of information power is not, to echo Walter Adams’s protest over forty years ago, “the inevitable outgrowth of economic and technical forces.”   Power hides inside technology so that it can, like  the Trojan horse, be stealthily ushered into our midst.  As we know from that ancient story, once it enters the gate we must fight to drive it back beyond the borders of our lives.  The difference today is that the horse is invisible. 
          The new military-informational complex is driven by  human choices. Big tech’s choices arise from its narrowly conceived economic aims and its imperial self-interest.

          The intelligence agencies, most notably the NSA, are driven by a self-perpetuating and inherently untestable manic belief that it is possible to control all outcomes through universal “information dominance.”  There is no inevitability here except, perhaps, the will to power.  The two sides of this complex converge at the interface with our lives, thanks to their mutual interest in power over information. Both have developed unchecked by either democratic oversight or the legitimate claims of consumer sovereignty and individual self-determination.

          Secrecy has been essential to this hijacking of commerce and governance, eliminating the possibility of  “an alert and knowledgeable citizenry.” As history has shown, when people can see how the machinations of power effect their daily lives ––my credit score, bank loan, insurance premiums, my rights, my freedom – they are more likely to revolt.  Ignorance for us is bliss for them.

          What Should You Wear?

          It’s personal.  The Trojan horse lives inside our phones and Google searches.  It grazes quietly in our fitness apps and trots alongside as we run our Saturday morning errands.  Even the most cursory reader of Orwell knows what research confirms about human behavior under surveillance. When people know they are being watched, they tend to conform, both consciously and unconsciously, to the expectations of the watchers. First the “facecrimes” disappear. (I’m aware of “managing” my face when I’m wending my way through airport security.)  Then “thoughtcrimes” are extinguished. (Have you ever thought twice before Googling certain phrases or about the subject line in an email?)  That self-censorship is a life sentence to an endlessly repeating present.   Nothing new can happen once we curb our thoughts.

          Surveillance and compliance also effect our bodies.  Studies demonstrate the relationship between one’s sense of personal mastery and control on the one hand, and health and longevity on the other.  Too much submission and compliance literally produces stress, disease, and early death.

          The ugly politics of information concentration and control can only be tamed by a new political response. We can and we must take back what is being taken from us.This is the necessary “social movement” to which Martin Schulz gives voice.  The goal is to assert our insistence on an alternative path to the future, one in which governance and commerce are reclaimed in the name of  inclusive democratic principles, rational capitalism, and the legitimate claims of individual self-determination.

          A century ago, as Mr. Schulz reminds us, our grandparents, great grandparents, and great great grandparents suited up for their confrontation with a new industrial power whose aims and methods were indifferent to their well-being. They knew what to wear: their allegiance to their families and their coworkers, their hunger and their aching bodies,  their yearnings for a better life, their fierce insistence on social justice, their determination to find voice in solidarity.

          It’s now time for us to suit up. Ours is another epochal contest with great power, but our wardrobe is different.  What should we wear?  I suggest we wear our commitment to the evolution of democracy, not its demise. We cloak ourselves in the knowledge that mutual trust, transparency, democratic control, shared accountability and responsibility, and creative invention are our best hopes for the future of connected people on an ailing planet. We clothe ourselves in our rights to personal self-determination: to choose how we live and to live effectively. We smooth the wrinkles from our aspirations for a dynamic economy in which prosperity arises from a trustworthy commerce aligned with our interests and a society in which all of us can participate and flourish. We wear our outrage. We wear our dread of a future squandered in stagnation, submission, and a savage struggle over scarce resources. Let’s get dressed.

          About the Author

          Shoshana Zuboff, author of The Summons: Our Fight for the Soul of an Information Civilization (Forthcoming, 2015), is the Charles Edward Wilson Professor of Business Administration (retired) at the Harvard Business School.

          Weitere Themen

          Laschet kann sie alle schaffen

          TV-Kritik: Anne Will : Laschet kann sie alle schaffen

          Will Armin Laschet Kanzler werden, muss er in die Offensive. Sein Stehvermögen zeigt er bei Anne Will. Nur Klimaaktivistin Luisa Neubauer liefert ihm einen starken Schlagabtausch.

          Regierung verschärft Emissionsziele Video-Seite öffnen

          65 Prozent weniger CO2 : Regierung verschärft Emissionsziele

          Die Bundesregierung will das Emissionsziel für 2030 auf minus 65 Prozent CO2 verschärfen. Das kündigten Bundesfinanzminister Olaf Scholz und Bundesumweltministerin Svenja Schulze (SPD) als einen zentralen Punkt des geplanten neuen Klimaschutzgesetzes an.


          TV-Kritik: Anne Will : Laschet kann sie alle schaffen

          Will Armin Laschet Kanzler werden, muss er in die Offensive. Sein Stehvermögen zeigt er bei Anne Will. Nur Klimaaktivistin Luisa Neubauer liefert ihm einen starken Schlagabtausch.
          Winfried Kretschmann, Ministerpräsident von Baden-Württemberg, verfolgt die Rede Boris Palmers beim digitalen Landesparteitag der Grünen.

          Ausschlussverfahren der Grünen : Belastet der Fall Palmer Baerbocks Wahlkampf?

          Die Grünen wollen den Tübinger Oberbürgermeister Boris Palmer ausschließen. Der sieht dem Verfahren optimistisch entgegen. „Es ist gut und reinigend, wenn jetzt die ganze Palette an Vorwürfen einmal aufgearbeitet wird“, sagt er der F.A.Z.


          Immer auf dem Laufenden Sie haben Post! Abonnieren Sie unsere FAZ.NET-Newsletter und wir liefern die wichtigsten Nachrichten direkt in Ihre Mailbox. Es ist ein Fehler aufgetreten. Bitte versuchen Sie es erneut.
          Vielen Dank für Ihr Interesse an den F.A.Z.-Newslettern. Sie erhalten in wenigen Minuten eine E-Mail, um Ihre Newsletterbestellung zu bestätigen.