Thanks to Edward Snowden the veil has been lifted, if only slightly, from the latest incarnation of an ancient dream: complete power through total omniscience. From An, the great sky god of the Sumerians, to the heights of Mount Olympus, to the grandiose pretensions of the Benthams’ late eighteenth century Panopticon, to the pinnacle of Mount Doom, to J.Edgar Hoover’s FBI, to the secret precincts of the NSA, this dream of power through perfect vision has been the siren song of all humanity. As Tolkien penned in the early 1950’s, “One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them…”
German readers know better than most that however close this dream has come to reality, it has not long prevailed. It is up to us, all of us, to ensure that it does not prevail now. Our responses must summon political solutions that insist on democratic oversight of surveillance procedures, commercial solutions in which we reject companies that externalize responsibility for the consequences of their activities, and individuals who are willing to make a stand on what is right and what is wrong.
The Information Panopticon
That the dream is old and runs deep reminds us that it is not a product of any technology, and certainly not of computers or the Internet. Rather, it is a human constant that hovers in the shadows waiting to pounce on opportunities as they arise, century after century. I spent ten years, from 1978 to 1988, studying the computerization of the workplace, the basis for my first book, In the Age of the Smart Machine. It became clear to me back then that information technology was the next vehicle for the dream. I had read about the misadventures of naval engineer Samuel Bentham in Russia and his invention of the panopticon. Left in charge of Prince Potemkin’s estate in the southern territory recently annexed from the medieval Grand Duchy of Lithuania, Bentham faced the task of improving the efficiency of factories manned by a labor force of unwilling serfs and fragmented by a dozen languages of conquered peoples. His solution was a polygonal centripetal design with a central viewing hub that allowed a few managers to oversee a large workforce, while themselves remaining invisible.
A visit from his brother, the philosopher and social reformer Jeremy, ignited a larger ambition. Jeremy saw in his brother’s design a way to extract discipline and order from large populations of the unwilling in prisons, madhouses, factories, hospitals, schools, and poor houses. Bentham reasoned that ideal behavior could be achieved by perpetual inspection, and the panopticon offered the same result at a far lower cost: The continuous and unverifiable promise of inspection imposed a state of uncertainty upon the watched. He wrote, “ The next best thing” to being perpetually observed was that a person “at every instant, seeing reasons to believe as much, and not being able to satisfy himself to the contrary, should conceive himself to be so.”
The Daily Routine of Surveillance
By the mid-1980’s the factories and offices I studied were deeply imbued with computerized systems that had been designed to increase efficiency, improve factory process control, enhance communication, or streamline the administration of work flows. Gradually, I observed each one fall prey to the old dream, as supervisors, managers, and executives turned to the new flow of information to inspect and discipline individual behavior. Like Foucault, I saw the imaginative power of the panopticon at work, a wordless power that reached into the mind and body of each individual, preempting and shaping behavior before it was even a thought. One worker told me, “we know there is something that will tell on us exactly...so we hustle more,” while his managers fantasized about wall-sized screens that provided moment by moment detail on every facet of their operations, “ I can hit the buttons and see all the data I want.”
I had skipped centuries from the Benthams’ dark enthusiasms and stumbled into a new incarnation of the dream that I called the „information panopticon”, where each day breeds a deeper sense of anticipatory conformity circumscribing behavior to evade detection in ways that are so subtle, we eventually drop them from awareness. The challenge for the overseers had shifted from the architecture of buildings or the tedious labor of administrative records to that of information systems designed to produce automated, continuous, frictionless, perfect accounts available for zoom-in detail and zoom-out patterns any time, all the time, everywhere.
It was around then that I first articulated Zuboff’s three laws: First, that everything that can be automated will be automated. Second, that everything that can be informated will be informated. And most important to us now, the third law: In the absence of countervailing restrictions and sanctions, every digital application that can be used for surveillance and control will be used for surveillance and control, irrespective of its originating intention. In the decades that followed, American workers, and many others around the world, learned the truth of this formulation as workplace surveillance and its consequences became a feature of their daily routines.
A Different Wind
For a while it seemed that the Internet was borne to us on a different wind. It was personal, as many forged their Internet connections outside of the hierarchical spaces of the workplace. With the Internet came the ultimate, so far, individualizing tools and resources: my email address, my smartphone, my desktop, laptop, pad, my bookmarks...Outside of work we were free to express ourselves, search, learn, and connect at will. Our hunger, our questions, and our zeal summoned all sorts of new capabilities into existence: Google searches, Facebook pages, YouTube videos, iTunes, networks of friends, strangers, colleagues, all reaching out beyond the old institutional boundaries in a kind of exultation of hunting and gathering and sharing information for every purpose or none at all.
There was, and I believe there still is, evidence of a wholly new economic and social logic at work that I call „distributed capitalism.” It recognizes the user as the real source of new economic value. It is on our side. It bypasses the old legacy systems to bring all kinds of assets –– music, college courses, books, teachers, health information, social connections, guitar instruction, Chinese subway maps ––– directly to individuals at prices we can afford, and it enables us to configure those assets as we chose, anywhere, anytime. This is the promise at the heart of the iPhone, Google, Facebook, and thousands of other companies, websites, and applications.
Zuboff’s Third Law and the Lords of Silicon Valley
Shortly after the Guardian first disclosed NSA documents obtained from Edward Snowden, journalist David Kirkpatrick, a Silicon Valley luminary and author of The Facebook Effect, published a piece on LinkedIn.com, “Did Obama Just Destroy the U.S. Internet Industry?” Kirkpatrick was vehement that the success of America’s iconic Internet companies –– Google, Yahoo, Facebook, Microsoft, Skype, Apple, and YouTube –– depended on the extraordinary value they created for their users “by fostering an openness and landscape for free expression and dialogue that is unprecedented.” Now, he claimed, that historic success was threatened by forced secret participation in the NSA’s PRISM program.
But Kirkpatrick was dead wrong; the wind from Silicon Valley had shifted long before we learned of “Prism”or „Boundless Informant,” and with it our attitudes had shifted too. While the Lords of Silicon Valley begged for public sympathy pleading coercion, and most of us sat stupefied, trying to reckon with the implications of the Guardian’s revelations, it was easy to overlook some big facts. Our trust in these companies had already been diminished or destroyed.
This erosion of trust is material, and it was accomplished by the Lords themselves, without any assistance from the NSA. They had debased their most essential value — the sanctity of the user experience, the notion that they were, at bottom, on our side. We had thought that because we owned our devices, we also owned the content that we produced with them. But when Google, Facebook, and many of the other companies needed more profits, the easiest way to make money was to sell our data on to advertisers and retailers who could track and target us to increase their sales of diapers or lawnmowers or diet pills. We owned the devices, but they owned the servers. They won.
A System Mistrusted
By 2010 Google was secretly sweeping up personal data from our computers as they mapped the streets. In 2012 Facebook executives announced at their New York Marketing Conference that “brands are people too.” They touted enhanced Facebook pages „just like your other friends.” More troubling, the company began allowing marketers to target ads at users based on the email address and phone number listed on their profiles or their surfing habits on other sites. They were also enabled to follow users outside the Facebook network, according to the Wall Street Journal.
The new digital companies first became rich by putting their users at the center of the universe. But rather than the difficult creative work entailed in building on this new approach to capitalism, they capitulated to the easier money of the old adversarial model. No longer exalted end users, we were demoted to data entry clerks, delivering content for them to hawk. We work for them in much the same way that we now work for the airlines –– analyzing flight times, inputting reservations, checking ourselves in, printing our boarding passes...all unpaid labor. We are the natural energy source that keeps many of the digital companies growing and profitable, like running water by a mill wheel.
Our hunger for information, connection, and convenience is so great, that most of us decided to live with the new digital quid-pro-quo, at least until a better option comes along. But few are happy about it. In a 2012 Harris Poll, only 8% of Americans considered social media companies to be honest and trustworthy, lumping that industry with only four other long-standing pariahs that garnered less than 10%: tobacco, oil, managed care, and telecoms. Polls suggest that Europeans tend to be even less trusting of the Internet and social media than Americans.
Germans are the most skeptical, with the strongest privacy protections in the EU. While Facebook is growing in many countries like Indonesia or Brazil, it has peaked in others. A recent Pew Internet report shows that most American teens are leaving Facebook for a variety of other sites that offer a more private experience. The Lords of Silicon Valley succumbed to the old dream and trashed their birthright. The NSA only amplified our sense of their corruption.
The Failure of the Big Companies
With the NSA revelations, the information panopticon swells in scale and scope, invisible, ubiquitous, inscrutable. We are now the unwitting source of its food supply too, dutifully contributing our bits to its gargantuan data appetite, enslaved to our own secret inspection. Even if we concede there are legitimate security reasons for their data mining, the Guardian’s recent reporting suggests that the NSA now functions without democratic oversight´or protection. This is intolerable.
We know the technology oligarchs ––Google, Facebook, Yahoo, Apple, Microsoft –– complied with NSA requests for user data. How compliant were they? Yahoo made its case in a secret court that the broad requests were unconstitutional, and they lost. This presumably discouraged other companies from doing the same. There remains a great deal to be learned about precisely what occurred, with more news breaking daily. Now we discover that, according to the New York Times, Facebook put together teams to enable more complete cooperation with the NSA, and that its security chief left the company to join the NSA. „The future holds the prospect of ever greater cooperation between Silicon Valley and the N.S.A. because data storage is expected to increase at an annual compound rate of 53 percent through 2016, according to the International Data Corporation,” wrote the NYT reporters.
What does seem clear is that none of the companies chose to resist NSA demands. Nor did they choose to stand together and fight, or to inform their billions of users about practices that some of them believed were illegal. These companies own the Internet! What might their shared strength and leadership have accomplished? Instead they chose to treat the whole problem as an externality, outside their range of responsibility. For whom did their bells toll? If they are ever to regain our trust, it’s precisely that mentality that has to change. They must stand with our interests. Is there any sign on the horizon that they understand this even now?
The New Lords of the Ring
On May 16, barely a month before the NSA revelations, a conference was held at SRI in Menlo Park, California on one of the hottest new themes in the digital world, „the Internet of Everything.” The vision is stunning...every thing will be connected. The unpaid data entry labor force expands from us to our bodies and the objects that surround us: lights, thermostats, cars, coffee pots, shades, doors, appliances, but also blood pressure and composition, body temperature, organ functions, pulse rates, skin response... If the new Lords of the Ring have their way, everything from our telephones to our toasters to our tears will be reborn in the next great wave of data. Yes, each step will originate in a good cause, but how long will it take before the old dream is resurrected in this „paradise” of perfect data...how long until our tears fund a new regime of inspection and conformity? How long before the nano-drones are programmed to detect your biometric imprint?
This is the next freight train primed to carry the ancient dream and loaded with cash. In a short briefing paper Cisco’s CEO John Chambers claimed that $14.4 trillion of potential economic value is at stake for global private sector businesses over the next decade. It will present „technology, organizational, process, regulatory, cultural, and other challenges,” he wrote, „we need to collectively solve them.” What will Cisco do to solve these challenges? How do we insist that there is no innovation we will accept without its relevant democratic and commercial safeguards? Addressing these issues must be integral to innovation, not external to it and left for someone, somewhere, someday to tackle.
„A lot of people think they know what’s coming, but they have no clue,” says Dave Evans, Cisco’s chief futurist. „It’s the world waking up,” explained Alex Hawkinson, one of the conference speakers. Facebook’s Vice-President of mobile engineering tried looking ahead “...you have 200 sensors in your home...you then have that data stream going up...if you share it, now you get into the very complex question of...are you sharing data in a way that you have some ability to … indicate that you don’t want it to be around anymore? But of course, once you’ve shared the data, that becomes a pretty complicated question.”
The real star of the event was computer scientist Gordon Bell, described as a legendary great man of technology who had invented or participated in much of the Web’s development. Bell is a pioneer in the „Quantified Self” movement, and a researcher at Microsoft’s Silicon Valley Laboratory. He voiced pessimism about the pace of change, complaining that this vital new phase of Internet development was encountering...friction. But how? “You talk about we want to get rid of friction. Where is friction? People. People do not want to be disrupted...that’s really the thing that’s going to limit us...That’s what bothers me.”
The Key to the Master Lock
I say, do not despair. The indifference of the Lords of Silicon Valley is not a harbinger of the end times, but rather a wake up call to remind us that we must undertake the work of every age. Do not forget that we summoned the Internet into our lives with our questions and our needs. But there is more to be done –– a new world to be forged. It too can be summoned with our effort. All that is digital can have a profound role to play in humanizing life on earth. Our job is not to labor for our over-seers, but to invent new ways for them to work on our behalf so that we all might flourish.
Here is the key to the master lock: friction. It may be an irritant to the companies driving the next escalation of metadata, but it is also the future of democratic aspirations and commercial renewal. This friction is a new era of democratic expression in legislation, oversight, and regulation that enshrines our freedoms in terms relevant to our new age: transparency, voice, informed choice, respect for the individual. Friction must be as vigilant and steadfast as the power of the old dream. It is our insistence on a new commercial model that refuses to externalize our well being, our freedom, our privacy, and our rights to live our lives and manage our data as we choose. It is a demand for companies to take responsibility and accept accountability to end users as the ultimate source of value and wealth. Finally, friction is you and me. It is our willingness to exercise judgement, to say what is right and was is wrong even when we are at odds with power and opinion. That 8% of Americans who trust social media is very good news. It means there are 92% for whom, despite years of exposure to the information panopticon, routine violations of personal sovereignty outside the workplace have not been normalized. It means that the new Lords can not hold sway if we all stand up and say “no.”