fuzzing, back-doors, or rootkits - and to develop new, more intricate ways to steal a machine’s mind. This race for mastery is a sprint, one that heads ever deeper into the depths of a vulnerable system. The closer you are to the very core of a computer program or a network, the more control you have. Real mastery of the heart of a system would be like having a spy win the Presidency, turning the whole US government into a “Weird Machine”. That prize of immediate, high-level and totally “trusted” access remains the Warez Dude gold standard. 18 The most dangerous - and therefore the most alluringly valuable — of these sorts of attacks, are known as “zero-day” exploits. The danger they represent only becomes apparent at some awful instant, “Day Zero”, when they are revealed to have been running wild inside some hapless network or machine. That first moment in the knowledge of the bug is like day zero in a cancer diagnosis, and it begins an immediate race to find and deliver a cure. Such completely unknown, secret vulnerabilities are fissures in the walls of computers that their manufacturers, system engineers, and security experts usually don’t realize are there. The dream of hackers and spies and greedy Warez Dudes is a version of this trick called an “Advanced Persistent Threat” - hidden, back-door access to a machine that endures even for years, through upgrades and security checks and system cleanups, all the while forcing the now-weird computer to do things its user won’t even be aware of: Send a copy of every keystroke to another machine, for instance, or serve as a robotic launching pad for attacks on other machines. All while acting like a perfectly normal machine. The best of the zero day exploits are based not on the idea of sneaking malicious software onto machines so much as on taking existing, trusted code and finding tiny holes that can be blasted into giant, insecure data tunnels. These tools for manipulation are digital opiates, in a sense, for connected nervous systems. Such attacks often rely on errors already knit into computer systems, or innocent seeming features that can be made dangerous. Every computer and software designer knows their systems are vulnerable. Mathematicians have proven you can never be absolutely sure a connected machine is safe. A mobile phone, for instance, contains nearly 30 million lines of code. The systems that run massive cloud computing basins like those at Google or Amazon are even larger, updated every day, and have to cope with unprecedented flows of data at very high speeds. Even expert programmers will leave four or five errors in every million lines of code.139 138 That prize: Sergey Bratus, Julian Bangert, Alexandar Gabrovsky, Anna Shubina, Daniel Bilar, and Michael E. Locasto, “Composition Patterns of Hacking”, Proceedings of Cyberpatterns (2012) 139 Even expert progammers: For a good explanation of how a machine’s code can be turned against itself, see Sergey Bratus, Michael E. Locasto, Meredith |. Patterson, Len Sassaman, and Anna Shubina, “Exploit Programming: From Buffer Overflows to ‘Weird Machines’ and Theory of Computation.” ;login December 2011, p. 13-21. The piece is co-authored by and in honor of Len Sassman, one of the leading thinkers of the Language Security or LangSec movement who died in 2011. Many of his talks, still available online, reflect an unusually powerful mix of philosophical and 100 HOUSE_OVERSIGHT_018332

Software and hardware manufacturers usually struggle to keep such exploits secret until they can deliver a fix, but this doesn’t always work. Secrets get out. And, anyhow, even once a patch is developed, it can take weeks or months before it’s widely installed. It’s not uncommon, therefore, that within hours of the announcement of a newly found zero day, attacks using that method explode around the net. Thousands of hackers try to take advantage of the vulnerability, to kick at the defensive corners of systems while they are down for repair or restart - or simply left vulnerable by slower-witted system administrators who don’t yet know that it is now open hunting season on a particular bit of code. Heartbleed, a “zero day” that permitted hackers to slip into your computer via holes in website and browser security, was disclosed to the world on April 7, 2014 - more than two years after it had apparently been put in place because of a programming error. Accidentally? By an overworked engineer? Deliberately? By some state security agency? In any event, two days after it was announced and long before it had been fully patched, attacks using the method grew from a few dozen per hour to millions as hackers tried to suck data from unsecured networks.1*° The exponential power of a connected system is as apparent in sickness as in health. 4, In recent years, hacking has moved deeper still, beyond the level of software and USB drives and into the very atomic level of computers, the places where the electrons that make up bits and bytes float. The technical elegance of these micro- level hacks has been, often, breathtaking - exploits that look like Wagnerian operas compared to the Cap’n Crunch’s thin, reedy weird-machine whistle. As companies like Intel and AMD began packing more memory cells on silicon wafers, for example, they noticed magnetic interference flowing across the surface of their chips like waves. Electrical signals, recall, have a magnetic element, so more tiny digital cells, closer together, is like a bow] of interacting magnets. Physics would have predicted such a result. In 2014 security researchers Mark Seaborn and Thomas Dullien, who worked at Google, discovered that they could use the magnetic vibrations on two parallel rows of memory chips to flip the electrical state of a third row - sort of like using a magnet under a table to move a paperclip around — in a way that the system might never notice!*!. This permitted them to reach “off limits”, super secure areas of the machine’s memory where they could do what they wanted. They called the break, “Rowhammer’” and it represented an ideal and essentially unfixable hole that affected nearly every small chipset made for a half a decade. They published the technical consideration about modern computing systems. See also Rebecca Shapiro, Sergey Bratus and Sean W. Smith, ““Weird Machines” in ELF: A Spotlight on the Underappreciated Metadata”, paper published online by Bratus. 140 In any event: Leyla Bilge, Tudor Dumitras, “Before We Knew It: An Empirical Study of Zero-Day Attacks In the Real World,“ Paper presented at ACM CCS ’12, Oct 16-18, 2012, p 10 41 Jn 2014: Mark Seaborn and Thomas Dullien, “Exploiting the DRAM rowhammer bug to gain kernel privileges”, March 18. 2015, Google Project Zero blog 101 HOUSE_OVERSIGHT_018333

result immediately as a warning to possibly affected victims, but the exploit hummed at such a basic level of the system that it proved impossible to fully patch. It was like trying to patch physics. Connection changes the nature of an object. This has a particular impact on the security of our systems, and by extension, our own safety. Computer researcher Nathaniel Husted has described a world of “emergent vulnerabilities” - wormholes in software and hardware, communications or finance, that pop up in the connected universe unbidden and unplanned. “The fundamental aspect of emergent vulnerabilities and attacks,” Husted writes, “are that they appear benign until certain criticality conditions are met, at which point they become malignant.” 142 These risks we don’t want sit right alongside all the things we do want from connectivity. In fact - and this is Hulsted’s point — they are the things we want from connectivity, perverted into danger. Paul Baran would have been impressed to see just how right he was, how connections between us now are like irresistible gravity waves - and how gravity always wins. In 2015, for instance, Israeli security developed an astonishing hack that proved that nearly spiritual claim that all objects were linked by connection- and demonstrated the way that slippery, hungry attacks can breach even the safest- looking arrangements. “It has been assumed that the physical separation of computers provides a reliable level of security,” Mordechai Guri and his team wrote in a paper describing how they had used one isolated machine to infect another. Physical separation is, in fact, one of the cardinal rules of safe computing, a kind of lemma to join Robert Morris Sr,’s “don’t connect” rule of network safety: Two machines, unconnected by a network, should not be able to affect each other. Imagine I put one kid with a flu in one classroom and a schoolmate of his in another building. The second kid should remain healthy. The Tel Aviv research team wanted to challenge this. They first placed two computers side by side on a desk, unconnected to each other by any wire or network. One machine was connected to the Internet. The other was completely isolated - it was “air gapped”, like the healthy kid in the distant building, a sort of digital “boy in the bubble,” in touch with only the air around it. Then, the researchers began their Houdini trick: Look! Watch us corrupt this completely unconnected machine! Running a set of programs on the network-connected machine, the Israeli team was able to warm the processor board of that computer like a revving car engine, eventually making it hot enough that the temperature changes were detected by sensors inside the secure, allegedly impregnable “boy in the bubble” machine sitting nearby. The heat wave triggered a fan system inside the clean machine, which in turn activated a piece of pre-installed malware that let the hot machine pwn the bubble machine through temperature variations. In a video demonstration of the exploit, you can watch the infecting machine glow ever hotter, issuing “thermal pings” as it sweats and then infects its 142 “The fundamental aspect”: Nathaniel Husted “Analysis techniques for exploring emergent vulnerabilities and attacks on mobile devices” PhD. Thesis (available online from Indiana University, 2013) p.v 102 HOUSE_OVERSIGHT_018334

safe, “unconnected” neighbor.'*? The heat transfer had a simple message: Nothing is safe. Why put such effort, worthy of the deepest physics problems, into the challenge of sneaking into a cellphone undetected? Well, for Seaborn and Dullien, the drive was part of a “discover and publish” effort to keep the overall system clean. It is better to hack, discover and patch than to be hacked, and remain undiscovered. But these “good guy” engineers are racing against different, equivalently sophisticated, less- decently inspired teams. The development and sale of zero-day bugs is, after all, a business. Modern versions of Cap’n Crunch whistles crack access to some of the most essential financial, political and security data stores on the planet. As the power and value of hacking targets has increased, so has the price of the exploits. Public “zero day markets” sponsored by companies like Google and Microsoft pay hundreds of thousands of dollars to researchers who discover holes in their systems. “Better to find them ourselves,” the thinking goes. Though that does not always make the embarrassment less acute when holes are spotted. At one of the most carefully watched public hacking competitions in early 2015, for instance, a skinny, smiling South Korean named Jung Hoon Lee took home $225,000 in prize money by pwning a series of some of the most important, common programs on the planet, Apple’s web browser Safari and Google’s Chrome among them. These systems had been constructed at the cost of hundreds of millions of dollars. They‘d been assembled under the gaze of some of the best PhD-led computer scientists in the world. Jung Hoon Lee’s exploits ran through their complete defenses in less than a minute. 144 As good and fast as someone like Lee might be, he’s nothing compared to what the best hackers do. They don’t work in public or compete in hotel ballrooms. They don’t brag. And they develop ideas that make $225,000 look like a bargain. These successors to the Warez Dudes work for cybercriminal billionaires, for intelligence agencies, and even (often) just for themselves. They help find and deploy the sorts of really deep system exploits that enable brazen cyber thefts of millions of pieces of personal data or attacks like the Stuxnet virus, which caused thousands of Iranian nuclear centrifuges to vibrate themselves apart. And they do still more: Most of the attacks we've talked about so far occur in installed, running boxes. But the companies that make those boxes oversee a whole, vulnerable process of building and testing and designing and installing them. And it’s here, with billion dollar budgets at work, that some exploit teams make and leave vulnerabilities that they can later, ruthlessly exploit. Every step of that gestation — from sneaking secrets into early code bases to intercepting and rewiring routers as they ship overseas — is now an opportunity for secret control. Or for unanticipated risk, for “emergent 143 In a video: For a description of this exploit see Mordechai Guri, Matan Monitz, Yisroel Mirski, Yuval Elovici, “BitWhisper: Covert Signaling Channel between Air- Gapped Computers using Thermal Manipulations” (2015) available on arXiv:1503.07919 [cs.CR] 144 Jung Hoon Lee’s exploits: “Chrome, Firefox, Explorer, Safari Were All hacked at Pwn2Own Contest”, PC World via IDG News Service Mar 20, 2015 103 HOUSE_OVERSIGHT_018335

misbehavior” that defies simple analysis. You can’t predict where you might be attacked by merely looking at the possible holes in each piece. Rather, it’s the whole system that breeds risks. It acts in ways that the designer could not have predicted in advance. “Clearly the system itself is misbehaving,” the researcher Jeffrey Mogul writes of his study of various cases where networks are cracked in this fashion. “However, none of the components have failed per-se.”!45 The complexity of the systems themselves has been, not surprisingly, mimicked in the design of hacking attacks. What was once done by a single Warez Dude is now handled with division of labor, technical specialization and intensive pre-attack research. Every innovation in “righteous malware” is quickly copied and used in dangerous attack tools. The clever modular design of Stuxnet, for instance, was studied by criminals and was found years later still echoing in weapons aimed at banks, credit card companies and health insurance firms. “We are not experts in military history, doctrine, or philosophy,” cybersecurity researchers Stephen Cobb and Andrew Lee have written, “so we are unaware of the correct word for the following category of weapons: the ones you deliver to your enemies in re-usable form.” Cyberattack systems can be dangerous not least because they boomerang. They are delivered intact, primed for re-use to enemies who may choose to bounce them back at your banks, hospitals and electrical grids. “Righteous malware is unique,” Cobb and Lee conclude. “You are giving away your weapons, tactics and designs simply by using them.” 146 It’s not only American services hunting and using such backdoor keys and battering rams, of course; not only the NSA that sees its viruses retooled and reused. Computer security researchers describe opening up the laptops of unwary business travelers and finding the machines blasted inside by malware and other technical cancers, carefully planted by a half-dozen intelligence agencies and criminal organizations. It’s like discovering a closet full of spies in your house, each being careful not to step on the other’s toes as they watch and listen to your life. Why is my computer so slow, a government official in a Eurasian capital might ask. It is because it has been simultaneously pwned by Americans, Russians, Israelis, Chinese, and maybe a local Mafioso or two - and their code is not running smoothly. A couple of years ago I had a naive moment when | thought, perhaps, it would be possible and in everyone’s interest to go back to those simpler, innocent Hacktic days, when information about vulnerabilities was widely shared and easily discussed - and holes were quickly patched as a result. | was thinking about the problem of cybertension between the US and China and suggested applying an 145 “However”: Jeffrey C. Mogul, “Emergent (Mis)behavior vs. Complex Software Systems”, HP Labs Research Papers, 2006, HPL-2006-2 146 “We are not experts”: Stephen Cobb and Andrew Lee, “Malware is Called Malicious for a Reason: The Risks of Weaponizing Code” in P. Brangetto, M. Maybaum, J. Stinissen eds., 6 Annual Conference on Cyber Conflict (NATO Publications, 2014) 71-82 104 HOUSE_OVERSIGHT_018336

important idea of the scientist Dan Geer to the ever more fraught relations.14” Perhaps the US and China could work together to buy up and then publish all the zero days as they emerged, | thought.!48 Instead of the dangerous code falling into the hands of cybercriminals, Mafiosi and terrorists, the two countries could lead an effort to jointly buy any new exploit for five times what anyone else would pay - and then immediately publish what they had bought and the necessary patch. This would make the network safer for everyone. I should have known better. The US and China and other nations were (and are) buying world-class zero days. But they were never going to publish them. They were buying them fo use. Sometimes against each other. Sometimes, unnervingly, against their own citizens. And they needed to keep on buying and developing and hiding such tools on an exhausting, never-ending security treadmill because, unlike traditional weapons, which could be stockpiled to use whenever they were needed in the future, the holes of the most valuable zero-days might be patched at any moment, making a once devastating bit of malware instantly useless. And, as the systems matured and accelerated, this meant that they had to run ever faster to keep up. Which further reduced their incentive to “buy and publish” what they did have. Little surprise then that all around the IT universe in recent years, the incidence of reporting dangerous bugs has been declining, even as we know the number of known security holes is certainly growing. 149 Grab the five nearest electronic devices near you and you can be pretty sure each is vulnerable; which of course means you are vulnerable too. Not merely to the loss of your secrets, but also to perversion and control. This is the cold truth: that old hacker ethos, the one spread out so warmly on Amsterdam grass 20 years ago, the be liberal in what you accept frontier society instinct, is dead. Weird machines and normal machines, weird networks and normal ones, people made weird by technological manipulation and those who have note - they will all, inevitably, live side by side. The more essential machines become to our connected lives, the more avidly weird and hacked the networks will become. The incentive to whistle up control of the systems, and control of those of enmeshed in them, is the only thing that seems to grow faster than the system itself. Ds This vulnerability of connected systems is an important mile marker in our route towards understanding some principles of power in a connected age. “Read over and over again the campaigns of Alexander, Hannibal, Caesar, Gustavus, Turenne, Eugene and Frederick,” Napoleon wrote once. “Make them your models. This is the 147 | was thinking: Dan Geer, “Cybersecurity as Realpolitik”, speech delivered at BlackHat, August 6,2014 148 Perhaps: Joshua Cooper Ramo “Talking Cyberthreat with China”, International Herald Tribune, July 10, 2013 149 Little surprise: Linder Gayten Back to Basics p. 58 105 HOUSE_OVERSIGHT_018337

only way to become a great general and master the secrets of war.”!°° I sometimes fee] the same reading over stories of zero-day attacks, clever hacks like “rowhammer’” or the Tel Aviv heat hack. You can distill from each tale of a broken, once-secure systems an essential principle: The hackers rush always, relentlessly at the central core of a system. They aim to make it weird, to manipulate it madly from the inside out. Network power doesn’t merely come from that 10 million device-per- day spread of global connectivity, after all, it also comes from incredible concentration of power inside certain systems we all rely on: Chips, data bases, centralized and gatekept platforms. Control of such hubs and roots of our world can influence everything; little wonder they are such an appealing target. “The conventional belief that all nuclear systems are ‘air gapped’ is a myth,” the Russian security researcher Eugene Kaspersky has warned. The result: “There are three types of people: Scared to death. Opportunists. Don’t care.”151 This sense that the systems are so vulnerable if you can get to their hearts is what lures hackers ever deeper, into the code kernels where the most basic instructions are decided. That they can often make machines weird by using the device’s own code against them, like some sort of autoimmune disease, is only a marker of the particular perversity of the problem here. Security researchers call such holes “vulnerabilities” in a system, but of course they are much more than weak spots. They are potentially fatal. In a way, the hot rush to touch and tickle and maliciously use these already waiting cancers reveals to us the essential Seventh Sense secret of the Warez Dudes: Connection makes an object vulnerable, yes; but it can also reveal the possibility of total control, of the fundamental root mastery of a connected system. Such a hole, when it is exposed by connection and then corruption, can be complete in the scope of its damage, devastating. Lord Acton’s famous line that “Absolute power corrupts absolutely,” twists in this age to something like “Absolute access corrupts absolutely.” Connection makes total exploitation, total control, possible. Every new generation of connected technologies is breeding essential black boxes, complex (not merely complicated!) containers filled with algorithmic levers and code tools for digital work that can be understood by only a few people, and exploited and used effectively — for good or ill - by a still smaller group. “The greater the dependence on a technology the greater the need to study and expose its inner workings,” one group of radical digital activists has argued in The Critical Engineering Manifesto. 52 They mean that as we turn our safety, freedom, and health over to a world of devices and their makers, we must know what goes on inside the very heart of such systems. It’s not merely that everything is connected now; it’s 150 “Read over and over”: Napoleon, “Maxims” from Thomas Raphael Phillip, ed., Roots of Strategy: The 5 Greatest Military Classics of All Time, (Stackpole Books 1985) p 432 151 “There are three types:” Eugene Kaspersky Talk at the Press Club in Canberra, Australia (2013) 152 “The greater the dependence”: “The Critical Engineering Manifesto,” The Critical Engineering Working Group, Berlin, October 2011-2014. Available online 106 HOUSE_OVERSIGHT_018338

also that everything is monitored. Remebered. Studied. The Warez Dudes’ drive to get ever closer to the core, to perform even that atomic level hacking, tells us something about just how much power is locked up in those central cores where this information accumulates. The 2600 hz whistle was, it seems, only the first of an endless series of battles for control of the roots and trunklines of modern power. All around us today, huge power accumulates to certain irreplaceable cores. We know this is a problem of connected age design: Giant search engines, certain algorithms, database or communications protocols overmaster us because they can gather so much data, so fast, and process it with unique fidelity. What makes a city? urban scholars often ask. We might wonder: What makes a platform for network power? The answer to both questions is the same: Density.15? If the first cities of Aztecs or Mesopotamiaman civilization differed from early tribal clusters because of their density, the same is true for our first platforms of instant connection. Facebook is denser than AOL ever was. More people, more data, thicker connections. Future platforms will be denser still. And if cities and density were once sadly unanticipated accelerants to plague, poverty and revolution, we should be aware of the risks of our own tight-clustered centers of dense connection. The security of these cores that link us to each other and our essential data - when jacked by hackers, by companies, or even by fast algorithms we don’t understand - is important not merely because of the possibility of total control a breach might represent, but because they show us the very fact of such totalizing control exists. To infect, surprise, sicken - all this is alluringly possible and dreamable for anyone with a hunger for mastery. Imagine if you knew your government could be switched instantly and invisibly to malice. (Or, to effectiveness!) Or picture a nation of connected citizens wired for flash-started nationalism and hate. Such a possibility exists on linked systems. This potential for total, weird control of the cores - and thus total control of anyone connected to them - should force us to wonder a bit. Every evil thing beats in those central nodes: The power to manipulate, to master, to destroy even. With such access I can change what you know about the world, how you vote, where your money sits, what you remember, how soon we spot (or don’t) a slipped knot in your DNA. “Just like every drinking binge ends at vodka, so every hacking session ends at kernel.org” Thomas Dullien, the mathematician and good-guy hacker who won a Pwnie in 2015 for lifetime security achievement, observed once!54. So much power in a connected system lies at its root: Kernel.org, for instance, is the reference copy for LINUX computer code that powers most digital machines on earth, sort of like the original DNA of the net. To manipulate Kernel.org would be to reach into the very spine of the Internet. If the aim is control, if it is to find and exploit the most fundamental of cracks in the surface of the black boxes, to get even deeper inside, 153 What makes a city?: Colin McFarlane, “The geographies of urban density: topology, politics and the city.” Progress in Human Geography (2015) p. 2 154 “Just like every drinking binge”: Halvar Flake/Thomas Dullien, “Why Johnny Can’t Tell If He Is Compromised,” speech delievered at Area41 Conference, (Zurich, June 2014) 107 HOUSE_OVERSIGHT_018339

then inevitably kernel.org or it’s equivalent in any system is the ultimate target. Hackers might start with beer (your phone), a few glasses of wine (your office email system), but what they really want is full blotto (kernel.org). Such central black boxes exist in any linked system and they represent, at once, both the greatest accomplishments of our most masterful systems designers and the point at which other, as masterful digital machinists direct their most relentless attacks. Dullien saw something else too as he considered the work of hackers. Comparing system cracking to drinking wasn’t an accidental, funny aside for him. Hacking was almost a kind of addiction. It became a chase after a bigger and bigger high, which in computing terms meant a race to compromise as many machines as possible. Rapid escalation, a loss of self-control, the need for more and more - all these are the hallmarks of the best widespread attacks, which aim to expand the “compromise boundary” until every machine has been made sick. This is why stealing source code, the original instructions that lay behind any computer program, are sucha prize for Warez Dudes. Source code is the DNA of the black box, in a sense, it can be used break into other machines to, well, steal still more source code. This looked, to Dullien, an awful lot like addiction. And it’s not just lone teen hackers looking for a dopamine jump who were chasing machines with a blind addicts urgency. It infects governments too. “Surprising realization (at least for me) after the Snowden leaks,” Dullien observed. “Hacking is so addictive that entire organizations can become addicts and show addict-style behavior.”!55 This was the NSA or GHSQ or who knew what other intelligence service, for instance, in a feral hunger for more and more and eventually all data. And the ultimate high? Imagine if you could crack the CA, the Certificate Authority that provides proof that anyone working on a network is trustworthy. CA “trustworthiness guarantees” are keys that permit access to any computer, phone and network. The whole connected world depends on CAs to know if software is safe. To control the CA would be a dream for an addicted organization, like a set of keys to the local pharmacy. No lock would really serve much ofa purpose. You could touch the beating heart of any machine on earth. The ultimate black box. We should all hope the CA, at least, always remains pristine in this dangerous world. It was hacked in the summer of 2011. 6. Dullien’s observation that “every hacking binge ends at Kernel.org” touches something far deeper than just the world of hackers. Sure, that urgent drive to get to the very core of systems is a very specific, unquenchable digital thirst. It’s where the most power is, as we've seen. There’s a rush in getting there: When brain researchers found a spike in dopamine levels of university students engaged in password hacking, that was hardly a surprise to anyone who’s ever written or used 155 "Surprising realization”: Thomas Dullien, “Offensive Work and Addiction”, keynote presentation delivered at ISACA Nordic Conference (2014) 108 HOUSE_OVERSIGHT_018340

an exploit.15¢ But the real drive to get inside is about more than an adrenaline rush. Remember that network power, the power that we're trying to figure out how to bend and shape for our own security, exists in a kind of dynamic tension. It’s like a stretched, taut fabric spread between concentrated cores and billions of connected users or devices. The logic of spreading and distributed power, the force that makes the network bigger, is driven by Baran’s principle of open design, and by our own hunger for communications and connection and cool new devices. But there is another side to this tension. In a sense, over the years, a whole set of hot, infectious pressures descended on the network of values and friendships and easy cooperation of the Hack-Tic days. “Be generous in what you receive,” had let the networks of our age grow at an incredible pace, but at the price of vulnerability, of commercial ambition, and of an eerie technological lemma that what made the systems faster and stronger might also kill them. A change in culture of the digital elite, naturally, followed. The brutal, inarguable, profitable demands of power and politics had cracked apart the unique social webs of the HacTic era. I did not like watching this sad evolution; none of us did, but anyhow it has produced the world in which our new sensibility will have to operate. The openness that we loved and craved in so many areas of life, from our minds to our markets, had now become a liability. “I remember what the Internet was like before it was being watched and there had never been anything in the history of man that is like it,” Edward Snowden once observed, nostalgic for the datascape he saw melt away during his time at the NSA. 157 I realize now that there is a whole new generation of young programmers that won't ever know that original generous ethos of a place like Hacktic, a fresh cohort of the digital age that operates at levels of technical mastery far beyond anything we might have imagined in the Citicorp Tower basement 20 years ago. They will confront endless battles to get inside and exploit and make “weird” the cores of network power. They will know and design and manage instead a world of gates, built for protection. Their instincts will be for opacity and control, not openness and generosity. Invariably this shift will affect the design of the systems this new generation builds which will, in turn, affect all of us. The black boxes and the crackers are in a dance, now, a sort of dangerous evolutionary waltz that offers a foretaste of what you and I will face as we consider the problems of attack and defense and strategy in a networked age!*@. It reminds 156 There’s a rush: Wael Khalifa, Kenneth Revett and Abdel-Badeeh Salem “In the Hacker’s Eye: The Neurophysiology of a Computer Hacker”, in Global Security, Safety and Sustainability & e-Democracy Volume 99 of the series Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, (Berlin: Springer), 112-119 157 “J remember what the Internet was like”: Snowden interview in CITIZENFOUR, Laura Poitras (2015) 158 The black boxes: Nikos Virvilis, Dimitris Gritzalis, Theodoros Apostolopoulos, “Trusted Computing vs. Advanced Persistent Threats: Can a defender win this game?” 2013 IEEE 10th International Conference on Ubiquitous Intelligence & 109 HOUSE_OVERSIGHT_018341

me of an old puzzle of Chinese history: Why were the very strongest dynasties - the Han, the Ming, the Tang - always confronted by the best organized, most deadly rebels? The answer was rooted in a violent, greedy tango of the development of each side. The better a dynasty defended its farmers or its trade or its annual rice harvest, the more the rebels from the steppe had to become strategic and unified and powerful. When it was easy to pick off single farmers, then the rebels had no need to be well organized. They could be a bit lazy. Could snatch a year’s undefended harvest in an afternoon. But the stronger the empire, the stronger the rebels had to become*5?. Amsterdam in 1994 was a bunch of lone, weak, unprepared farmers. Twenty years later we have a digital empire, and the attackers it deserves are appearing, refined and evolved by this same competitive logic. They are better organized, smart, intent on getting to the heart of the black boxes of power. And the more we defend against them? The better they will become. That our most essential systems are vulnerable to loss of control is a chilly feeling. It is a reminder of the power of the people who know how to crash and manipulate - or build and operate - parts of our world most of us barely understand. It’s like discovering someone could take over your lungs or your heart. We don’t understand those devices either, most of us. But we depend on them entirely. Some of these hackers are moved to mischief by technical beauty. Some by the giddy smashing rush of breaking in, of touching the core. Others by greed or patriotism or by secret, zealous, unlawful obsessions. What the technically best of this group share, however, is a pressing desire to get as close as possible to the kernels where inarguable and even invisible code decisions are made, where digital DNA is printed in a sense, and where a total mastery of the binary guts of the system is possible. That Cap’n Crunch thrill, the dream of whistling up control over the thick, helpless trunk of a network, that remains the dream. Remember Conway’s Law: The design and activity and control of a network redounds on, even determines the real world. If the whole network is, in a sense, filled with holes, if it contains inherently the possibility of being turned into a “weird machine” - what does that mean for the real world? The Seventh Sense insight of this chapter, the key feeling, is that all the systems we rely on, that we think we control - financial or political or digital - can all be made weird and pwned by forces we cannot see and struggle to stop. Our markets, our elections, our knowledge - all of these, dependent on linked systems themselves, can become weird. We can no longer regard them as certain and harmless. The hacker’s drive to get to the kernels is not merely an information technology problem. It’s a larger statement about networks, about the way that power and danger are still and always one and the same. The desire to hack the cores of our world is a marker of just how profoundly, even secretly influential the kernels have become - in ways we're just now beginning to understand. Bratus was right, that we don’t really understand any system until it’s been exploited, pwned. This is as true Computing and 2013 IEEE 10th International Conference on Autonomic & Trusted Computing 159 But the stronger the empire: See Turchin, p. 3 110 HOUSE_OVERSIGHT_018342

for our minds as it is for our markets. The Seventh Sense feeds a bit on the trail of hackers; it is a way to tune our protective instincts. The cracks they find and use do tell us an awful lot. But not everything. For that we need to look beyond the people attacking. What we really need is a feel for the temprament of the fresh group of innovators and investors and technologists who run and own the dark cores of power now - and who, before long, may also run and pwn each of us as a result. 111 HOUSE_OVERSIGHT_018343

Chapter Seven: The New Caste In which we meet a powerful group defined, enabled and enriched by their mastery of the Seventh Sense. 1. Looking back over several hundred years of European history, the Oxford professor David Priestland found that the movement of power might be scored by reviewing the alliances and hatreds and hopes of three distinct, interacting groups. He called them “castes”: merchants, soldiers and sages. By merchants, Priestland meant the bankers, traders and industrialists whose capital, goods and political power bent Europe’s once-feudal economy into something modern and industrial.!@° The Medici, Dutch coffee traders, Scottish cotton barons. By sages, he had in mind the churchmen and later the technocrats of various empires, the men who helped birth and then manage the problems of an Enlightened, urban social order: Locke, Bismarck, Disraeli. And by soldiers he had in mind both the great aristocratic warrior classes of Europe and upstart, genius figures like Napoleon or Wellington - men who handled martial force with the fresh, surpasssing brilliance of a paintbrush or chisel, not an instrument of mass murder. The aligned, shifting interests of these three castes, Priestland wrote, were like gears of sorts, each offering special leverage, meshing together to drive nations to great power. Mix the influence of France’s sage-bureaucrats with her artful soldiers and you get the French Imperial period. Marry the interests of Britain’s shrewd 17% Century trading bankers with her martially inclined sailors and globe-spanning Victorian dominance results. Today, of course, the merchants and soldiers and sages of our era are also at work. They sit in sovereign wealth funds, wired situation rooms and madrasas, churches and research labs. The force of America’s merchants and financiers, bolstered by Washington’s security caste, defines much of American power. No other nation could, at the moment, comprehensively replace what the country does with such breadth, intensity and speed. And now, all around the globe, we're seeing the emergence of what we might think of as a New Caste joining the merchants, soldiers and sages. This is the infotech caste. What was going on in that field in Amsterdam back in the summer of 1993 was nothing less than the birth of the first figures of a connected technological era, of a new elite. This caste | have in mind is defined by their personal proximity and fingertip feel for the linked machines that drive so much of our world. They represent a tiny fraction of our population, but operate with non-linear, massive levels of influence. This New Caste clusters, in ever tighter circles of intimacy, around the systems and networks we depend on. They are building new connected 160 Looking back: David Priestland, Merchant, Soldier, Sage: A New History of Power, (New York: Penguin, 2013) 112 HOUSE_OVERSIGHT_018344

financial systems. Fresh AI designs. Protocols and platforms for secure communication. Maybe one million people can write object-oriented code at a high level. A hundred thousand of them can shape that code into some sort of innovative data structure. A few thousand might be able to use it to access and manage a large data center. But get down to the couple of dozen who know how Google or Intel or bitcoin really work, the group who can make machines seem to think, who know and use backdoors at that atomic level of hacking - well, then you have a tight, powerful elite. They are called system designers, algorithmic traders, growth hackers or any of a dozen hazy and somewhat unnamable lines of work that fuse network mastery with economic, political or social power. If connection changes the nature of an object it also elevates, to a level of rare power and influence, those who control that connection. Through the networks and black boxes they control, the group of transcendent talents working on search algorithms, data management and machine learning touch, at any one instant, more parts of our lives than any group of elites ever has. That many of them are billionaires as a result should hardly be a surprise. This is a caste marked by constant one upsmanship, by endless and compulsive innovation, by a cold and ceaseless fear of obsolescence. They are marked, of course, by a full expression of the Seventh Sense. Their every instinct for starting, financing, growing and using new firms or technologies or data-spinning biological tools reflects complete confidence in network power. If the leading figures of the Enlightenment shared a certainty near to faith that reason would unlock nearly any puzzle of mind or politics, the New Caste shares a certainty too, that connection can produce a new, better order. This caste battles with each other. They struggle for dominance over markets and - more profoundly - for control of the lucrative systems they have built. My point in this chapter is simple. It is to introduce this new and powerful caste and see what their successes and failures might tell us about the world we now all share. Even now, however, I think we can make this sobering judgment: Whatever nation cultivates and trains legions of this New Caste best will rule the future. Think of the legendary Xerox PARC research lab, which in the 1970s produced not just some of the best early members of the New Caste - like Alan Kay or John Seeley Brown- but also a series of fundamental inventions that generated trillions of dollars of value?@!. The mouse. Laser printing. A graphical user interface. PARC was arguably the most economically significant small group in human history, an entire Renaissance packed into a couple of dozen offices. Just as the nations that produced the finest mariners once dominated global commerce and decided political questions, so countries (or really any group - even terrorists or criminals or bankers) that can breed, train and equip and deploy members of this New Caste will possess a really unusual power. Calling them a “caste” is not accidental: There’s a way in which we are all now tied to the virtual spaces they control, much as serfs were once bound to the land by feudal lords. The Italian historian Giambattista Vico, in developing his 161 Think of the legendary Xerox PARC: Chunka Mui, “The Lesson that Market Leaders are Failing to Learn from Xerox PARC”; Forbes Leadership Blog / Alan Kay 113 HOUSE_OVERSIGHT_018345

new, scientific view of history in the 18" Century, once observed approvingly, “The Egyptians reduced all preceding world time to three ages, namely, the age of the gods, the age of the heroes and the age of men,” he wrote. “During these three ages, three languages had been spoken....Namely the hieroglyphic language, the symbolic language and the vulgar language of men.” It’s hard not to feel the new age now arrives with its own baffling and incomprehensible modes of communication. A whole freshly demanded language. In all of human history only a few languages ever evolve to become honestly global in reach and influence - English now, French in the European centuries or Chinese in the Asian imperial era. But why English? Why not more decades of French? German? “Why a language becomes a global language has little to do with the number of people who speak it,” the British linguist and historian David Crystal has written. “It is much more to do with who those speakers are.” What made Latina global influential language wasn’t that millions of people spoke it, rather it was who did: The elites at the very peak of 1,000 years of European power. What Latin had, in a sense, was the ears and tongues of some of history’s most influential men.1°3 The private, technical language that connects the New Caste to their machines, to each other and to us is one of the sources of their power. Their code marks, like a trail, the path to the cores of a vast and modern power apparatus. You could, if you wanted, compare the New Caste a bit to an earlier generation of empire-deciding figures: Ocean explorers. Columbus, de Gama, Magellan. Backed by a primitive version of venture capital, the “risk finance” of trading houses, these discovery captains had a hunger to test their certain masteries - navigation, sailing, trade - against the uncertanties of geography, weather and luck. There was as much sheer nerve in these adventures as there was real knowledge. What lay five weeks’ sailing time away from Cadiz? If you were willing to endure the difficulties, to believe in what might be out there - and your own ability to handle it - then fortune awaited. “Early intercontinental travelers not infrequently had to pay for access to distant shores by enduring bitter asceticism,” the German philosopher Peter Sloterdijk has written.!@* Months at sea, risks of ocean turmoil, starvation, endless boredom - all these sacrifices marked sailing adventure. But they knew the rewards for areal mastery of the sea: Fame, riches, knowledge, adventure. Sloterdijk cites Goethe, who reflecting on the power of nautical life in 1787, defined the edgy advantage of that ocean-mapping caste in his age: They had perspective. “No one who has never seen himself surrounded on all sides by nothing but sea,” he wrote, “can have a true conception of the world and his own relation to it.” 162 “The Egyptians”: Giambattista Vico, The New Science, (Cornell Press, 1948) 69 163 What Latin had: David Crystal, English as a Global Language. (2nd ed. Cambridge, UK: Cambridge University Press, 2003), 7 also Shahar Ronen, Bruno Goncalves, Kevin Z. Hu, Alessandro Vespignani, Steven Pinker, and César A. Hidalgo. 2014. “Links That Speak: The Global Language Network and Its Association with Global Fame.” Proc Natl Acad Sci USA 111 (52) (December 15): E5616-E5622 164 “Early intercontinental travelers”: Peter Sloterdijk, In The World of Interior Capital, Polity 2013, Ch. 13 114 HOUSE_OVERSIGHT_018346

We might say the same about our New Caste, of their sense of the massive datascape on which we all sail and operate. They too head off to test their skills at coding, encryption, system design. They are almost all auto-didact - self taught as programmers or hackers or AI designers or investors. There is no established path to power for them, no route that has been marked before. No wonder the greatest figures of the era are college dropouts. This is a group that looks largely ahead: Their work demands constant upgrade. Every device is shipped imperfect. It’s not to say they are completely denuded of the Sixth Sense. That their feeling for history has been stripped. But if to be honestly in love with version 1.0 of your product and then to shred it for version 2,0 does demand a certain ahistoric distance. The New Caste explorers have a hunger for fresh worlds and landscapes. To get there they endure privations of long isolation - not so extreme as a long ship journey — but they burrow deeper and deeper into systems, construct new links and new gates, in closed rooms. “When building machine learning systems, making good decisions is a strategic skill,” the computer scientist Andrew Ng has said. “Every day you wake up and you are in some totally unique situation that no one in the planet has been in before. It’s not a fact, there’s no procedure.” 16 This is our world now. No fact, yet. No procedure. A totally unique situation that demands a fresh instinct. We can follow Goethe into our own age: No one who has not seen and felt himself surrounded on all sides by networks can honestly understand. If that motto of the last great epochal shift was Dare to Know, it was certainly enacted by the sea captains. Discovery and risk were nothing but a nervy kind of scientific experiment, where the laboratory was the Earth’s surface and the tools of calculation were ships. And in our age? If we're right that our historic mandate is Dare to Connect, well the New Caste is certainly doing that, furiously. They are working away at much that is hidden in our world, just like those old explorers. The billions of phones or sensors or Al-rooted sensor engines spilling out around our world are stripping away layer after layer of opacity. Pictures of far away places, traffic lessons learned from movement patterns recorded by GPS- enabled phones, medical treatments refined by constant measurement - these are all knotted, one after another, into the fishnet. If the ocean voyages of that older explorer caste mapped our world, the voyages of the New Caste are as much inward as out. They move inside the world of connected systems. Remember: Network power is defined by dynamic tension - the pull between center and periphery. The same dynamism is at work with the New Caste. While they are, firmly, in the business of going out and opening our eyes to the world with wider, faster connection, they are also draping huge, essential parts of our lives with an impenetrable fustian: The essential algorithms of search engines, the code of machine learning tools, the design of micro-targetted political influence campaign. The sources of New Caste power are wrapped inside layers of complex computer code, machine learning and data security protocols that few people can 165 “When building”: Andrew Ng, speech at GPU Technology Conference (March, 2015) 115 HOUSE_OVERSIGHT_018347

completely understand. The French philosopher Bruno Latour, the father of “Actor Network Theory” has called this process “Black Boxing”. The better your phone works, the less you notice it. The more precisely some machine feeds you your news, the less you wonder what might I be missing? “Scientific and technical work,” he says, “is made invisible by its own success.”166 The operating system and network protocols of your tablet device are opaque to you now in a way they never would have been two decades ago (when even the casual user computer had to type to a C: prompt or wildly rage at a crashed “Blue Screen of Death” from time to time). But, in fact, the system is incalculably more complex. “Each of the parts inside a black box,” Latour remind us, “is a black box full of parts.” And it is in the winding and linking of all these pieces that action in a connected world is made possible. “It is by mistake or unfairness that our headlines read ‘Man Flies’”, Latour says. “B-52s do not fly,” he writes. “The US Air Force flies.” Every plane that ever makes it into the air does so because of the clicking coordination of thousands of linked, black boxed systems. Your stock portfolio or your computer or your bio-sensored heart is not a lone object; it’s a feature of a connected landscape. We're surrounded now, connected to, essential black boxes we’ve no way of understanding and whose development and operation we've left to the New Caste. Look around you, how many screens do you see? Each is a billboard: New Caste at Work. It’s not only the hardware in our lives that | mean, but the bits of knitted programming that decide how we search, when we communicate, and if we can exchange information or money. The virtual and the real are in constant contact and it’s the New Caste that does the stapling. In fact, one of the magic tricks of power in the connected age is an ability to flop easily back and forth between network and reality. It suggests other dangers too. As legendary machine systems designer Leslie Lamport warns: Computer scientists collectively suffer from the confusion of language with reality.‘°7Anyone who’s ever written a computer program knows this sense: You write some code. You compile and run the program to see what happens. You go back and work on the code some more to refine what you’ve done. You run it again. You touch the virtual; the real reacts. This seems in a way like the most trivial thing, the writing of a computer program or an AI bot or a trading order, but in fact 166 The French philosopher: Bruno Latour, “On Technical Mediation — Philosophy, Sociology, Geneaology” in Common Knowledge, Fall 1994 Vol 3 No 2. p. 23 167 It suggests other dangers: Leslie Lamport, “Computer Science and State Machines”, Contribution to a Festschrift honoring Willem-Paul de Roever on his retirement” (Redmond: Microsoft Research, 2008). In its entirety it runs: “Computer scientists collectively suffer from what I call the Whorfian syndrome—the confusion of language with reality. Since these devices are described in different languages, they must all be different. In fact, they are all naturally described as state machines.” The Whorfian problem is a linguistics observation about the way in which our thinking is limited by whatever language we have to describe what we see or contemplate. 116 HOUSE_OVERSIGHT_018348

something incredible is underway in this easy movement from machine to reality.1° The New Caste takes these moves, this easy slip from their keyboards and programs to our lives, for granted. They adjust code and networks and formulas; they watch the effects on us. They do it again. The idea that such a move is natural, comfortable even, reveals a new and important temprament. It draws a line betweent the people writing the code and those who are snapped about in the world they are coding. Do you know who decided what you see when you search? Do you understand what the data on your phone reveals about you? Who will snip at and work on your DNA? Your children? Are you trading stocks against some invisible high-velocity connected master who will always be one profitable nanosecond ahead of you? In this sense, network power involves something very much like the intentional creation of concealment. Your Internet search results, for instance, contain a sharp tension. Yes, data from all over the planet, from all of history sits rather amazingly in front of you. But that bit of computer code deciding what you see is engaged ina kind of digital book burning: It’s making whole sections of knowledge invisible even as it is unearthing an ever more precise answer for whatever question you have. What don’t you see? - is a question that hints not only at what is left out of your search horizon, but generally at the way in which connected systems establish necessary gates. Part of a Seventh Sense, then, is the ability not merely to look at the virtual world and know how it becomes insidiously real, but also to feel that all the connected points of the real world - markets, weapons, social movements - must be pulled upon by code and links and networks. “Any technology depended upon,” as The Critical Engineering Manifesto, says is “both a challenge and a threat.”@? Human experience is, we know, unboxable, uncontainable - our joy, hopes, sense of freedom, these all defy boxing. Yet here, all around us, are containers that affect our every choice. Who knows what happens inside all the difficult boxes? The creeping, essential opacity of power now reveals a twisted puzzle, a really fresh aspect of this New Caste and the revolution they are making: As much as they are in the business of making knowledge widely and instantly available, they are also madly black boxing our world. This breeds a sly, unintended (I think) tension with Kant’s Enlightenment admonition to “Dare to Know.” Would you like to Dare to know why your computer is secure? How your genetic information will be studied and used? How encryption works? Mostly the answer to is: You can’t know. It’s too complex - and, anyhow, if we told you it would make the whole system less secure. There is nothing disingenuous here: You likely wouldn’t understand. It is too complex. You'd be lost at the first turn into strange technical language, where simple words like “object” or “edge” have specific, essential, different meanings. And telling you would, in fact, expose you and everyone else to all sorts of risks. It’s as if we’ve returned to that famous debate of millennia past, the one lingering between Athens and Jerusalem: Could the world be known and atomized and understood as the Greeks would have it? Or was mystery, inscrutability and opacity the nature of truth, 168 This seems in a way: See, for instance, Bret Victor in his speech, “Inventing on Principle” at CUSEC 2012 Turing Complete Conference available online. 169 “ Any technology depended upon”: The Critical Engineering Manifesto, as above. 117 HOUSE_OVERSIGHT_018349

apprehensible, the Rabbis said, only by its movements. Are we back at the first chapter of Genesis and its absolute prohibition against eating from the tree of knowledge. Or, from the Talmud, “For him who reflects about four things - what is above, what is below, what is before and what is behind - it would be better not to have come into the world.”!7° We want to reflect about what goes on inside the machines. Can we? Should we? How does Dare to know face off against these impenetrable systems. It is little surprise that places like Silicon Valley often leave a visitor with the feeling of a town where work is done in rooms within rooms within rooms. To drive along the dulled, anodyne asphalt stretch of road that runs in front of Sand Hill Road in Menlo Park almost hurts your head: Inside the offices on revolutions are dreamed, debated and funded. And it looks, for the most part, like a row of mildly prosperous dental practices. The real import of the work is, on the outside at least, nearly totally muted. The corporate structure of the most powerful tech companies are padded with this sort of deadening fustian too. Founders control the majority of voting stock; shareholders are more like lucky “users” than owners. Control, security and speed in decision-making are secured from the inside, free of exploit risk or interference. The companies are like computers. Of course the founders know where real powers sits. But this shouldn’t distract us from the human energy breathing in the code itself. The programs are “permeated by all the forms of contestation, feeling, identification, intensity, contextualization and decontextualization, signification, power relations, imaginings and embodiments that comprise any cultural object,” the computer science historian Adrian Mackenzie has written.171 Each of the parts of a black box is a black box. The famous billionaires of our technology age operate for the most part as their systems do. Their tight, well- engineered clusters of machines produce fortunes from connectivity, even as they obscure some of the deeper nature of the connections that are essential to their success. They are themselves at times obscured, human black boxes in a sense. “Linux is just an enabler,” the genius programmer Linus Torvalds once observed about the code language that undergirds much of the connected world. “It’s a solid base, but like all good, solid bases, it really is something that should be almost entirely hidden and out of people’s minds.”!” It is a hard paradox for us. The work of the black boxes, of connected systems or protocols such Linux is miraculous. It is wonderful in so many ways. And the roots of it are, and seem like they have to be, obscure. But this cuts very fast into the arteries of a healthy democracy. “Democracy,” Arthur Schlesinger wrote in his famous post-war book The Vital Center, “has no defense-in-depth against the neuroses of industrialism.” It’s easy to see how the system might also have a weakened immunity to the subversive forces 170 Or from the Talmud: See Leo Strauss, Persecution and the Art of Writing, (Glencoe, Ill.: Free Press, 1952), 21 171 The programs: Adrian Mackenzie, Cutting code: Software and sociality, (New York: Peter Lang 2006) 5 172 “Linux”: Andy Meek, “Linux creator explains why a truly secure computing platform will never exist” on bgr.com Sep. 25, 2015 118 HOUSE_OVERSIGHT_018350

of network power: Contagions of fear, manipulation of data, the subtle and invisible influence of the boxes we depend upon but don’t understand.!73 Recall Francis Bacon’s Enlightenment line? That human knowledge is human power? Well, what is computer knowledge? It is human power? Or something else, in its entire, hidden immensity? You have to wonder if this packing of insight and vision and control into black boxes, or the hands of a small New Caste will bleed us of our liberty as a result. “Our constitution is called a democracy because the power to make decisions is not in the hands of a minority but of the whole people,” Pericles reminded Athens in his Funeral Oration 2500 years ago. “We regard a man who takes no interest in politics not as harmless, but as useless.”!”4 Vital engagement is the food of democratic life. To be baffled to the outside of the essential boxes of power then, seems an instant sort of cancer on liberty. What do we make of a man who takes no interest in the networks of networks that control the power to make decisions? Perhaps you've heard of the famous manufacturing trilema: You can get something made any two of good, fast and cheap. If you want that custom table made quickly and well, it won’t be cheap. If you want it good and cheap, you had best be prepared to wait. In networks a similar puzzle emerges in my mind. Systems can be any two of fast, open or secure. A computer system that is really secure can be open, but it will be very slow, inspecting each packet and instruction like a bank security guard watching customers in a bad neighborhood. Think of the like an airport. Want it to be fast? Secure too? Then it won’t be very fast. Mostly what we want today are fast, secure arrangements for our markets, our nations, our data. So these will become, | think, ever less open. It used to be that history was made in public: Big visible wars and social shifts and revolutions. Pericles in the Athenian square; the churning protests of Jefferson’s Paris or the massing of armies. Now, however, subtle manipulations of technology, invisible to most of us and maybe even accidental, maybe weird, will produce historic-scale external effects. Changes to the network design will become political and social exploits in a sense, living versions of that atomic-level “rowhammer” hack that work on the connective energy of our world. Already social network analysis can be used to manipulate voting patterns. Soon, it will be possibly to precisely target any potential voter with a message engineered like a custom-made drug, designed to bind right to the DNA of your habits and beliefs. It represents the possibility for the complete technical perversion of politics. As the New Caste operates on the systems that are at the core of connected politics and economics, on how we vote or think or shop, they will vibrate the system in invisible ways even when they don't mean to be nefarious. Improvements may be as dangerously unpredictable as bugs. Small changes to algorithms or links or protocols mean our whole system may be pwned before we're quite aware. Such 173 “Democracy”: Arthur M. Schlesinger, Jr., The Vital Center: The Politics of Freedom (Boston: Houghton Mifflin, 1949), 246 174 “Our constitution”: Thucydides, A History of the Peloponnesian War, (Oxford: Oxford University Clarendon Press, 1881),119 119 HOUSE_OVERSIGHT_018351

adjustments will be started and managed and mined for fortunes, of course. The Seventh Sense feels the way in which power has shifted, is shifting, from public to private in this way, masked by coded language, hardware design, corporate structure and the demands for speed and safety we all agree on. Huge shifts in power will occur before we are even aware if we don’t have a sensibility to feel them out before they occur. Decisions about code, search, machine intelligence, DNA alteration rules in labs - all occurring in black box machines or corporations or governments. Several years ago it occurred to me in an unsettling flash: “The most important things that will happen in my life will happen in secret.” I’m not sure I’ve quite recovered from this insight yet. 2 This same unnerving worry has troubled many people who think about connected machines, and for some time. It’s not only those among us who are pointlessly nostalgic for a different era of devices, when the default setting of our instincts was open, who now worry about this strange tension between function and opacity. And remember - I’m running through all this here so we can all understand how to really grab and use this new source of power in the service of what we desire, and to protect the things we care about. But: What are we to make of systems that work better when they are obscured from us? That we could never understand even if we could see inside? Documents such as The Critical Engineering Manifesto, pulled together by a collection of uneasy designers and engineers, reflect anxieties that stir even inside the hearts of many in the New Caste. “The Critical Engineer considers engineering to be the most transformative language of our time,” the Manifesto begins. “Each work of engineering engineers its user, proportional to that user’s dependency on it.” That is Conway’s or Wittfogel’s unsettling truth. The technical layout of a social network, a medical diagnostic tool or a financial market affects how we or our markets or our government (or our enemies) behave. We are, ina sense, being engineered by the damn opaque systems we are using. In 1965 the MIT computer scientist Joseph Weizenbaum found himself, somewhat unexpectedly, considering this same problem. 175 Weizenbaum had written a primitive computer program to perform what is now known as natural language processing, essentially a bit of code designed to translate what a user tells a computer into clear instructions. This is the work that converts your request to a machine for weather updates into a query that can be turned into an answer. Teaching a computer to answer a question such as “it is it raining?” demands layers of logic and calculation. In 1965, long before our era of simply talking to our phones or computers, Weizenbaum’s program was a revolutionary breakthrough. So after developing his algorithms, he wanted to find an intuitive demonstration. He decided to build a 175 In 1965: Joseph Weizenbaum, Computer Power and Human Reason, W.H. Freeman and Company, 1976 p. 7 120 HOUSE_OVERSIGHT_018352

system that could, in a very limited sense, converse. The computer would take typed input from a user and then, based on what it understood, ask a question back to continue the conversation. The approach was borrowed from the psychological theories of Carl Rogers, an American therapist who believed the best way to talk to patients was with open ended, careful questions. Weizenbaum developed two programs. The first was called DOCTOR, a script that determined what the computer might say. The natural language processor responsible for carrying on the human side of the conversation he called ELIZA, after the heroine from Pygmalion, a poor cockney girl who is trained by a phonetics professor to pass as a duchess. Here is a discussion between ELIZA and a user that Weizenbaum recorded in 1965: Patient: “Men are all alike.” ELIZA: “IN WHAT WAY?” Patient: “They’re always bugging us about something or other.” ELIZA: “CAN YOU THINK OF A SPECIFIC EXAMPLE?” Patient: “Well, my boyfriend made me come here.” ELIZA: “YOUR BOYFRIEND MADE YOU COME HERE?” Patient: “He says | am depressed most of the time.” ELIZA: “I AM SORRY TO HEAR YOU ARE DEPRESSED.” Patient: “It’s true, I’m depressed most of the time.” ELIZA: “DO YOU THINK COMING HERE WILL HELP YOU TO NOT BE UNHAPPY?” Patient: “I need some help, that much seems certain.” Weizenbaum’s program quickly became famous around MIT. (The joke was that it was a better conversationalist than most of the awkward shoegazers studying at the Cambridge campus.) ELIZA gave users a chance to have a taste of a machine with something like a mind of its own. The program spread to other campuses, to different research institutions, and with every step of its success, Weizenbaum found himself more distressed. He summarized his worries in his 1972 masterpiece Computer Science and Human Reason. What troubled him was not ELIZA, but her users, the humans. Even when they were professional psychologists, many of them quickly considered it reasonable to think that one day the work of diagnosis and counseling might be turned over to machines. This felt like a natural next step in the ceaseless progress they were used to in their lives. Better refrigerators, stronger seat belts, faster jet planes, more plastic - why nota computer doing therapy? It 121 HOUSE_OVERSIGHT_018353

sounded kind of wonderful. “A number of practicing psychiatrists seriously believed the DOCTOR computer program could grow into a nearly completely automatic form of therapy,” Weizenbaum wrote. “I had thought it essential, as a prerequisite to the very possibility that one person might help another cope with his emotional problems, that the helper himself participate in the other’s experience.” To use a machine for such a task? He was horrified. Weizenbaum knew the empathy ELIZA was exuding was faked. It was just code. “Science,” he concluded, “has been gradually converted into a slow-acting poison.” “Would you mind leaving the room,” Weizenbaum’s secretary said to him once, lost in a particularly personal discussion with ELIZA. “The reaction,” he wrote, “showed me more clearly than anything I had seen hitherto the enormously exaggerated attributions even a well-educated audience is capable of making, even strives to make, to a technology it does not understand.” This was black boxing at its worst: “I have no idea how this thing works. And it’s wonderful!” What makes the New Caste so particularly powerful is that their essential work is to build and operate the cores that control these systems. And the more people they lure onto them, the more powerful the platforms - and the people who run them - become. “The computer programmer,” Weizenbaum wrote, summing up his lessons from ELIZA, “is a creator of universes for which he alone is the lawgiver.” Each of these cores represents a fusion of power and politics and technology like nothing the world has ever seen. They are assembled mostly from scratch, they represent the concentration of billions of connections, and their direction is determined by technological and market factors as much as by any democratic twitch. The strategic power of societies that train the best of the New Caste is probably self- evident by now. To educate and deploy masses of people capable of such transcendent design genius will mark a difference, an electric gating line between the nations that succeed and those that fail. But such training brings a real tension, if this group is allowed to really rip away at their work. What won't they attack? Control over the protocols that answer questions, move money, protect data, analyze your DNA - it’s hard to think of any single locus of power that will ever be greater than the tight, gravitationally inevitable platforms emerging around us now. These essential webs are filled, as we’ve seen, with complex bugs and errors and loopholes. They depend on design decisions whose implications resonate for decades — both inside the black boxes and the external world that vibrates to their quiet demands. “If builders built buildings the same way programmers write programs,” one famous coding lemma runs, “then the first woodpecker who came along would destroy civilization.” Who would know if rot is spreading in these systems? Who would stop it? Recall Paul Virilo’s line that trains produced train accidents, planes produced airplane accidents. So: Black Boxes? As much as the work of the New Caste looks tactical in nature - what protocols to use, how to engineer networks or design machine boards - the reality is that most of what they do would be blind without a strategic urge. Behind even the smallest 122 HOUSE_OVERSIGHT_018354

advance, whether it is fingerprint recognition on your phone or some new auto- translation app, a set of careful, deterministic values and calculations linger. One feature that defines the New Caste is unadulterated, unquestioned faith in the continued network revolution, and that the values that underlie the best programs may be the values that should gird the world that depends on that same code. The danger here is clear enough. “Respect, understanding and love,” Weizenbaum wrote as he considered ELIZA’s effects, “are not technical problems.” The biggest of the platforms controlled by the New Caste herd together, remember, billions of people, bind them with ever thickening cords. The revealing tics of every movement in the virtual and every step or drive in the real world are marked down, remembered and scored. To operate the strategic levers of such a force is, in all reality, no less significant than leading a nation. The distinction between a CEO ofa major connected firm and a head of state lies less in the depth and efficacy of their influence than in the questions of how they got such power, and how they might use it. The New Caste has an admirable conviction near to faith that their products are truly universal. They are absolute technological determinists. Watching their services and influence expand often has that strange aura of the irresistible force taking on an immovable object. They believe that their black boxes will bulldoze concerns of politics or history. And soon. Historical ambition of this scale, the sort that touches really countless lives, has always blended a commercial and technical mastery -- the moves of the East India Company turned as much on better ship design, maps and navigation as on imperial objectives. But the aim of the New Caste is the same as it was for those three older castes - the merchants, soldiers and sages: To put the tools they’ve mastered and built in the service of still more dominance. The commercial calculations of the most powerful figures of the New Caste carry a sense of seeing many moves ahead, a very real kind of chess. Their billion dollar acquisitions, investment in moon-shot R&D ideas, the hundred million dollar payouts for great engineers — all of these mark the astonishing scale of what they have in mind: To have and control ever more essential cores of power. Are they seduced by having a billion users? Sure, but not because of the billion users, but rather because of the seductive allure of the black box, of what it means to control such a central point of connection. 4, I remember sitting with a member of the New Caste the week the first batch of mimeographed and laser-scanned Snowden papers were released, as we both discovered that everyone we knew was devouring the documents. Like a novel. People were texting one another - “Have you seen this?!” - and you couldn’t get through a dinner without a debate over the technical merits of what was on display. The Snowden files were fascinating to the New Caste in a way few others might understand, in the way a room of ballplayers might examine Ted Williams’s swing mechanics. Let me try to explain it this way: When I was younger, people called our generation — those of us born between 1965 and 1980 more or less - a generation of slackers. Generation X. Generation Nothing. There was an argument to be had about 123 HOUSE_OVERSIGHT_018355

the Baby Boomers. Had they been the most destructive, selfish generation in American history? A reaction against the selflessness of their parents? Retired to leave the rest of us to pay their future medical bills and oggle their underfunded pensions, to cope with the manipulated political system they’d sued into existence. Or had they left a legacy of tolerance, an echo of 1968's optimisitic energy, a firming of American confidence. But, anyhow, Generation X? By comparison irrelevant: A collection of sad, passive slackers. But the great Internet companies were largely built by Generation X. The foundational experience of 1989 - the fall of the Berlin Wall - bred optimism. It created, in fact, the possibility for a new exploration. When we were told “Be generous in what you accept,” this seemed reasonable and, eventually, lucrative. The logic and power of networks became apparent by itself, the moment we began connecting the world. So linkage in trade and finance and friendship was pressed out into a new era of globalization, pushed as much by the smashing, enthusiastic removal of so many historical limits as it was by the technology itself. Yes WiFi and TCP/IP and other advances made wiring the world possible, but I wonder if they would have developed so quickly if the context for using them hadn't provided a feedback loop of such quick profit and reward and, frankly, amazement. So, in this fashion, we laid the groundwork for a world of billion-plus user platforms. For a new concept of power. But the Snowden papers were a shock. We knew him, in a sense. His mannerisms and thinking and technical instincts harmonized with our own. It was as if the NSA had enrolled most of the digitally visible world into a twisted panopticon of a social network, one where your “membership” began the moment one of your data packets was sniffed or chased along fiber optic lines. Here was a secret three billion- user platform, in a sense, that had enmeshed, without their knowledge or permission, a tremendous chunk of humanity. People of interest. The one or two or three billion people swept into systems like the NSA’s Aurora fell victim to the powerful leverage of scaled systems: The more people monitored by the NSA, the more leads to follow, which meant still more people needed to be followed and knitted unwittingly into the web. No one in my generation who had been around tech for long was naive. We knew that like soldiers coming back from a war there were things done on the network - the spreading of danger, of inequality, of pollution - that would have to be paid for in balance by the benefits the system promised to bring. The immensity of what had been built already in networks was clear enough; the even greater power yet to come as trillions of more connections piled together was implicit, obvious even. The perverse, sneaky side of this growth was known to anyone who'd spent time working a connected machine. But that people of our generation could on the one hand listen to “Karma Police” and on the other enact the sick OK Computer logic of surveillance, even name a British GCHQ monitoring program after the Radiohead song? The aim: Provide "either (a) a web browsing profile for every visible user on 124 HOUSE_OVERSIGHT_018356

the Internet, or (b) a user profile for every visible website on the Internet."1”77 We'd not honestly thought the end of this would be a need to so deeply question or defend freedom or liberty. The whole idea of connectivity had begun with liberty, after all. The Fall of the Wall. We'd not known or expected to lose it through the very systems we’d constructed in the open space a 20-year peace had presented. So: We'd been naive after all. We’d been united in a tacit claim that there was nothing bad about connectivity. We believed it for the most part and convinced others. We didn’t mean it that way, | felt the New Caste readers of the Snowden Papers were saying to each other, paging through the sick, ineffably banal logic of those NSA Power Point presentations. They were brochures for totalitarianism. If you have not read them, it is worth looking them over. They have, you'll see, all the anodyne banality of insurance company pitches, even if the dull language was expressing something vivid, nothing less than the potential murder of some very basic rights. We were reading and discussing and debating the documents so avidly because, it seems to me, each of us had a kind of horror of what we might have done. Here was a massive technological and insidious web, a totalizing virtual machine of collection and analysis and instant observation and reaction. It relied, for its safe operation, on the humans in the loop of that sensing and seeing and machine thinking. And what was it, exactly, that the humans had appeared to do in the face of such urgent responsibility? Almost like that silly secretary chatting with the ELIZA machine, they had suspended their sense of warm humanity in the face of the magical charm of electrical promise. They had let the machine run. The whole system had been built and operated by members of the New Caste who were, in turn, beguiled and charmed out of a sense of certain essential limits as they stared at the beautiful face of machine power. Looking back on his formative years, before Europe was ground up in the First World War, the powerfully brilliant and sensitive economist John Maynard Keynes bitterly recalled the iron certainty of his set of friends and their confident, diffident arrogance: “We were not aware that civilization was a thin and precarious crust erected by the personality and the will of a very few, and only maintained by rules and conventions skillfully put across and guilefuly preserved. We had no respect for traditional wisdom or the restaints of custom.”!78 It was the war, then the depression, then a war which taught them this expensive lesson. What is fracturing around us now, with our own willful and diffident support, is that very thin and precarious crust, cracked and assembled as Keynes said by “the will of a very few”. What if those very few are the New Caste? 177 The aim: GHCQ “Pullthrough Steering Group Meeting #16” available from theintercept.com 178 “We were not aware”: John Maynard Keynes, “My Early Beliefs” in Two Memoirs: Dr. Melchior, a Defeated Enemy, and My Early Beliefs (New York: A.M. Kelley, 1949) 99 125 HOUSE_OVERSIGHT_018357

The essence of the Seventh Sense will be not merely to be beguiled by our technology, by the way it smashes old systems, not to ask “Could you please leave the room while I use bend these electrons to even more omnipotent control,” but rather - in the same instant as the networks snaps into its full power- to grasp the full nature of the connected age. To see how it might be used to further, not erode, the things we care most about. To ensure that if we are not among the very few, at least we can guarantee that their will bends towards justice. We will see, ina moment, just what that means in practice. How our best technology and our most avid hopes for what a technological, connected future could look like might just be yoked together. But before we can do that, there is one final question we need to answer about the networks all around us now: What, in the end, are they really for? 126 HOUSE_OVERSIGHT_018358

Chapter Eight: “MapReduce”: The Compression of Space and Time In which we learn what networks are really, rather wonderfully, meant for. 1. Starting in the springtime of 1997, the American scientist and inventor Danny Hillis began what has since become an every-few-months sort of ritual. He packed up from his home in Encino, a short drive over the Hollywood Hills from Los Angeles, and headed off for rural Texas for a few days that would largely defined by rock and dynamite. Hillis, who was born in 1956, has spent most of his life working at the electron level of the world, crafting some of the most significant computer processing systems of our age. So the sort of paleolithinc earth moving he was heading off to manage in Texas was a departure from his usual scale. His aim was to work on blasting and then refining a space in an isolated mountainside for the construction of a towering clock that he had designed, one intended to run for 10,000 years. That ten-millennia span was not accidentally chosen. Humans, when Hillis began his work on the clock, had been around about that long already. We were, as he pictured it, ata midpoint on that 20,000 year stretch of time. Hillis and the group of tinkerers, thinkers, and engineers who had backed and designed the clock - people such as Amazon’s Jeff Bezos, spreadsheet inventor Mitch Kapor or investor Esther Dyson - were planning on a project that would stretch as close to eternity as they felt reasonable. “The Clock of the Long Now” they called it. I remember pulling into Danny’s driveway in Encino one afternoon as he prepared to depart for Texas and being struck by the contrast between the lovely, innofensive suburban blandness of Southern California and the tools he was taking with him to make an assault not merely on a mountain, but on a whole conception of time. | had met Hillis in an unusual fashion. I’d been asked to chair a committee that would award a million dollars to a figure who had made an essential contribution in the world of technology. The directors of the foundation behind the then-new prize had been, from the start, slyly dropping big names - Bill Gates! Steve Jobs! They hoped such a laureate would cast a bit of glamour on the first year of their award for “Contributions to Man’s Present Condition.” But when our committee sat down to talk it over, we knew that the boldfaced names didn’t want or need a prize. They certainly didn’t need a million dollars. As we considered people we all knew who’d made fundamental, essential contributions but had not been as boldfaced as they might have been, Danny Hillis’s name came up immediately. Hillis had developed a revolutionary “massively parallel” computer in the 1980s. The machine had helped create an entire discipline of high-speed computing by tying together tens of thousands of processors to tackle a problem at once. Traditional computers worked problems the way you or I might, step by step. Hillis’ design was the equivalent of millions of minds, all moving at once. Coordinated, connected and awesomely fast. In the years since, he’d played a key role in a dozen 127 HOUSE_OVERSIGHT_018359

other breakthroughs, from designing artificial intelligences to fine-tuning classified military aircraft systems that depended on mathematics for their stability. When you wander into a deep part of Google’s technical database systems, you're touching his work. When you talk to your phone, the interface bubbles with some of his patents. How did Baran’s 1960s idea of a survivable, packet-based system at ARPAnet become the Internet in the 1970s and 80s? Danny was part of a cluster of dirty-fingernail engineers - along with engineers such as Vint Cerf and Jon Postel - who'd done the work to make it possible. His centrality in that project was memorialized in a famous speech he once delivered in which he described having one of the very first email addresses in history - and then whipped out a sheaf of bound pages that represented the entire Internet address list at the time. It ran about 50 pages. To the extent there were membership cards in the New Caste, Danny’s would have had a very low number. It was an easy decision for our prize committee. No Bill Gates. No Steve Jobs. So, here’s how | met Danny Hillis: I called to tell him he had won a million dollars. (I recommend this as a way to start a friendship.) Hillis had been a tinkerer since he was a child and never seemed to have lost the pleasure of a wild intermingling of joy and practice. You couldn’t tell with him where passion ended and work started. He was so technically adept that he could inject even the coldest digital projects with a bit of hot emotion, like Bernini breathing life into a block of carved marble with one, “just so” grace note of his chisel. One of Hillis’s most famous projects, for instance, was a 15-foot high tic-tac- toe playing tinkertoy robot he’d built when he was 20 years old, in his second year as an undergraduate at MIT in 1975. Made from 10,000 wooden spindles and poles, it was an early attempt of his to show how machines, even simple ones, might seduce us with both brains and looks. The effect of a giant tinkertoy pile sitting there at The Massachusets Institute of Technology had to make you giggle, even as your mind boggled at the fact that this heap of sticks, strings and dials was beating you again and again ata child’s game. Hillis was an artist as much as an inventor - one reason he’d not become Bill Gates or Steve Jobs. (And why Gates and Jobs maintained a consistent, admiring respect for him.) He’d once spent a decade at Disney designing rides or thinking up new dreams as a kind of real-world mayor of Tomorrowland. He liked to joke that he knew he was at the right place when, on his first day, he asked where he might find a parachute harness for an experiment and heard, in response, “What size?” Hillis was an avid reader, and he has the habit to think of his bleeding-edge work in the context of long historical gulps. Conversations with him often tie back to Paleocene era biology or some other deep root. That long-term view, married to his unmatched hands-on feel for complicated systems, made him the ideal designer for the clock, a machine intended to last millennia. The problems associated with such an undertaking were, honestly, as unreal as you might expect. How to power the clock? (Hand winding, the better to ensure it was not forgotten.) How to protect it? (Put in the middle of nowhere.) Did you need to plan for global climate change? (Yes. The design was adjusted to accommodate shifts in the earth’s spin when the planet’s icecaps melt off.) Do you write a users’ manual for people 10,000 years from now? 128 HOUSE_OVERSIGHT_018360

(Yes.) Do you write it in English? (To be determined!) Working with composer Brian Eno on the sound of the clock chime, and with a team of geologists and physicists, Hillis had made the clock into a natural extension of his tinkertoy computer, a device that both served a purpose and sent a message. If there was an emotion it conveyed, a feeling that it tickled in the way Bernini’s Apollo and Daphne might inspire terror or joy or faith, it was meant to be awe. Stewart Brand, one of the supporters of the clock and an early member of the New Caste too, would tell you that the idea for the clock had emerged from a desire to emphasize, to physicalize in a way no one could forget, the importance of longer- term thinking. We'd all arrived now, Brand and the other clock masters worried, ata moment in history when no one had a view that extended much past their own lives — or sometimes past the next election, or year, or the next financial quarter. Our modern “on to the next thing” economics and politics were eroding every slow, patient instinct. “Civilization is revving itself into a pathetically short attention span,” one manifesto for the clock began. “What we propose is both a mechanism and a myth.”!7? With its steady 10,000 year ticking, the Clock of the Long Now was meant to make us think in longer jumps. The human winding mechanism, for instance: Generations of clock-winders had to share in the work, and they would be connected in a long thread over the 10,000 years. A sacred priesthood of time. Moving slowly. As I spent time thinking, dreaming about the clock, I found myself too craving the solidity and patient isolation it promised. Who among us these days doesn’t want a break from the instant nowness of our age? Yet, the more | understood the clock, the more | realized something else was at work. Stop fora moment to consider who was backing and building the device. It was a cluster of people who had, as a common link, the fact that they had their hands honestly sunk into the guts of the Internet. Hillis, after all, had been waving more than that slim book of email addresses when he talked about the early days of the Internet. He was waving the credentials of a man who had been living in the virtual cyber neighborhood of Web connections from its very first days. He was as close toa native of the connected, fiber optic, light-speed world as you could find. All the names supporting the clock smelled similarly of burning electrons: Jeff Bezos had built Amazon into a high-speed marketplace whose backbone was the Web itself. Another backer, Mitch Kapor, had cracked apart several centuries of slow accounting habits when he created Lotus 1-2-3, the first successful computer spreadsheet program in 1983, software that permitted you to see and change your whole business one keystroke at a time. Kapor’s software helped move finance from quarter-by-quarter calculations to a really instant-by-instant sort of business - more or less the opposite of the “long time frame” the clock team was aiming to preserve. Esther Dyson was one of the earliest, best investors in network companies. This was a collection of men and women unified by a genius for connected change, sure, but also by a desire for ever faster clock speeds, ever speedier delivery, ever faster 179 “Civilization is revving itself’: Stewart Brand, The Clock of the Long Now: Time and Responsibility. (New York: Basic Books, 1999) 2 129 HOUSE_OVERSIGHT_018361

processing. They had lived this. Enabled it. Profited from it. If there was ever a group you might hope to take aside, pull into a quiet room and ask gently What are the really networks for, anyway? this would be it. The act of keeping time, of marking it, is embedded in the nature of any age. Our lives are, after all, dictated by timetables: School schedules, the seasons, rush hour, the burning candle of birth-love-marriage-death. Time, in the days before industry, was measured by nature’s schedule. How long it took a crop to mature. The solstices. A beehive filling with honey. It was marked by moving tides and shifting seasons, and it demanded a slowness, a personal presence on the shores, in the oceans, atop the fields over generations. “Summer afternoon,” the novelist Henry James remarked in a précis of a slower age he felt passing away in 1895. “To me those have always been the two most beautiful words in the English language.” 18° Then, in the industrial revolution, time became money. Electric lights, for instance, undid the restful distinction between night and day - and made 24-hour life and manufacturing and economics possible and then, of course, inevitable. Movement from countryside to city established a really new sense of what the German critic Georg Simmel, writing in 1903, called tempo. “With each crossing of the street, with the tempo and multiplicity of economic, occupational and social life, the city sets up a deep contrast with small town and rural life,” he explained. “The technique of metropolitan life is unimaginable without the most punctual integration of all activities and mutual relations into a stable and impersonal time schedule.”18! Punch cards. Bus schedules. The forty hour work week. Our education, our manufacturing, our markets and our lives all began to run on timetables. They had to, or the whole project of industry would collapse. “Summer Afternoon” became a time to work. Simmel worried, for instance, over the diffusion of pocket watches. To carry one was like looking at a constantly draining bank account. This sense of humans reduced to cogs - churned, run, disposed of on a schedule not their own - unnerved the residents of that first mechanical age. Cities had been the very first sorts of tight-packed networks; industrial cities ratcheted this further still. They succeeded and failed by the degree to which they geared themselves and their citizens to machine speed. When the Austrian novelist Robert Musil began The Man Without Qualities, his classic story of the era, with the flattening of a Viennese citizen by a speeding delivery truck, he meant to point out how urban speed and urban life (and urban death) had become inseparable. And also the mismatch between the weak breaks of the age and it’s acceleration. The book is alive with that pre-accident slipping sensation you may have had: You are pressing hard on the brakes of the car; you are going to hit something anyhow. “Cities can be recognized by their pace just as people can by their walk,” Musil wrote in a line that any modern New Yorker or 180 “Summer afternoon”: Edith Wharton, A Backwards Glance, (New York: D. Appleton-Century, 1934) See chapter 10 for her tale of an afternoon with James. 181 “With each crossing”: Georg Simmel, “The Metropolis and Mental Life” (1903) in Gary Bridge and Sophie Watson, eds. The Blackwell City Reader. Oxford and Malden, MA: Wiley-Blackwell, 2002, p. 11 130 HOUSE_OVERSIGHT_018362

Parisian would endorse.1@ “A man returning after years of absence would have known, with his eyes shut, that he was in that ancient capital and imperial city, Vienna.” To really get a feel for that age, watch just a few frames of Russian director Dizga Vertov’s jittery 1929 black and white film Man with a Movie Camera’, Each moment of the movie is alive with the tension of a new, rushing and industrial age. You won't be surprised Vertov’s list of requirements as he prepared to start filming began: “1. A rapid means of transportation.” His aim was to immortalize urban speed on the new medium film; he knew he needed to be fast.184 To know a city by it’s pace. Musil was touching something deep here, an instinct that beats in each of us and runs from this fact: The speed of an event affects how we perceive it. The difference between what you will notice when walking up a hill - chirping bugs, tiny rocks, changes in color and gradient - and driving up that hill is so complete as to be almost different experiences entirely.!@° When the whole world tumbles upon us at fiber-optic speed, when invasions and revelations and accidents all spread at the rate of WiFi or cell phone radiation, our sense of time blurs. You have to wonder what Simmel would have made of a smart phone. “It is not merely that the medium is the message, but the velocity of the medium,” Paul Virlio observed once in one of his many studies on speed and mind.186 Life in our connected age is both instant and always on, what Simmel might have called “the technique of network life.” This demolishes an older, easier sense of pace. Computers were once switched on at 9 and off at 5 — just like their human masters. But digital activity is constant now. The networks are paying attention all the time. They have to. Our machines - tractors and trains and cars - used to echo our pace of life. Now we echo theirs. The machines, the New Caste, the black boxes - they tick along constantly, ever faster. We rely on them, as we've seen, for our safety. We want them to be fast. To be instant. But what is this doing to us? Is completely unhooking us from any sense of time really a good idea? It was certainly true, as Brand insisted, that the Clock of the Long Now was meant as a reminder, as a kind of constant totem to the fact that we’re all just a small tick on 182 “Cities can be recognized”: Robert Musil, The Man Without Qualities Vintage (1996) 3 183 To really get a feel: Vertov’s movie is shot in Moscow, Kiev, Odessa and Krakow. In 2003 The Cinematic Orchestra produced a modern soundtrack to accompany it. 184 “A rapid means”: Jonathan Dawson, “Dziga Vertov” Great Directors March 2003 Issue 25 185 Ajahn Brahm of the Buddhist Society of Western Australia tells a wonderful story in his Dharma talks about a day in which he decided not to be driven up the road to his monastery in Perth, but instead walked the road. He was flooded by the sensations of nature, all invisible from the inside of a car. One way to change your perspective, he means, is to change the speed at which you are moving through life. See the Buddhist Society of Western Australia website for a catalog of his talks. 186 “It is not merely that the medium”: Paul Virilio, Information Bomb trans. Chris Turner (London: Verso, 2000), 141 131 HOUSE_OVERSIGHT_018363

the endless long continuum. We do think in too short a time frame. But the clock also, I began to suspect as I considered it, had another role. Those 10,000 years of marked time were an attempt to scratch an itch bothering these pioneers of cyberspace. It might even have been a sort of guilty sensation. After all, there was something that they had demolished in their ceaseless fast connecting of the world - maybe accidentally, but anyhow it could never be put back to where it was. If the great industrial titans before them had, over several hundred years, vanquished distance, lacing the world with trading networks, the men and women behind the clock were fracturing something else, something that for all of human history had seemed the only reliable, safe, sad constant of our condition: Time. For most of history, time and space were seen as facts, immutable forces that could not be overcome or adjusted or fought. Time particularly. The quintessential, tragically non-negotiable condition of life. The backers of the clock were, in their “day jobs”, in the business of overcoming, adjusting, fighting and even destroying an older sense of time. What might once have taken years, they were committed to making happen in an instant. The Long Now project, then, was like one of those carefully isolated arctic freezers, where samples of essential grains and DNA from Beethoven’s hair and Einstein’s brain are sunk and iced against the godforbid day in which our basic feedstock or a chunk of humanity has been wiped out by accident or disaster. Blasted into a mountain hole, designed to last thousands of years, the clock is a repository for time itself. Itis a defensive museum, built against the moment when instant networks finally devour the off switch and kill an older, essential feeling of time. The clock- makers knew, | think, that they had helped to demolish a particular sense of pace with their fast, instant networks. They wanted, with the knowledgeable keening of the guilty, a new device, one carefully gated away from the very revolution they were encouraging with their business and technical obsessions. This hunger for speed had built their fortunes. Their revolution (and its IPOs) had paid for the clock, frankly. And the hope for “instant world” had inspired their dreams long before deep, fast connections were possible. Bezos’s first name for Amazon, Cadabara, captured just this sense of waving wands and the eye-blink “Now you don't see it, now you do!” appearance of just what you want or need. Murdering time. You could ask: What are the really networks for anyway? Would they feel guilty when they confessed? 2. Danny Hillis’s father was an epidemiologist. His mother was a bio-statistician. And his childhood was a blur of infection-led family migrations. “Anywhere in the world there was an epidemic, we would go,” he recalled. As the family bounced from Dehli to Cairo to Dakha to Nairobi, racing diseases one after another, Danny developed a vivid, energetic autodidactisism. He would collect knowledge from his parents, from the streets around him, his new friends, anywhere. In a library in Calcutta, for instance, he once found a copy of George Boole’s 1854 book “An Investigation Into the Laws of Thought”. Boole invented symbolic logic on those pages and though his instincts were grounded in an age of steam and machines, his vision still echoes in 132 HOUSE_OVERSIGHT_018364

modern computer design. “Language,” Boole wrote, “is an instrument of human reason, and not merely a medium for the expression of thought.”187 Hillis has a magnetic intellectual charisma, as you might have guessed by now. An afternoon with him resembles nothing so much as a lingering mental theme park: Roller coasters of big ideas (a 10,000 year clock!) mixed with smaller sugary treats (how to design a better fencepost). No wonder he fit in so well at Disney. Critics accused Steve Jobs of having a “reality distortion field”, in which the Apple founder’s charisma bludgeoned the boundaries of the practical. Hillis, by contrast, has a sort of “reality enhancement field” in which much of the world as seen through his eyes or heard in his light-hearted voice is sharper, filled with possibility. From an early age Hillis had been interested in the dream of a thinking robot. Maybe it was that the constant uprooting of his childhood left him with a giddy sense that it was easier to assemble your own friends than to try to make them new at each stop. But somehow this led him to the idea of an artificial brain, which was Danny’s main idea when he arrived at MIT in the fall of 1972. The tinker-toy tic-tac-toe computer he built was a nod to this hope, but it’s jerry-rig aesthetic masked deeper ambitions. “Someday, perhaps soon, we will build a machine that will be able to perform the functions of a human mind,” Hillis wrote at the start of his PhD thesis a few years later. “A thinking machine.”188 What Hillis and others like his mentor Marvin Minsky, realized was that the human brain works differently than machine logic. Life, after all, is not a series of linear math problems. (Much as we might wish it was at times.) You look outside. It occurs to you to say to your wife, “What a lovely day.” This is not a result of some “a then b then c” calculation, but rather the product of thousands of simultaneous inputs and twitches dancing through the space of your consciousness. If you were to process that same thought in a linear fashion, it might look like this: First, look at the sky, examine the cloud-to-blue ratio, check for too much wind, sense the temperature, open mouth. Your wife would be out the door before you'd even begun to speak. The ability to operate on many different pieces of data all at once is one of the most striking, enviable features of the human mind. But of course, you've probably, recognized: That is fundamentally a network problem. How do you act instantly, everywhere? Connection. So it was that, a dozen years after his tic-tac-toe machine, Hillis began work on a device designed to think super fast. Faster than any computer ever had. He called it The Connection Machine. “The ability to configure the topology of the machine to match the topology of the problem will turn out to be one of the most important features of the Connection Machine,” he wrote. Adding, in case the 187 “Language’: George Boole, An Investigation of the Laws of Thought: On Which Are Founded the Mathematical Theories of Logic and Probabilities. (New York, N.Y.: Dover Publications, 1951) 188 “Someday”: William Daniel Hillis, The Connection Machine, PhD. Thesis Submission, MIT (1985) p. 2 133 HOUSE_OVERSIGHT_018365

academic panel at MIT missed the point: “(That is why it is called the Connection Machine.)”189 Hillis’s ambition to build this device boiled in him while he was at MIT, and it finally outstripped what MIT could support, so he gathered a group of students and started a small company. The Thinking Machines Corporation, blessed by some combination of Hillis’s charisma and the fantastic promise of the project, became a magnetic field for talent, ideas and money. In the early days of the firm, for instance, the hunt for investors led Danny to the luxurious New York City apartment of William Paley, the founder of CBS. Hillis lived then in a ramshackle house close to campus at MIT. He drove a surplus fire truck to work most days.1%° Faced with the urbane, powerful 81- year old founder of the largest radio and TV network in America, Danny jumped right into a passionate introduction of his ideas about connection and networks. Paley, cooly: "I didn't understand a word you said." Then: A check for $5 million. Or there was the time that Hillis asked Nobel-prize winning physicist Richard Feynman to tip him off about any smart scientists Thinking Machines might hire. Feynman, pushing 60, volunteered himself and spent his summer vacations for the next ten years with Hillis and his team.1?! When it came time to test the first Connection Machine, it was Feynman’s data that revealed how well the black box was doing its job. The architecture they had designed cranked through what would have been a month’s worth of complex chromodynamics problems in hours. As the machine got better, these already fast processing times improved by another factor of 1,000. Such a machine, for scientists who were desperate for computed answers, was like adding years to their life. If they could solve a problem in a week instead of years? The whole texture of their careers would be altered. “At times,” one fellow computer developer remarked, “the Connection Machine seems so different from current computers that it seems more akin to science fiction than high technology.”!92 Thinking Machines Corporation sold Danny’s computers to Lockheed to model stealth fighters. Chevron used one to model oil fields. The US government bought several to help to predict the weather. Puzzles long resistant to mere power melted in the face of parallel consideration. Nothing was more exciting about Hillis’s machines than this intimiate, unprecedented link between intelligence and speed. If you have twice as many processors as | do, you can perhaps crack a puzzle of genomics or cryptography a year faster. But say you have figured out how to have 275,000 machines linked together and I have 1,000? You can solve a 189 Adding: Hillis thesis, p. 19 190 He drove: Po Bronson, “The Long Now: Time Traveling with Danny Hillis”, Wired (May, 1998) 191 Feynman: W. Daniel Hillis, “Richard Feynman and the Connection Machine” Physics Today February 1989, p. 78 192 “At times”: Michael J. Black “Book Reviews: The Connection Machine”, Al Magazine Volume 7 Number 3 (1986) p 169 134 HOUSE_OVERSIGHT_018366

problem eight years sooner!*3, Between 2007 and 2015 the number of connections a Hillis-style neural computer could handle grew from 1 million to 100 billion. This speed did produce things very like science fiction: Accurate voice recognition. Real- time genetics. And it also began to mark out, clearly, the powerful network territory where our future will be decided. <P Of all the things that mark a change between our modern lives and the days of those who came before us, few are as sensationally obvious as the sheer acceleration of life, the reduction of delay and the emerging instantness of experience. Faster. What is going on inside the machines, as Mel Conway’s old law would have told us, is happening too on the surface of our lives. A feeling of breathlessness in the face of speed isn’t new, of course. When Anna Karenina folds herself under an oncoming train at the end of Tolstoy’s novel, for instance, her suicide is as much metaphor as personal tragedy, a comment on the disorienting steam, engine, and rail pace of modernity. Speed kills, old habits and ideas particularly. Between 1840 and 1940 travel times between Anna’s St. Petersburg and Vronsky’s Moscow shrunk by 10 minutes every year on average, loosing deep cracks in Russian economics and politics, tearing apart Anna’s slow-moving world of glittering balls and hereditary estates with the fast force of industry, modernity and then the awful pliers of communism. Tolstoy’s own death in 1910 held a bit of this acute tension between old and new velocities: At 82, hoping to live out his final days in the peace of a small hut, he left his family for the rural Russian town of Sharmardino. By train. He died at a station on the way, stopped quite literally, like an absurd figure in a Gogol novel, as he was enacting the tragedy of trying to use the modern to get to the past. At the same time in the late 19 Century, the American rail system was working its own transformation, but with almost no ambivalence. America was using the modern to get to the future - as fast as possible. This was a decisive difference in temperament. “The American frontier,” Frederick Jackson Turner wrote in his famous 1894 essay about borders and American life, “is sharply distinguished from the European frontier—a fortified boundary line running through dense populations.”1% American rails and roads (and trade) encountered no substantial fortifications. They ran nearly unchecked into the wilderness. The only apparent limit to expansion, that generation thought, was technology itself. The trains had, from the start, an unusual purchase in American life. During the three decades after 1840, the refinement of small but important details - faster train engines, stiffer carriage design, tracks that were straighter, an ability to move and re-load boxcars at night, better time-table management - eased America into the steam engine age 193 You can solve: Adam Beberg lecture “Distributed Systems: Computation with a million friends”, speech at Stanford University Computer Systems Colloquium (April 30, 2008) available online 195 “The American frontier”: Frederick Jackson Turner, “The Significance of the Frontier in American History”, The Annual Report of the American Historical Association, 1894, pp. 119-227 135 HOUSE_OVERSIGHT_018367

at faster a pace than any nation. “The most significant thing about the American frontier,” Turner explained, “is that it lies at the hither edge of free land.” This brisk acceleration of rail transit revealed an axiom of speed that matters to us still: The faster your speed, the less distance matters. Accelerate from five to fifty to five hundred miles an hour and the mileage becomes less significant with each notch on the speedometer. It all takes the same amount of time. Marx called the process “the annihilation of space by time”. He was right. Speed kills distance. The simple algebra linking increased speed and reduced distance had been apparent already in the shift from rowed galleys to sailing ships, but the age of industrial transport by rail or air meant that rapid changes, changes that affected the very quality of movement, took place within the time frame of a single lifetime. The acceleration from horseback to train to plane happened over a period of 150 years or so. Each new acceleration diminished the impact of distance. There’s a phrase for this process - “Space-Time Compression” - first identified by the American sociologist Donald Janelle in 19651. Janelle saw that the technologies of transportation, trains and planes and boats, and all the little innovations that made them move ever-faster, were disrupting old spatial habits. They helped move goods more quickly, sure, but in the process they were also making the old, geographic maps less useful. When you could fly over a mountain, its importance diminished. In a wagon train you might have contemplated the desert with fear, by car you'd merely consider it with care. In a plane it was irrelevant. Janelle concluded that raw economics drove this compression as much as science. Centuries of constantly collapsing space and time had been driven not least by the hunger to poke into distant markets, to latch onto cheap labor, and to pull natural resources to wherever they were needed. This was “civilization” as a verb. The demands for ever more commerce, ever faster, ever more profitably suggested that the horses-to-trains-to-cars-to jets acceleration was an inevitable feature of modern markets. We should expect it to continue, Janelle figured. Great fortunes would accumulate to those who mastered speed. To be fast is a competitive advantage; to be faster! decisive. Absolute speed is absolute power, as the philosopher Paul Virilio has framed it.19” That idea of “Space Time Compression” sounded felicitous enough, like the name of a clever magic trick. Space compressed? Time reduced? But that felicity hid the violent, revolutionary nature of the mechanism so powerfully at work. It meant that the battlefields of power, which for most of human history had been over the control of space, would now become - rather incredibly - about the control of time. 196 There’s a phrase: Janelle identified what he called “Space-Time Convergence;” Later geographer David Harvey renamed it as “Space Time Compression,” which is more commonly used today. 197 Absolute speed: See Armitage, John, Ed. The Virilio Dictionary (Edinburgh: Edinburgh University Press, 2013) 136 HOUSE_OVERSIGHT_018368

Janelle published his first paper on space-time compression in 1965 in the pages of the reference journal of mapmakers, The Professional Geographer. But he was, of course, blowing up nearly everything professional geography thought it was about. “Geographers, as physicists, have traditionally been concerned with the positions of points (places) in space,” he wrote. “However, geographers have not employed the concept of ‘velocity’ in studying spatial relationships. Yet it might be of value and not too far-fetched for the geographer to ask ‘at what ‘velocities’ are settlements approaching one another?” We should ask ourselves the same question. At what velocity are you and | getting closer to other points on the planet? Janelle was writing in 1965. He was concerned then about the sound barrier as the practical limit to speed. But imagine his insights applied to an age where networks are switched always on, constantly packing the world ever tighter? Where a mistake or an innovation or an attack in one place can happen instantly and everywhere, because the speed limit is the speed of light? 4, At first glance, of course, geography seems the least dynamic of sciences. It is rooted in the glacial-paced realities of geology, a discipline where speed is usually measured in the creaking, inches-a-century advance of tectonic plates. The faster links of transportation, whether they are trains or planes or data connections, lay blanket-like atop that slower-moving geological layer. These high-velocity networks are, in a way, a new geography. Mathematicians and data architects call the landscape they represent a “topology”. The word refers to maps that can be re- arranged as a result of connection, the way in which speed and distance between two points does affect how “far apart” they are. You can think of it this way: Geographies are pretty much constant; topologies can change in an instant. In geographic terms, Moscow and St. Petersburg are always 450 miles apart. When you hear a network engineer talk about designing for a certain “topology” you should think of hearing an architect describe the natural geography where a bridge or a sky-scraper will one day sit. When you use an app, link to a finance market, or wire yourself with sensors you're connecting to a topology that has been designed and mapped - and where what you can do is determined by that landscape. An appreciation for tone and movement on a topology is a sign the new sensibility we have called the Seventh Sense. Napoleon saw the battlefields of his age differently than his enemies. They saw flat surfaces for the collision of soldiers, he saw a third dimension, saw air filled with artillery and decisive top-down dominance. Our Seventh Sense masters, those who can perceive a new order emerging, see wired topologies in this same fresh way. Even though these landscapses are often invisible or made up only of narrow fiber strings, it’s important that we try to picture network topologies as real, as alive and buzzing and rich with connection and data, places where fortunes will be made and lost, wars fought - and every bit as influential as physical geography. Topologies represent the landscape where real-world edifices like the Web or the NYSE or Hizballah are built. They can change shape, instantly, depending on who is 137 HOUSE_OVERSIGHT_018369

connected, and by the speed and thickness of that connection. The topology of Wall Street in the 1920s, for instance, was largely defined by who happened to come to the trading floor on a given day; today it is a global landscape, influenced by news, rumors and real-time profit twitches anywhere on the planet. Just like moving a river from one place to another would radically change the utility of a bridge, a change in topology changes the shape of systems that depend on it. That Seventh Sense instinct - the powerful can become useless because of connection, the useless can become powerful - is earned first through a fluency and even faith in these sorts of rapid, fate-changing topological shifts. In recent years the topologies of our network world have changed at the pace of technology, which is very fast indeed. Every new piece of a network, every new platform or protocol, alters how we connect. This process works on our sense of distance like an efficient, strange sewing machine: Something very far away can be, suddenly, with one stitch of innovation, right on top of you. The speed and the quality of a connection is what determines how honestly “near” or “far” something is. Location is, in a sense, as changeable as velocity.178 Distance, on any living, networked web, is an endlessly pliable sheet. Just as you can bring two distant points on a piece of paper right next to each other by folding the sheet, so you can glue points in networks together by bending the space on which they are connected. A map of the networked world or of nations or even of our city is not some given, settled graph. One small twist and we are, like it or not, right on top of each other. This makes it particularly murderous to hold onto the old idea that you and | are unconnected points. Do countries like America or China have legitimate interests thousands of miles away from their coastlines? Of course.199 In this way, the entire premise of Enlightenment life, the atomic focus on the power of the individual, becomes dangerous. It is now essential to use virtual topologies to operate in the real world, to bend these ethereal elements of connection to influence and even total control. Thomas Dullien, one of the researchers who discovered that “rowhammer’” chip hack captured this in a new law of network security that echoes, in fact, through all of connected life: You don’t have to possess an object in order to control it. “Being hacked,” he explained in a 2011 speech called Why Johnny Can’t Tell If He’s Compromised, “is loss of control without change of ownership or possession.”2°° Your phone, resting constantly in your pocket, may in fact be pwned at every keystroke by someone thousands of miles away. This is an extremely important idea, an expression again of how connection changes the nature of an object: It makes it controllable without possession. An army might be able to master an 198 Location: John May and Nigel Thrift eds. Timespace: Geographies of Temporality (London: Routledge, 2001) 2 199 Of course: Distributed Lethality in Proceedings (U.S. Naval Institute) Jan 2015 p. 343 200 “Being hacked”: Halvar Flake, Why Johnny Can't Tell If He Is Compromised, keynote Area41 Conference, June 2014, Zurich 138 HOUSE_OVERSIGHT_018370

enemy’s territory without ever possessing it, for instance, if it can manage to own the crucial topological infrastructures: banks, databases, communications systems. One nation might be able to pwn another in this bloodless fashion. Networks, you recall we said, will break nations in the future. This is just how such smashing control will be achieved, from the linked mesh running silently and irreplaceably under every element of national life. Today billion-dollar firms control cars, tools, or hotel rooms without possessing them. The links draw out value. Michaelangelo’s famous urging resonates here: Every block of stone has a sculpture inside of it, and it is the task of the sculptor to discover it. Every network has a topology. It is the task of each of us to discover it. Topologies linger everywhere there is connection. Networks can be designed in countless ways: The fishnets of Baran, the hub and spoke of a data center, the ever- changing mesh of a trading system. But what they all share is connective topologies of one sort or another and - as a result - the fact that risk that lingers any one place in the system also exists nearly any other linked place. Constant connection produces, as an unsettling result, constant threat. Connection spreads, distant parts of the world are superglued via that topological folding. Topology is not marked out merely by a description of how we connect. Rather it is scored on what is called a “trust graph”, a kind of map of who you or a machine or a network trusts - and how much. An older generation still thinks a network is something made of wires and switches and plugs. But their real power comes from something far more ethereal. When you connect to a person or an object, you connect as well to its whole history of decisions about who to trust. Every EU country connects to the choices ofa single border guard, for instance. Who does he trust? Is he right? Financial systems and technology webs are the same. If you are what you are connected to, you are also - rather unnervingly - the sum of every trusting (or untrusting) choice someone or some machine has made. This creates a worrying result: “In the systems we've built now,’ Dullien has explained, “there is no way to establish who is in control.” If you or anyone your're linked to has made a trust mistake, you may be pwned, vulnerable, and one hacked slice from loss of control. Any object - your tablet computer, a digital currency, a hacked drone - can become dangerous now in this sense, hacked by Warez Dudes into lethality. And nearly any place can be attacked in some fashion or another, as long as it is connected. The old chestnut of military strategy, that a clever or desperate army can always “trade space for time” -make a gradual retreat in order to buy time, is nearly gone now. Space is a wall that can be breached by time manipulation tools; there’s no place to retreat to. Markets in Mongolia, airports in Europe, Chinese urban landscapes - all of these now can be struck, more or less at any time, because they are all connected. Unlike traditional conflict, where the location of your most terrifying dangers might be exactly pinpointed and watched, where military zones and civilian zones were carefully divided by front lines, a connected world is one of potential universal peril.2°! The distinction between battlefield and everyday streets disappears and it 201 Unlike traditional conflict: Derek Gregory, “The Everywhere War” The Geographical Journal, Vol. 177, No. 3, September 2011, pp. 238-250 139 HOUSE_OVERSIGHT_018371

takes with it - if we don’t handle this properly - something even more precious: The division between periods of peace and of war. That the knotting together of distance, speed and power changes the nature of an object was something Janelle, the father of “Space Time Compression,” anticipated. He labeled it “Locational Utility’, the way in which something becomes more useful or powerful or relevant as it is drawn closer to us by increased connection and speed, even if it stays the same “distance” away’. A nuclear weapon three hours from landing and one that is three months away are, nearly, a different object entirely. Adam Smiths’s famous remark in his Theory of Moral Sentiments that most people would be more perturbed by the loss of half a finger than the news that a million Chinese had perished, begins to take on a different color in an age when a billion and a half Chinese are nanoseconds away.2°? When we say that connection changes the nature of an object, we mean this; Networks change the “locational utility” of anything they touch. When connection makes an object instantly, clearly visible, it revolutionizes its potential. Little wonder so many great fortunes are being made in applying this trick of plugging goods into compression engines like apps or matching services. It’s hard to know if firms like Airbnb or Uber will be around ina decade, but the economic energy they release emerges right out of Janelle’s theories: Connect a car seat or an empty room and you change its nature. You give it value. Part of our unease now - and part of the problem we have in strategizing about the world or our businesses - is that stability on our topological maps is some time off yet. There is so much yet to be connected. So many new topologies to be built. “Time is a ride,” Danny Hillis once remarked in an early meditation about his rock- sunk clock, “and you are on it.”2°4 He was right. That ride takes place, in a connected age, on topological rails. And just how “instant” you are will be a mark of what sort of ride you're on. In the same way that rivers and oceans and mountains define different landscapes in real geographies, topological neighborhoods too will each bounce with funny quirks. Some will be super fast. Others yoked by politics. Citizens of Santa Fe or Mumbai may choose to compress time in different ways. But they will all share a common desire. To do more with less. The German philosopher Peter Sloterdjik, writing about the way in which some people breeze through airports and borders (with first class tickets and pre-approved immigration) while others struggle to move at all out of refugee camps or poverty traps, has labeled the winners of this new order as a kind of “kinetic elite”2°5. They are a kind of temporal elite, in possession of golden keys to a special, frictionless topology. The technology to eliminate space and capture time. 202 He labeled it: Donald G. Janelle, “Spatial Reorganization: A Model and Concept”, Annals of the Association of American Geographers (Volume 59, Issue 2, 1969) 203 Adam Smith’s: famous remark: Adam Smith and Knud Haakonssen. The Theory of Moral Sentiments (Cambridge, U.K.: Cambridge University Press, 2002) 198 204 “Time is a ride” Stewart Brand, The Clock of the Long Now: Time and Responsibility. (New York: Basic Books, 1999), 67 205 The German philosopher: Sloterdjik, TK 140 HOUSE_OVERSIGHT_018372

The computing programming technology that lingers inside most data centers now and that makes sense of the ceaselessly arriving bits - Here is information on a billion people. How many are named Bob? - is known as MapReduce. The name combines two well-know computer fuctions, Map and Reduce, and it is just the sort of program Janelle and Hillis would have dreamed up, had they ever met. MapReduce divides computation work among thousands of machines, just the sort of massive parallelism Hilli had in mind. What it implicitly does, however, is crunch what might have taken years into microseconds. If the charmed phrase of James’ stately Victorian elites more than a century ago was “Summer Afternoon,” ours might be “MapReduce”, a kind of magic two word code for a whole way of living and thinking. The most successful political and economic systems of the past, the ones that marked the winning “convergence” club of humanity’s last great leap, let people liberate themselves into a life of their dreams. Liberty meant tearing down old barriers to influence and security and knowledge; tipping over the Bastille, escaping colonialism. And it meant, too, providing a scaffolding of education, of social support, of laws and stability for citizens. The industrial, urban and rich countries that populate our world now evolved that way because their citizens made them so. They escaped the tyranny of the pre-modern, of a world where the small town you were born in and the pattern of your parents’ life were a kind of prision for your hopes. This was the whole trick of the Enlightenment, after all: What do you want to be? You decide! What do you want your nation to be? Decide that too. Dare to know! Look ahead now. The very best future political and economic arrangements will need to do something more than simply liberate us. They will have to enable and permit citizens to compress time. Just as the idea of a democracy was shocking once, this concept of a system tuned not merely for liberty but for compression of time should rattle us a bit. Certainly the attempt to achieve such an aim, to build the scaffolds and rails a time compressed world demands, will jostle much of what existst now. But to be modern means, in essence, the right, the ability to engage in compression. Nations or corporations or ideologies that can deliver this liberty of velocity - and that can construct and manage the flat-out fast, networked, artificially topologies such a life demands - will grow, thrive, accelerate even faster. The others, slowed by history or blocked by social or ideological designs, will miss the turn. They will be the new divergence club. Fast networks will elude them. Self-defense will be impossible; their time will be as vulnerable to easy manipulation and attack as the resources of China and Africa and Latin America were in the face of greedy colonialist plunder several hundred years ago. There is no equality of justice between the weak and the strong, the Italian historian Giambattista Vico once noted. So in our age. Between the fast and the slow? No 206 Vico, New Science 141 HOUSE_OVERSIGHT_018373

equality either. Partly this is because of something Janelle spotted: The remarkable advantage, the wealth and opportunity, that accrues to the people and nations and businesses that can compress space and time. Die Ware liebt das Geld, Marx famously wrote: Commodities love money. Speed is now the decisive commodity - and it loves money. (The feeling is mutual.) Velocity lets us achieve that ancient goal of doing more with less. The race for speed lights something competitive: The faster I go, the faster you feel you need to go, the more powerfully you feel your slowness. The centripetal charm of acceleration, the way that speed attracts us, and then makes us demand even more speed, honestly surprised the earliest architects of steamships and rail and airlines and roads. They under-guessed how popular their tools of space-time compression would be. Surely the maximum number of people who would ever want to zip from LA to New York would be about 1,000 per week, jet airline pioneers assumed. Would more than a few hundred engineers really want their own computers, Gordon Moore asked at a dinner party shortly after Intel proposed putting his chips in the first PCs. Yes, it turned out. Billions more. Highway designers call this surprise “induced traffic”: The faster a highway, the more people pile onto it. Urban planners in Los Angeles in the 1950s looked at their packed, congested roads and thought they could fix them by adding lanes. They embarked on construction programs, tore up the transportation network that girded the city and built a new one featuring optimistic 20-lane highways as wide as a football field and flat as a plate. Traffic got worse. Say’s famous economic law that “supply creates its own demand” seems especially fulfilled in an age where velocity is so valuable. Speed too creates its own demand. The faster any one piece of the network starts to rattle and move, the more profoundly we notice how slow the leftover bits are. We want them accelerated too. Total network bandwidth grew by an heroic factor of 1000 times between 2005 and 2010. Speed? It increased by just 20 times. Induced traffic. What is it, exactly, that we’re so hungry for? The extreme end of fast connectivity is what computer systems designers call “statefulness” —- a word that has nothing to do with states like nations, but rather with the condition of a connection, the “state” it is in. Early electrical circuits were either in a charged or uncharged “state”, switched on or off. Today when we talk about a “stateful” connection, we mean a link that we maintain, always on. It’s the difference between a video call and a letter, between looking at your wife here and now (a “stateful connection”) and a photograph of her (“stateless”). Real-time everywhere connection doesn’t merely kill distance; it attacks delay too. Older generations would “break state” when they left family at home or friends at school with a “see you later.” Our generation? “See you always”. We never quite leave. Technology permits us to remain in constant touch this way, to neve break state. “We don’t have a word for the opposite of loneliness, but if we did, that’s what I’d want in life,” the milenial writer Marina Keegan wrote ina famous essay that captured more than a little of this zeitgeist. “More than finding the right job or city or spouse - I’m scared of losing this web that we're in. This elusive, 142 HOUSE_OVERSIGHT_018374

indefinable, opposite of loneliness.”2°” The early interface of Snapchat, where you had to leave your finger resting on the screen in order for the video to unspool, was a kind of metaphor for this unbreakable relationship between touch and connection. (As was, in a different way, the diffident “out of my life” left swipe of Tinder.) “Good theories of the mind,” Hillis’ mentory Marvin Minksy once observed, “must span at least three different scales of time: slow, for the billion years in which our brains have evolved; fast, for the fleeting weeks and months of infancy and childhood; and in between, the centuries of growth of our ideas through history.”2°8 What we face now is a new time scale to add to Minksy’s list: The instant. Super-fast networks are different than slower ones - even ones as “slow” as what we have today. The ideal network is one that hovers near zero latency, where the time between what you want (or what the machines want) and the effect is as short as possible. You click a button to watch a movie and it starts instantly. You want to shut down an enemy air force, you do it with a single switch. The fantasy of a really “zero latency” system is impossible of course because even electrons moving through copper are not instant, but near-zero? Light-speed? You've probably heard stories of high-frequency stock traders who move next door to exchanges so they can capture and profit on an extra sliver of a millisecond. That’s the quest for low latency. (And more proof of the profitable link between speed and money.) Our challenge will not be about being faster - the technology will make that inevitable - it will be about managing the insane, still unknown demands of a world of suddenness. “In a distributed system, it is sometimes impossible to say that one of two events occurred first,” computer engineer Leslie Lamport wrote at the start of his famous essay “Time, Clocks and the Ordering of Events in a Distributed System” - a sort of technological parallel to Simmel’s “The Metropolis and Mental Life” written a century earlier. The problem, Lamport explains, is that what happens in one place can happen nearly instantly everywhere. There’s no time to react; so your entire worldview has to be capable of update at an instant’s notice. When the time between a Warez Dude finding an exploit and your own systems being compromised is zero, then the discovery of the hole and the creation of your vulnerability happen effectively at the same time. “The relation ‘happened before’”, Lamport explains, “is therefore only a partial ordering of the events in the system. We have found that problems often arise because people are not fully aware of this fact and its implications.” 2°9 Networks, we are discovering, don’t only compress space and time, they are compressing in the process the path to knowledge. We might call this “skill-time compression”: Techniques that once took a decade of training or that demanded access to million-dollar machines, can now be understood, applied, and then evolved 207 “More than finding”: Marina Keegan, The Opposite of Lonelinenss”, Yale Daily News, May 27, 2012 (accessed online). 208 “Good theories”: Marvin Minsky, The Society of Mind (New York: Simon & Schuster, 1986) 209 "In a distributed system”: Leslie Lamport, “Time, Clocks, and the Ordering of Events in a Distributed System”, Communications of the ACM, July 1978, 558 143 HOUSE_OVERSIGHT_018375

unimaginably fast. No one had ever heard of the Syrian Electronic Army a year before they were hijacking famous websites, injecting world-class malicious code into opposition computers and demonstrating a digital attack fluency. 21° Of course the charming side of such a shift is evident too: Walk the Vatican with an historian in your ear, master sourdough in a weekend. There’s something not a little miraculous in the way the networked tools to recombine DNA or hack computer code or design viral software are getting both more sophisticated and simpler. If earlier eras put epoch-making implements into human hands - the knife! the train! - our age is now placing new, mind-shaping forces within instant reach. In a networked world, economic or military power doesn’t come just from controlling territory or information alone. It comes, more and more, from this matery of the temporal. Securing territory today does not, alone, solve many problems of safety. Control of information, of topologies and finally of time - this is what matters most. Such temporal security will be elusive; always in need of defense. The arrival of airpower in World War Two, for instance, shifted battles from two to three dimensions. “Only large states are able to resist three-dimensional envelopment,” the historian Nicholas Spykman wrote in 1942.21! Even today, “Air Superiority” is the precondition of nearly any American war. If we can dominate you from above, nearly anything seems possible. But networks add a fourth dimension. “Time Superiority.” Can you move faster than your enemy? Can you bog them down? Or are you a victim of fourth-dimensional envelopment. Control of time - yours, your enemies - this will decide your strength. 6. Back in the fall of 1988, at about the same moment that Danny Hillis and his team were busy peddling their amazing Connection Machine - and trying to smash every world computing speed record they could find - another device appeared in the world of massively parallel super computers. It was, everyone who saw it agreed, an extremely strange machine. It’s appearance was completely unexpected. It’s designer was not a famous thinker about parallelism, cavorting with TV network founders and physics Nobelists. In fact, its very success emerged from this strange face: The creator knew basically nothing about the sort of parallel design that informed Hillis’ thinking. Which was strange, because it was far more “parallel” than the Connection Machine could ever be. It was also cheaper. Simpler. And: It was faster. In fact, it was the fastest parallel machine in history. The machine began, quietly enough, in the mind of a 28-year old Cornell graduate student named Robert Tappan Morris. Morris came by his computer chops honestly enough: He was the son of Robert Morris Sr., the legendary NSA scientist we encountered several chapters ago, the man who penned those partly amusing, partly 210 No one had heard: Edwin Grohe, “The Cyber Dimensions of the Syrian Civil War: Implications for Future Conflict”, Comparative Strategy (2015), 34:2, 211 The arrival of airpower: Nicholas John Spykman, “Frontiers, Security and International Organization,” Geographical Review, Vol 32, No 3 (July 1942) p. 439 144 HOUSE_OVERSIGHT_018376

terrifying Golden Rules of Computer Security.“Rule One: Don’t Own A Computer.” The machine that Morris Jr. created was madeentirely of software. It took the form ofa compact, simply designed computer program he’d written and designed to spread quickly and easily on the young systems of the Internet. It ran a mere 99 lines, took most computers nanoseconds to execute and it worked like this: The program - it later became known as a “worm” by the police who would come to find and arrest Morris Jr. - would find an open door on a network-connected computer. (In 1988, the pre-Warez Dude era, finding such doors was not difficult. Finding locked doors was probably harder.) Once Morris’ program had slithered inside and loaded itself onto the machine, like a dog slipping through an unattended puppy door, it would sniff around, rattle a few more doors to find any passwords that had been left unsecured. Then it would move on to the next machine. Knock, knock. Rattle, rattle. Next machine. Morris designed his code to simply repeat this process over and over. Filling, as a result, each machine’s memory with multiple, peformance-deadening copies of the same program. A house full of puppies, in a sense. After several hours of this flu-like spread, a wave of unplanned, unending computation began choking the net. Morris later explained he’d only meant his program as a demonstration, as a test of sorts. He wanted to show how machines might be made more safe. But he seemed to grasp, almost immediately, that he’d made a mistake and that the worm was running away from him. He emailed a friend: How the hell to stop it? His friend had no idea either. They scrambled at least to warn system administrators about the dangerous code that would shortly devour their machines. “There may be a virus loose on the Internet,” they wrote. But that note, in a bit of bad luck, was quarantined inside a Harvard computer that had already been unplugged. So, a few hours after Morris released his code, unwarned and unprepared, the Internet froze. 212 On November 2"¢ and 3rd, 1988 machines around the world were shut off, cables were pulled out of walls, and systems were wiped and restarted in a race to stop the robot-like spread of the disease and then to finally kill it off. 213 The Morris Worm was, on those fall 1988 days, acting out a sober-minded insight of the famed biological historian Alfred Crosby: “The nineteenth century was followed by the twentieth century, which was followed by the...nineteenth century.”214 Crosby meant that our age of topological connection meant we were back, again, in an age of infections. And this was true: Morris program was mapping out, like an epidemic, new routes in the age of high-speed digital contagion. But - and this is why we care about it here - by the time it reached “peak infection”, the worm was 212 So, a few hours: David Moore, Colleen Shannon, k claffy, “Code-Red: a case study of the spread of a worm on the Internet” IMW ‘02 Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment (2002), 273-284 213 Morris later explained: Ted Eisenberg, DavidGries, Juris Hartmanis, Don Holcomb, and M. Stuart Lynn, Thomas Santoro, “The Cornell Commission: On Morris and the Worm”, Communications of the ACM (June 1989, 12: 9) 706 214 The Morris program: Alfred W. Crosby, America's Forgotten Pandemic: The Influenza of 1918 2nd ed. (Cambridge: Cambridge University Press, 2003) xii 145 HOUSE_OVERSIGHT_018377

also doing something else. It had infected tens of thousands of machines, which were all cranking away in unintended harmony. During the 48 hours of its brief and unforgettable life, it was later calculated, the Morris Worm had become the most powerful parallel computer in history. At its peak, it managed to achieve a processing speed of 400 billion operations per second - about twice the speed of the most expensive supercomputers of the day. Like any unexpected epidemic, the worm became a social, cultural and technological milestone. First, it caused Morris to be arrested. He was handed a $10,000 fine, some community service and several years of probation. He later went on to found an important Internet company, to join the faculty of MIT and to receive the highest honors in computing for his (other) efforts. Then, a year or so after the virus had been finally corralled, the computer scientist Fred Cohen, one of the earliest specialists in malware - in fact, the man who invented the term “computer virus” - wrote an article that challenged the notion that all computer viruses are, inherently, bad.215 What drew his attention was that fabulous, unbelievable record of the Morris Worm: 400 billion computations each second. “The features that make computer viruses a serious threat to computer integrity,” he wrote, “can also make them a powerful mechanism.”2'6 This optimistic gloss, which sounded an awful lot like “The Plauge was great for humanity!” if you didn’t follow Cohen's logic, triggered a furious response. Eugene Spafford, also a well-regarded computer researcher, fired back: “For someone of Dr. Cohen’s reputation to actually promote the uncontrolled writing of any virus, even with his stated stipulations, is to act irresponsibly and immorally.”21” So here, then, is a line, of sorts. The Morris Worm, an expression of really massive connection and interaction and speed, is a model for the world we live in now. Fast, linking with a mind of its own, running best (or at least fastest) when it uses the design of the system to spread on a topology optimized for speed. But who is right about the implication of such an historic design. Cohen? Spafford? Do we want our whole world cranking away, super fast, compressing time to nothing? We can, all of us, decide to fight against the very ideas of a network. Or we can, like Cohen suggested, look at the terrifyingly fast nature of this world and begin to see something else. The totally unimaginable. Of course Spafford was not wrong. There is something scary and thought-muddling about the idea of intentionally authored computer viruses, running wild, ever faster. But there is something thought- muddling about this whole world we're entering. Networks are pulling at every existing structure. They hum, really, with the most elemental and precious human 215 Fred Cohen: Frederick B. Cohen. A Short Course on Computer Viruses (Pittsburgh, PA: ASP Press, 1990) 216 “The features”: Fred Cohen, “Friendly Contagion: Harnessing the Power of Computer Viruses”, The Sciences Sept/Oct 199, 22 217 Eugene Spafford: Eugene Spafford, “Three Letters on Computer Security and Society,” Purdue e-Pubs 1991, 91-088 146 HOUSE_OVERSIGHT_018378

data - our DNA, our wedding photos, our hopeful voice mails and most essential knowledge, our small savings against disaster. In its speed and its depth, in its increasingly comprehensive grasp of each of us and our world, this new network order is at once the most amazing thing we've ever created and the most terrifying. But recall, for a moment, Hillis’s dream for his parallel computer: “The ability to configure the topology of the machine to match the topology of the problem.” What if we really could rewire our thinking, our networks and politics and economics to match the problems we face now? Finally, with the Seventh Sense kicking alive in each of us, we can at least see the topological landscape where such a construction might occur. Now we must turn to the exciting question of just how we’d configure it to do our bidding. We've covered a lot so far; perhaps it is worth a glance back. First, we’ve come to see how networks really operate, the way in which they breed a lively connected skein that concentrates and distributes power. These systems are, as a result of their very design, plucking apart many of our old structures and ideas. Recall my dad, the doctor, ripped one direction by millions of disease advice websites (distribution) and in the other by massive diagnostic databases (concentration). This pulling process creates too, as we saw: Billion-user firms (and billion dollar fortunes) bred with breathtaking speed. Drones, derivatives, waves of migrants torn from their states but plugged into technological tapestries - all of these are products of network power. Connection, we learned, changes the nature of any object. You. Me. Money. Terrorists. Pretty much anything. And because there is a lot more of the world that has not yet been fully connected, it’s easy to see that we live in a revolutionary age. Second, we learned that the world of networks is complex. It’s made up of many complicated pieces, but complexity is something entirely different: It’s the unpredictable and colliding evolution you might see in a rainforest, where uncountable forces intermingle to produce life and death, growth and change. Scientists call this “emergence” - and it is happening on all our connected systems. Connection produces new, ripplingly powerful and unpredictable structures. The Arab Spring. The 2008 financial crisis. Connection changes complicated objects not least by making them complex. This is why our whole world, even the parts we might expect to be most stable, whips around now with a new and wild energy. Ina way, we Saw, this should reward a careful confidence: Go ahead and break the old systems. Something new will emerge. Third, we followed the trail of hackers like the Warez Dudes and discovered something unnerving about the networks around us. Not only are they honeycombed with dangerous and unpatchable holes, but historically unmatched amounts of power rests in their central cores. The reason hackers are so eager to get to these kernels of power is that in doing so they can manipulate entire landscapes of power with a profound efficiency. A “weird machine,” fired with an invisible and dangerous hacked logic, is a possibility that flutters through any network. Trade. Politics. Finance. Choices made in the center of network systems will redound on 147 HOUSE_OVERSIGHT_018379

each of us. This is why we say that if in the past the most important things happened in public - wars, riots, and elections - in the future many of the most powerful shifts will, rather worryingly, occur in secret. Fourth, we met the New Caste that masters many of these hot cores of power, for good and ill. Past eras were dominated by merchants, sages and soldiers who competed and collaborated in the pursuit of power. Now a new, young technological group is appearing. The nations and companies that train and equip them best of them will have an incalculable advantage. But there’s a hitch: As much as this group knows about networks, they know little about history or politics or economics. Dangerously, they often see the world as a machine to be coded. Fifth, we unearthed a new and invisible set of landscapes that will decide much of our future. These are called “topologies” and they are the connected fields on which power moves now. The web where stocks are traded, cyber attacks occur, imports are moved, or biological data are recorded and studied - each of these is a crucial topology. Control of them will be as important in the future as control of sea or air or capital once was. And, finally, we learned what the networks are for: The compression of time. For all their technical magnificence, we find that beating in the cold technological heart of these systems is a most human desires, to negotiate a bit the one really inarguable constant of our lives: We are all burning candles. The compression of time is why we connect. It lets us do more, experience differently, live longer. What the demand for liberty was to the Enlightenment, the call to compress time will be in our future - a fundamental political demand. None of our existing institutions have been built to answer this cry. These six elements make up a rough outline of a new sensibility. To see them at work in the world is the mark of a powerful way of thinking and feeling. The Seventh Sense. And this is important, we saw, because the shift ahead of us really is like the Enlightenment in its scale. It will tip everything over. It can’t be totally understood in advance. We can see now that our old institutions are failing. Their strategies for solving problems only makes them worse. We saw how, at that level of war and peace, old ideas are laying new traps. No one in power seems to have a clear, convincing picture of just what is going on. We can feel the danger we face, like rippling heat from a nearby inferno - our world tugged into the future by a class of old leaders who don't understand networks and a collection of new technologists who don’t understand the world. We had better find some way forward that does not depend entirely on either of these groups. The first may destroy our liberty in the name of an elusive security; the other will consume our freedom in pursuit of a mad efficiency. And there’s one last thing we didn’t really learn here, but I think you probably suspected it even before you started reading many chapters ago and met Master Nan and his warning of impending “spiritual illness”: If we’re going to shape this world at all, we don’t have much time. 148 HOUSE_OVERSIGHT_018380

Part Three: Gateland 149 HOUSE_OVERSIGHT_018381

Chapter Nine Inside and Out In which the Seventh Sense brings us face to face with the most powerful feature of our age - and perhaps of any age. 1. The Shangani River runs ina small, green vale through some of Southern Africa’s most remarkable nature. It marked, a bit more than a century ago, the northernmost line of the British presence in Africa. If other parts of Queen Victoria’s colonial empire crackled with desert harshness, the mountains and hills rolling up from the Cape and down to the Shangani were notable for a pleasing softness, a shading towards pastels in the changeable light of the region. For London’s colonial mapmakers, South Africa was a long-eyed treasure, an ideal restocking and transfer point for British ships headed for Lombok, Calcutta, Pondicherry and beyond. “We have lost America,” the explorer William Dalrymple wrote Prime Minister William Pitt in 1785. “An halfway house would secure us India, and an Empire to Britain.”218 The African Cape would be that halfway house. Following the 1814 Anglo-Dutch Treaty, which gave the British control of the Cape, the English pressed into Africa and found each newly opened district delivered more wealth than the last. A colonialist’s dream: Diamonds. Gold. Endless fertile fields. The efficient engines of industry and exploration (and exploitation) of the British African Company chewed easily, profitably, into the land. “Having read the histories of other countries, I saw that expansion was everything,” the mining baron Cecil Rhodes wrote in 1875. “The world's surface being limited, the great object of present humanity should be to take as much of the world as it possibly could.”21? And so the British did. If there was a moment that showed the tenor, the power of this ruthless asymmetry most clearly, it was the battle that exploded along the Shangani in 1893. The Matabele, a powerful local tribe had been smash-and-run fighting the colonists for years. The British had tried to charm, pacify and bribe the Matabele and their Chief Lobengula with money and land. None of it worked. They tried threats. That did not work either. “The Chief has had all your messages,” an imperial adjutant reported back to Cape Town after another frustrating, pointless discussion in late 1892. “But he has the art, not unknown to civilized despots, of ignoring what is not convenient.”22° Or perhaps the instinct of knowing what to avoid. One Boer 218 “We have lost America”: David Johnson, Imagining the Cape Colony: History, Literature, and the South African Nation (Edinburgh: Edinburgh University Press, 2012), 67 219 “The world’s surface being limited”: F. Vershcoyle Cecil Rhodes: His Political Life and Speeches, 1881-1900 (London: Chapman and Hall, 1900) 7 220 “The Chief has had”: Copies and extracts of further correspondence relating to Affairs in Mashonaland, Matabeleand, and the Bechuanaland Protectorate, Nov 1893, 150 HOUSE_OVERSIGHT_018382

commander, a blood enemy of Rhodes, had warned Lobengula: “When an Englishman once has your property in his hands, then he is like a monkey that has its hands full of pumpkin seeds - if you don’t beat him to death, then he will never let go.”221 So when, in October of 1893, the British finally tracked Lobengula down to the banks of the Shangani, the two sides faced off for what promised to be an intense, decisive battle. “It was just after 2:15a.m., a peaceful night, clear sky but on the dark side,” one of the British infantrymen later recalled. “The bugles gave the alarm, the camp was all excitement in a moment, all noise with the opening of ammunition boxes and shouting of officers, the men were getting into their places. There was a din outside from the on-rushing Matebele impis that had decided to attack in the usual Zulu fashion.” The British soldiers were outnumbered. They were thousands of miles from home, hanging on the thin end of a 5,000 mile supply line. The Matabele knew the territory. They were fighting for the lives and families and honor. But one sound was the decisive noise of the scale tipping towards the British soldiers. A hushed clicking against the yelling all around. The opening of ammunition boxes. The British, for the first time in African action, had mounted machine guns. The weapons worked that morning on the Shanghani with a violence you and I would have expected. They reversed, more or less instantly, the Matabele advantages of men, familiarity, and even furor. Machine-gunned Matabele were found, in the hours after the attack, perched in trees, dug into dirt mounds and piled desperately atop each other, killed as they had scrambled. One British soldier wrote later that that the weapons had mowed down the Matabele “like grass.” Lobengula survived, but his army was massacred down to a squad and he was reduced to pleading. “Your Majesty,” he wrote to Queen Victoria in the days after the battle, “what I want to know from you is: Why do your people kill me?”222 With this missive, the Chief entered the ranks of the Queen’s powerless correspondents, once- omniscient feeling men in Africa or Asia or India who wrote her after some devastating battlefield reverse - baffled, confused, overwhelmed. Did she even read the letters? It was hard to know, but that only made the pleading more perversely imbalanced. The locals had no idea, really, what they were up against. Martial leverage. It was the inarguable force of the 19 Century. It made Europe’s colonial masters. Of course they lied, stole, fought - did whatever sensible and sleazy thing Cecil Rhodes and his ilk suggested was needed. The monkey with the pumpkin seeds. Expansion was everything. Imperial dreamers in London, Berlin, Brussels, Vienna and Paris saw with total clarity the immense historical imbalance across the chasm of industry and science and reason. The “Convergence Club” Her Majesty's Stationery Office, Telegram from Assistant Commisioner, Palapye, to his Excellency High Commissioner, Cape Town, p 13. 221 “When an Englishman”: Vershcoyle, p 191-192 222 “Your majesty”: Report, Report, Volume 61 Sessional papers, (London: Commonwealth Shipping Committee, H.M. Stationery Office, 1893), 77 from University of Michigan online digitized library 151 HOUSE_OVERSIGHT_018383

against the divergent, the left-behind. Rhodes confessed ambition for his company to “take as much of the world as it possibly could,” was simply an armed, greedy version of Kant’s “Dare to Know”. Just as no question was was unaskable, no place was too far off to possess or exploit. No position was secured by history or distance or sentiment. This was, for instance, the lesson taught to Lin Zexu, the Qing Dynasty bureaucrat sent out from Beijing in 1839 to stop the British opium sales that were reducing China to a drugged, useless coma. “Suppose there were people from another country who carried opium for sale to England and seduced your people into buying and smoking it?” Lin wrote the Queen. “Certainly your honorable ruler would deeply hate it and be bitterly aroused?” Lin thought that he was speaking as the voice of a great, eternal empire. Victoria never replied. To the extent Victoria was aroused by anything in Southern China, it was likely by the way in which, a few months after Lin’s letter, the British emasculated the Qing military and moved into Hong Kong for a 150-year stay. “Whatever happens,” the Hilaire Belloc had his Colonial character Captain Blood famously quip in an 1885 poem, “we have got the Maxim and they ‘ave not.” 224 The machine guns were a totem of dominance in Shangani and on other colonial front lines; they marked a gulf between modern and unmodern, between industrial and agricultural. The weapons had first appeared in the mid-1800s on battlefields in the American Civil War, after the inventor Richard Gatling sent a package of samples to the White House and convinced President Lincoln - a famous gadget freak - that their firepower might bring the Civil War to a faster close. Lincoln ordered the Army to try the guns, but Gatling’s early attempts were honestly too immature to tell decisively on the battlefields of the American south. Within a few decades, however, the guns were perfected in places like Africa, or on the frontlines of the 1904 Russo- Japanese War. They represented a compelling, inarguable logic of industrial war: A machine and a gun. We mowed them down like grass. You could read that line as metaphor: Mowing grass was, in the end, the act of a machine killing a wild, natural world into a clean, useful order. The Europeans were the grass-clipping machine; the rest of the world was, well, the grass. For a colonial temperament hardened for “The Great Game” of empire on the playing fields of Eton, preparing a lawn for tennis and a territory for exploitation were not particularly distinct acts. The Shangani battle gunshots struck the European mind as powerful confirmation of everything they suspected about the magic violence of an industrial age. The image of such a device suited the aggressive, engineering-led mood of era. As Gatling - and his competitor Hiram Maxim - peddled the guns, they faced a predictable resistance, of course: Europe’s cavalry officers were in love with their well-bred horses. But the age was, finally, the story of aggressive industry flattening old habits. Trains were assaulting the countryside. Factories were pounding apart the habits of labor. Social stampedes of speed-climbing nouveau riche, political attacks of new industrial unions and counter attacks all expressed this aggressive energy. The soundtrack of Germany in the decades after 1869, as Bismarck stitched together a new nation 224 “Whatever happens”: Hilaire Belloc, The Modern Traveler, (London: Edward Arnold, 1898) 152 HOUSE_OVERSIGHT_018384

from dozens of hereditary principalities, was the ceaseless ping-ping of rail-building and welding and industry. How natural it must have seemed to add the rat-a-tat-tat of a Maxim gun. Crown Prince Wilhelm, the Kaiser's eldest son, wrote that defensive thinking was, “utterly foreign to the German spirit.” Bismarck’s “Iron and Blood” national motto became, finally, personal for many Germans, who were prouder to leave their universities with a hot-red dueling scar on their faces than a subtle ownership of Goethe in their hearts. “During the decades before the First World War,” the political scientist Steven van Evera has observed, “a phenomenon which may be called a ‘cult of the offensive’ swept through Europe.”225 Wars, it was believed, would run with the same swiftness of trains or the new industrial sewing machines or steam-fired printing presses. It was this instinct that led German generals to assure the Kaiser in 1914 that a war begun in August would be finished by Christmas. English university students sprinted to enlistment centers in the days after the war began, worried the fight might end before they tasted blood. French farmers moving from their crops to the trench lines of Flanders, Russian aristocracy crowding towards the Danube, the politicians who led them all - they operated, mostly, with this same conviction. British Foreign Secretary Sir Edward Grey’s mournful meditation on the evening of August 3,1914, the first night of the war, was a lonely one: “The lamps are going out all over Europe,” he said. “We shall not see them lit again in our lifetime.” The First World War was a kind of engineering tragedy. The disaster had deep roots: domestic politics, the insecurity of kings, profound colonial greed. But alsoa fundamental miscalculation about the nature of war and peace in an age of industry. Machine guns - and all the tools of industrial war, from gas to battleships - were not magic tricks of fast victory or permanent peace, as some had thought. A mechanized modern army wasn’t, as much as it may have seemed in theory or drills or in midnight massacres in the African bush, some steam-press built for cold rolling the armies Belgium and Prussia and France. In fact, the weapons encouraged battle as they piled endlessly in national arsenals. They tickled fears of fast or surprise attack, even as they gratified that weird continental hunger for violence. Hilare Beloc’s poetic joke, the sly we’ve got the Maxim, and they've not took on an unexpected character when both sides had ‘em. The machine gun reached the fiery acme of its purpose not as a spur to end wars altogether, as Gatling once hoped, but rather when it was married to barbed wire, to shovels and to gas - and then admixed with the trigger-tugging fear of 20-year-old boys. So: Sixty thousand British casualties in one day alone, July 1, 1916 at the Somme. The rhymes changed. Hilare Beloc’s jigs were a distant memory for the starved, surprised and shocked men in the field. Siegfried Sassoon: You smug faced crowds, with kindling eye Who cheer when soldier lads march by, 225 “During the decades”: Steven Van Evera, “Cult of the Offensive and the Origins of the First World War”, International Security (Vol 9, No 1, Summer 1984) 58 153 HOUSE_OVERSIGHT_018385

Sneak home and pray you'll never know The hell where youth and laughter go. *7@ As soldiers dug into trenches that would endure for a half-decade, a terrible strategic fact dawned on the generals who led Europe’s armies. The Great War was going to bea charnel house. The continent had built itself into a battle machine, wired by trains and telegraphs and armies. There was no reverse gear. There was not even a switch to slow it down, let alone turn it off. A massive, technology- powered, fast-moving system with revolutionary implications, built beyond the comprehension of any one figure or nation, had slipped out of control. And the men in charge of planning and directing the use of this super-fast complex? They failed everyone: their soldiers, their kings, their armies. They were all but insensible to the real nature of their age. Sound familiar? 2. Here, then, is a question of the sort - violent, loaded with the possibility of tragedy - that you'd rather not have to consider: A new way of war arrives, a new weapon, a fresh idea about fighting. Does it make your world peaceful or treacherous? The lethality of the equation of guns x machines at the end of the 19 Century appeared to some industrialists and bankers and statesmen inarguable evidence for peace. Everyone with such a violently efficient weapon; who dares start a war? As we now know, machines x guns was a formula for some of the worst killing in human history. Gatling’s fond hope that his weapons would stop war was naive, insane even. His competitor Maxim had been clearer eyed. A friend told him: “Give up your chemistry and electricity. If you want to make a pile of money, invent something that will enable those Europeans to cut each others’ throats with greater facility.”227 So: Let’s be a bit warmer about this. Networks x weapons = what exactly? Is there some disaster lingering in our own future, as unimagined from our current perspective as machine guns and trenches were a century ago? Do we consider war impossible now? There’s something sickening in such puzzles, of course. Think of the men and women who, over the millennia, have contemplated similar questions knowing full well the answer would be measured in blood and treasure and children. Put yourself in the place of the population of Melos, a peace-loving Mediterranean island whose destruction 2300 years ago was chronicled by Thucydides in The Peloponesian Wars. “Surely you have noticed that you are an island and we control the ocean,” an unwelcome Athenian general intimated to a Melian citizens’ council one day 243 BC as his soldiers and ships collected menacingly outside the city’s walls. Athens wanted the Melians to join an alliance 226 “You smug faced”: Seigfried Sassoon, “Suicide in the Trenches” in The War Poems of Seigfried Sassoon, (Mineola, N.Y.: Dover Publications, 2004), 64 227 “Give up your chemistry”: John Ellis, A Social History of the Machine Gun (New York: Pantheon, 1975) p.33 154 HOUSE_OVERSIGHT_018386

against Sparta. The Melians - like poor Lin Zexu or Lobengula of the Matabele - yearned only to be left alone. “You would not agree to our being neutral, friends instead of enemies, but allies of neither side?” they asked??8. No, the Athenians replied, and then a line that has resonated through the problems of nations ever since: “It is the nature of power that he who has it takes; he who does not must submit.” The Melians voted stubbornly against surrender and for hope. Perhaps the Spartans would mount a relief raid? The Athenians might change their minds? Neither happened. The Melian men were betrayed and then massacred. Their wives and children were sold as slaves. What do networks do when they touch the balance of war and peace? How might we use what we know, what we sense, about a connected age to manage the dangers ahead? If an insane Cult of the Offensive flavored the end of the 19 Century, our own age vibrates, as we've seen, to a Cult of the Disruptive. The great tale of our times is the diffusion of a new, promising and disorienting network order. We've been told that all this interconnection makes war an impossibility. Everyone would be a loser in such a war. But the way in which that earlier age was so horribly wrong about the result of machines x weapons, should unnerve us. We don’t yet really know what networks x weapons means - to say nothing of networks x networks x weapons. Or, to sum up what we've seen so far in this book, very fast networks x artificial intelligence x black boxes x a New Caste x compression of time x everyday objects x weapons. Would you look at that weird formula and say conclusively: “Hey, we'll all get along.” Me neither. We should worry about the day we might face a Melian choice of our own, when some general or infomanagerial despot - or some clicking computer - shows up, unwelcome, and says to us: It should be obvious you are merely a node, and I control the network. When leaders label the rise of China or cyberweapons or terrorism or the decline of the US as the “main problem” of our age - and all of these have been designated as such by famous foreign policy figures -they are missing the revolutionary, uniting force that animates them all. Networks. Whether we are trying to slice apart the roots of the Islamic State or slow Russian territorial dreams or understand narco- economics or hedge-fund finance - connectivity touches and defines each problem. New and essential platforms for finance and biological data and artificial intelligence are emerging now, blossomed out of network connection. These ecosystems must be design, built, protected. All while the world we know kicks back around us, sometimes crumbling, sometimes fighting for its dear life. ] promised earlier that we'd apply the Seventh Sense to some practical problem, and the deadly test of war and peace is the one I'd like to consider now. The sharpest challenge of truth for any view of the world is, after all, the design of a grand strategy. Get it right and you can secure your Safety. The energy of the age can be your tool. Get it wrong and you reap the Somme, Melos, Canton, the Shangani. Recall General Liu Yazhou’s line: “A major 228 The Melians: Thucydides, The Peloponnesian War, Steven Lattimore trans., (Indianapolis: Hackett Publishing, 1998) Chap 5.; See also the excellent BBC performance of certain elements of Thucydides in “The War That Never Ends” available on YouTube 155 HOUSE_OVERSIGHT_018387

state can lose many battles, but the only loss that is always fatal is to be defeated in strategy.” The leaders of our major global powers may be as blind to the dangers and possibility of our world as Europe’s heads of states were to the nature of their era 100 years ago. You know what the Seventh Sense is now. Who among them has it? We should wonder if we’ve merely now done Hiram Maxim one better: Have we developed something with our age of connection that will reach its fullest potential in allowing us to slit each other’s throats more efficiently? I don’t think so, but understanding why means we need consider the networks, to feel them out with our new sense. The essential problem of politics is not difficult to state. It was true for Seneca in the Roman Forum as it was for Lobengula on the Shangani riverbank or is now for congressmen in Washington or cadres is Beijing: Who has power? Why? By power | mean the ability to control others, to tell them what to do - or what not to do; and of course to avoid being dictated to yourself in that “Surrender or die” way. Max Weber, the German sociologist of the last century, had it right: Macht, the ability to achieve what you want despite the resistance of others.22? Don’t develop nuclear weapons. Or Don’t attack us. Or, Join our alliance against Sparta are all examples of power in action. The movement of power, from the balled-up fists of Kings and Popes and Emperors, to the hands of markets and voters and citizens, and now into a fresh, connected dynamic is the story of history. This is a lurching, worrisome, dangerous process even if it suggests some miraculous transformations. Whole new topologies of vital, life-giving control are emerging; surely contests for their mastery will occur. The terrible human infection of a hunger for power and security isn't, much as we might wish, something quarantined safely to the pages of Homer or Hitler. So let’s pass beyond describing the new and surprising elements that make up a Seventh Sense and onto its use. Examine our world with this instinct, for a moment. Picture that constantly stretched, complex network, spitting out drones, viruses, fortunes and disruption as it is wont to do. Recall the seething, insidious power of the Warez Dudes, the New Caste, the black boxes they all hunger to penetrate and make weird. Think of the compression of time, of twisting topologies that can place the distant atop us in an instant. What single feature stands out? Our world shuffles now, as all these forces slip into an order defined by closed spaces, by fresh in and out borders. What the Seventh Sense reveals as it feels at this new arrangement is gates. Everywhere. Facebook, bitcoin users, doctors with privileged access to genetic databases - all are gated, in-or-out worlds. Look around and see how many gates enclose you or your family or your company. The Internet. The FTSE 100. Your Apple or Android operating system. In our connected age, the act of drawing lines 229 Max Weber: Max Weber and Sam Whimster, The Essential Weber: A Reader. (London: Routledge, 2004), 355 156 HOUSE_OVERSIGHT_018388

between points is also an act of drawing a line around those points. It is not simply that we’re enmeshed in networks now; no, we're enclosed even entrapped by them. If the great ambition of Cecil Rhodes’ era was for the expansive conquest of territory, in our own it is for the construction and manipulation of gated spaces. Gatelands. In an age of network power, no position is more important, formidable, influential or profitable than that of the gatekeeper. Defining who is in or out of any network is among the most essential moves of design. In financial markets, on the Internet backbone or inside the human immune system, the accept-or-reject decision determines a great deal. The first sign of order breaking down, whether it is the Roman Empire or your lungs, is an inability to manage what slips in and out. Flows of bits, of migrants, of gold and patents and medicines -— all of these life-giving forces can be controlled, bent for good or stopped for ill, as they pass through or collide with gates. By gates I mean not only in-out passages but all of the tools that meter and enclose the various Gatelands: Protocols, languages, block-chains. Whatever binds and shapes an information topology. Any sort of code or encryption or binary instruction that can unlock an in and out. If you want to make a fortune or a revolution (or both), if you hope to shatter some barrier of tools or ideas between you and a dream, or to lead a religious revival, spread an infection of hate or revolution or insidious Morris-style computer code — then fundamentally this is what you have to consider: Where are the gates? How to smash them? How to build your own? We are entering, as a result of our dependence on networks of all sorts, a landscape where the very clustering of power creates new border regions, bridges and gates. If older, hierarchical systems craved a top - a king, a superpower, a Pope - our connected, meshlike age demands valves and protocols and gates. It hungers for connection, which means it hungers too for throttles and accelerators and brakes. And of course people to run them. To speed them up. Slow them. Finger off switches. This reordering of power will produce, is producing, a fight over topological spaces for finance, biology, trade or pretty much any source of power. The scramble is as urgent, decisive and essential as the one Rhodes and his peers embarked upon. Inside or out? A fresh Meliananxiety haunts us: Are you the gatekeeper? Or the gatekept? We wander into Gateland the moment we switch our phones on. We enter it when we book an airline ticket, when our genetics folds into a pool of data, or when we take a new degree, master a computer language, or check on friends via one stateful connected platform or another. We enter it, in short, when we connect. Gatekeepers choose what we see. They determine the rules we follow, what we can and can’t change. They reward us too - once we're inside — with benefits of speed, knowledge and safety. Gatekeepers, which can be people or protocols or code, decide who can join closed communities and who is left out and why. They pass us the fine benefit of the compression of time, even as they expose us to the zipper of instant potential disaster hitting everyone in a closed space at once. Gatekeepers control, for instance, how (and how fast) financial data moves between members of light-speed “in the know” trading pools, and the suckers outside. What you can see in your phone or your university computer, why, how - all of these choices have to be made by 157 HOUSE_OVERSIGHT_018389

gatekeepers. They can, if they wish, manipulate any step of life inside their enclosed orbs of power - and by extension, they can twist data and machines and you. The line between perverting search results and election results is a thin one. Such a manipulation of data is trivial, which means the manipulation of you and 1 is, technically at least, trivial. It involves the laying of our preferences - what do we usually read, watch, who do we talk to, where do we live - against machine logic and vast data fields. Manipulation of data = Manipulation of us. The idea of gatekeeping first emerged as it related to newspapers, back in the 1920s when politicians, advertisers and a few social scientists watched a print-information explosion - and developed an uneasy feeling about how the world looked through many newspapers. The personal whims of an editor, his political bent, his boss’ economic interest - all of these laid on “facts” like a heavy distorting blanket. Minor twitches were turned into fear-mongering bait. Major global shifts were ignored. The papers are (nearly) gone now, of course. And the idea of gatekeepers merely bending headlines to suit a personal whim seems charmingly nostalgic. Gatekeepers today have a far more profound, subtle reach. They might be governments or regulators or CEOs or machines or research committees, each controlling the design and development of some “rich get richer” tool we depend on and - by marking that “in or out” line - exerting historic, invisible control. Genetic engineering secrets. Unthrottled data flows. Product releases. Do you want accurate DNA analysis? Fast protection from epidemic? A cyber-defense system? You can’t have any of these, you know, unless you’re in someone’s fiercely guarded gateland. Even systems that look open — the Internet, the world of US Dollar transactions, the election rolls - are gated in certain ways. Of course there is — at times - a balance between the gatekeepers and the gatekept, between those of us inside a system and the tools and people who overmaster us. “In many cases, traditional literature focuses mainly on gatekeepers as elites who hold power in their hands while the gated are treated as powerless,” the information theorist Karine Nahon has written. “In networks, however, it is necessary to give sufficient weight to the role of the gated, since being subject to gatekeeping does not imply that the gated are powerless, lack alternatives, or that gatekeeping is forced on them. Actually, being a gated sometimes is a matter of choice.”229 But sometimes, of course, it is also a matter of inarguable necessity. In the slower, less wired worlds of our past, gates mattered too of course. Nations, governments, militaries, religious orders - all of these clustered behind (and defended) marked lines. Map lines, front lines, dogmatic lines. The Triple Entente that bound Britain, France and Russia together in the last century was as mucha gated system for their own security as the Peloponnesian League 2500 years earlier had been. Deciding who could swap silk for spice beyond the Tang Dynasty’s border was a gatekeeping choice, as consequential for Chinese strategists of the 8 Century as the decision about what might or might not be wheeled into the city was for the 230 “In many cases”: Karine Barzilai-Nahon, “Toward a theory of network gatekeeping: A framework for exploring information control.” Journal of the America Society for Information Science 59(9), 1493-1512. 158 HOUSE_OVERSIGHT_018390

tragically uncareful councils of Troy. But now, in an age where connection decides so much, control over gates has a unique leverage. When you finally can feel out the topology of our age, when in anger or frustration or hope or wonder you are ready to act, then this is among the first questions you have to ask. I’m on the topology now, where are the gates. Gates in an age of instant, everywhere, smart networks are, you can imagine, different from the ones that girded Troy or the Tang dynasty. It’s not merely that they’re made of bits and algorithms not bricks, it’s that the underlying nature of their power is different. The most visible evidence of this distinction was first observed by economists a couple of decades ago, as they contemplated the fortunes of the information age, wealth that had been assembled at an eye-watering pace. Unlike traditional businesses which turned over time into competitive slugfests with very low profits, many high-tech firms seemed to run with a new, nearly inverted logic. “Our understanding of how markets and businesses operate was passed down to us more than a century ago by figures such as Alfred Marshall,” the economist Brian Arthur wrote in the Harvard Business Review in the summer of 1996. “It is an understanding based squarely upon the assumption of diminishing returns: products or companies that get ahead in a market eventually run into limitations.”23! Marshall had been the first to name this phenomenon in the 1890s: “Diminishing Returns”. As any line of business gets more competitive, the profits - or “returns” to investment - shrink. Henry Ford invents a car, he has no competition at first and fairly prints money. But Ford doesn’t enjoy his monopoly for long. Pretty soon the Dodge brothers follow him into business, as does Walter Chrysler and then a cascade of new auto companies. They all take a piece of the pie; profits for every carmaking firm diminish. Then the Japanese pile in. The Koreans show up. These new companies compete with growing intensity. Marginal profits decline for everyone. Then the Chinese. And the Indians. As he studied the balance sheets of infotech firms, Arthur noticed something strange: Returns were increasing over time. As their markets matured some companies made more marginal money with each passing day, not less. Marshall’s 19t Century industrial economics had never contemplated such a lucrative arrangement. “Increasing returns,” Arthur explained, “are the tendency for that which is ahead to get farther ahead. They are mechanisms of positive feedback that operate—within markets, businesses, and industries—to reinforce that which gains success or aggravate that which suffers loss.” In other words: Winner takes all. No second place. Arthur was thinking, as he wrote, about the then-nascent computer software business. Say for instance, Arthur sent you a copy of his paper to read before publication as a Microsoft Word document. Well, if you wanted to see what he had to say, you'd pretty much have to own a copy of Word yourself. If you then 231 As he studied: Brian Arthur, “Increasing Returns and the World of Business” Harvard Business Review (1996) 159 HOUSE_OVERSIGHT_018391

sent it along to some friends for their input, they’d be in the same bind. One after another, in just this fashion, users tumbled into the program. It became a standard, a “platform” in industryspeak. And Microsoft enjoyed a particularly appealing economic leverage: Developing Word may have cost millions, but once that work was done, each additional copy cost just cents to produce. This astonishing speed loop of profitability demanded a whole new economics. It also forced a reconsideration of what “competition” might really mean. Once Excel or Windows had settled into place, had become a standard, you couldn’t really compete with it. New, optimistic maybe even better rivals rushed into the marketplace, but they were all assaulting the impregnable wall of habit, of a locked-in technology. Should this be legal? Arthur wondered. Traditional economics said such monopolies were bad for everyone. (As did the Department of Justice and their global peers as they chased Microsoft for a decade.) But was that right? The “platform dividend” that accrued to Microsoft was surely large, but if you could somehow total up the benefit to the rest of us? The convenience, the efficiency, the benefits of Microsoft's billions of research spending might dwarf even Redmond’s massive profits. “Increasing returns,” Arthur wrote, “cause businesses to work differently and they stand many of our notions of how business operates on their head.” The essential phenomenon Arthur spotted at work two decades ago is something we now know as “network effects” - an idea that changed how we think about businesses, and particularly about the sticky and alluring power of gated, connected systems in nearly any setting. In the years after Arthur's paper, billions of us ran madly along a course he had anticipated: We crashed our way as fast as possible into those single, winning businesses - rewarding them with near monopoly positions in exchange for the benefits of being “inside”. In the twenty years since Arthur spotted increasing returns in software, eight different billion-user worlds have emerged - and others are not far behind: Microsoft Office and Windows, Google Search, Google Maps, Facebook, Google Chrome, YouTube and Android all exhibit that appealing, “If you use it, I’ll use it!” logic. Profits and power, just as Arthur would have expected, followed right along. The most valueable company on earth in the early age of connection was one that, particularly, had perfected the idea of a closed iWorld where the efficiencies and charm of inclusion made outside assault difficult. Younger companies like Instagram, WhatsApp, Weixin and others hover not far from a billion users and point us to this gated model spread and repeated. It is easy enough to imagine a world where billions will be enmeshed behind certain gates. It was just as Arthur predicted: If ten people use WhatsApp or Facebook or YouTube, it’s hard for the eleventh to do something different. And when the eleventh person joins in, they make it harder still for the twelfth to walk a unique path. So: Windows runs on 90% of the globe’s PCs, nearly 30 years after its first release. Google has 65% market share. Android runs on 81% of new phones. WhatsApp neared a billion users with less than 50 engineers on staff. Facebook passed a billion connected people and faced no real competition. The rich got richer. 160 HOUSE_OVERSIGHT_018392

How? “Seven friends in ten days,” Facebook growth hackers repeated like a mantra in the early years, a humming meditation that carried them from dorm room to nearly every corned of the world??2. If you or I joined the service and found seven friends in ten days, we would most likely stay, enjoying the benefits of the gated world, making it that much harder (impossible really) for friend number eight to wander somewhere else. Pretty soon, there was essentially nowhere else to go anyhow. The network magnetism worked so well that, as a result, Facebook’s speed- looping connection machine cut the famous “six degrees of separation” posited by Stanley Miligram - the number of leaps between any two people on the planet - to four.233 Network theorists who came after Arthur call these “rich get richer” systems “power law distributed” because if you line up all the firms in a digital industry you find the winners are exponentially - by a power of ten or one hundred - ahead of everyone else. They slip free from the average gravitational center of a normal bell curve that marks most traditional business. A normal distribution would shape up like a chart of people who own cars: 20 percent driving Fords, 10 percent Nissans and Toyotas, and so on. Or it might look like the distribution of height: Most men are between 5’7 and 5’11, but 50% are scattershotted at different heights. Network systems, however, can breed commanding winners. It’s not like 50% of online users are on the Internet and others are scattered across different systems. Users huddle into single winning clusters. It’s as if 90% of the world always bought a Ford; or 90% of people were exactly 5’ 11”. These systems run faster and better and more profitably because they are locked-in, gated by technology standards and by common connection. When we say that networks crave gates, this is the sort of gate we mean. If you had to look for your friends one-by-one on Facebook, Friendster, MySpace, and GooglePlus you'd exhaust yourself. So, one winner emerges. Data scientists attribute the success of these winning nodes to “preferential attachment” - the idea that if Brian Arthur is using Microsoft Word and I’m using it you are likely to do so too. But there’s another secret: More widespread adoption makes the whole system faster. Winner-take-all marks that network hunger for the compression of time.?34 There’s an additional feature at work in the very newest of these billion-user clusters that’s worth our attention: It’s not merely that we’ll use them because everyone seems to be doing so, it’s also that as more users weave themselves into each others’ lives and the machines into too, these nodes of power get smarter. Google Maps can predict the fastest route from your house to your office because it can watch the movements of hundreds of millions of users, each silently pinging their location and speed as they creep through rush hour or sprint down an empty motorway. As more people use GPS enabled devices, the quality of this data gets 232 The network magnetism: Chamath Palihapitiya, “How we put Facebook on the path to 1 billion users.” Udemy Growth Hacking: An Introduction lecture published January 9, 2013 and available on YouTube. 233 Winner-take-all: van der Hofstad, p 24 234 But there’s another secret: Albert-Lazlo Barabasi “Network Science”, Philosophical Transactions of the Royal Society A: (2013) 371 161 HOUSE_OVERSIGHT_018393

ever better, like a video resolving itself from low-quality to HD in front of your eyes. Success attracts still more users. All of them are Google’s sensors, in a way. Medical diagnosis, cybersecurity, trading algorithms, search - pretty much any linked ball of chips and humans and sensors throbs with this logic. The best of the leading technology firms understand the power of this logic: Google’s TensorFlow artificial intelligence engine, for instance, was largely regarded by experts as nearly a decde ahead of competitors in 2015. So the company began giving away access for free. In traditional economic terms this would be insane; but with network logic the strategy is clear: The more people who use TensorFlow, the smarter it gets, which in turn attracts still more users. Dense and learning fusions of mind and data like TensorFlow and other soon-arriving AI systems are all gated universes. The “increasing returns” for those inside - you, me, our neighbors - breed mutual efficient success and, of course, massive power for their owners. We're part of the game too: The more people tied in, the better our lives get. The topological charm of these explosively growing clusters was first teased apart by the electrical engineer Bob Metcalfe in the 1970s. Metcalfe was hunting fora better way to send data - say grocery lists to his wife - through Menlo Park and he perfected a connection protocol called Ethernet, which soon became a standard for linking machines. What Metcalfe noticed, as more and more users piled into the gateland of Stanford’s Ethernet-connected machines, was that the power of the system was growing exponentially with each additional user. This became known later as “Metcalfe’s Law”: The power of a network grows, massively, with each additional user. A system with one phone, for example, is really not very useful. Who would you call? A system with two phones means one possible connection - we can call each other. But when you increase the number of phones by a factor of two - from five to ten, say - the number of possible connections more than doubles from ten to 45 . The difference between Bob Metcalfe and his wife sharing grocery lists and a connected national network of husbands and wives is immense - an insight that led Bob Metcalfe and his wife to start a networking company that made them billionaires. Metcalfe’s Law has another angle, and it’s here where some of the unnerving, dangerous political power of network gates is revealed: It’s not merely that the power of a network grows exponentially with each additional user; it’s that the cost of being cut out grows every bit as fast. Maybe even faster. If | shut you out of Google today, it’s painful. But tomorrow - after a day of new information and websites and services come on line - it will be even more costly. The network scientists Rahul Tongia and Ernest Wilson have called this “The Flip Side of Metcalfe’s Law.”235 To be excluded from a database of cancer genetics, for instance, when it has a million members is probably not such a painful problem; to be locked out of the chance to compare your genes with a billion others, however, is hugely costly. Maybe even fatal. Imagine if I cut you out tomorrow from the NYSE, your phone system, smart diagnostic webs, cybersecurity patches? It’s not like you can swap our your Ford for 235 The network scientists: Rahul Tongia and Ernest J. Wilson III, “The Flipside of Metcalfe’s Law”, International Journal of Communication 5 (2011), 665-681 162 HOUSE_OVERSIGHT_018394

a Dodge. There is nowhere to go. Gatelands produce “winner take all” systems, but they also produce these “loser gets nothing” dynamics - and an absolutely chilly, fatal cost of separation from the winning system. A discipline of network science known as “queing theory” helps us understand why. In studies of massively connected systems, the more time machines spend on their main task - hunting prime numbers or DNA patterns, for instance - and the less time chattering with each other about how they will compute, the faster they run. Winning protocols avoid this terrible inefficiency of translation because it can be spread across so much connection. In fact, the great breakthrough of computer systems in recent years has been the ability to handle massive amounts of data all at once, to maintain versions of information in a concurrent state many places in the world. This is the essential technical leap that permits compression of time. And it depends entirely on careful and gated design. To be inside a gated system is, then, really to be faster because of the slickness of communication that becomes possible. The very structure of the system accelerates that compression of time. This design feature of networks, in which winners take more and more, is why gates, and their careful use, will become the most dramatic lever for business, research or international politics in coming years. It also explains why our modern gates are different than older ones. Why it is so damn costly to leave them; why mastery of them is even more insanely lucrative than Cecil Rhodes’ gold mines. Think of the old industrial age power games for a moment: Britain and Germany tried to match each other with their industrial output during their fatal competitive sprint 150 years ago; but imagine if network effects obtained? If Britain’s initial head start in the industrial revolution had given them 90% global trade share? Germany would never have even tried to compete. They would have been the Friendster of the 20 Century: Isolated, slow-growing, powerless and finally consumed by the winning system. Networks crave gates. Once a billion people were connected together, of course there was going to be something like Facebook, a Gateland where they could link one to another in ever-thicker cascades of connection. Once everyone could record and watch and share videos, something like YouTube was inevitable. As we try to picture the world ahead of us, as we try to ask what tools of power we can acquire to twist this dangerous landscape into something we can manage and predict and control, we must ask of it: What does it want? The world wants a protocol for the fast exchange of money. It wants a basic language protocol. It wants a place to swap information about IT security holes. It wants instant translation systems to replace the need to learn to English or Chinese or Spanish so the world can move yet faster. | believe it wants certain sorts of alliances, a particular type of superpower and even craves a new form of politics. For any nation that controls these gates, there is a possibility to use that position to create still more gatekept platforms, to shape the protocols that tie platforms together in the way roads or jet planes link the physical world. This is the iron law of Gateland: Connectivity is power. Which means that gatekeeping is, at the end of the day, our most powerful point of a control. 163 HOUSE_OVERSIGHT_018395

It won’t surprise you that, in recent years, for instance, the world has seen an acceleration in the construction of physical barriers, of fences and walls running between nations and defining and in and out. Roy Hassner and Jason Wittenberg, two American political scientists, scored out the pace of global wall building and found a sharp acceleration: In fact, of 51 national enclosures built since the end of World War Two - the Berlin Wall being the most famous example - more than half were constructed in a rush of self-protection between 2000 and 201426, And more are coming: Hungary, Kenya, Algeria and India now posthole their borders in initial exploration of what might be built. There’s a frantic urgency to some of this. The Spanish government, for instance, raised a ten-foot high, razor and camera topped fence around their Saharan footholds in 1998 - the enclosed land was controlled by Madrid, so it was technically “Europe,” which made an irresistible target for would- be migrants. The fence wasn’t enough to stop the flows. So they built a second one to run around the first in 2001. Then, in 2005, thousands of desperate Africans launched a coordinated charge against. A couple of dozen migrants died in the attempt; a thousand made it through. The Spanish responded with a third line of fence, this one 20 feet high, electic, camera-watched. This pattern of ever more stacked defense is repeated everywhere. The walls, fences and trenches of the modern world seem to be getting longer, more ambitions, and better defended with each passing year, Hassner and Wittenberg concluded. Unlike traditional lines of defense, the Maginot Line or the Great Wall of China for instance, the aim of 21st century barriers in places like Israel or the US or Spanish Morocco have been less to stop a rolling armor blitzkreig than to slow the insidious movement of smugglers and spies and criminals, or the hopeful dashes of fleeing refugees. There’s an affective and - to those on the inside - appealing asymmetry to these borders. They are mostly marked and built by richer, more modern, more stable nations desperate to control in and out. The creation of gates is, we should sense now, the corollary of connection. That they can sometimes be piled into the “winner take all” efficiencies Brian Arthur first teased out should give us a sense that there is a logic to this emergence. Reviewing the problems of deadly disease contagion after the 2015 Ebola pandemic, Bill Gates examined this connection-and-gate lemma in the sharpest, most worrisome historical terms. “There is a significant chance that an epidemic ofa substantially more infectious disease will occur sometime in the next 20 years,” he wrote. “In fact, of all the things that could kill more than 10 million people around the world, the most likely is an epidemic stemming from either natural causes or bioterrorism.”23” This was the cost of a fast-moving, interconnected world. It was what floated free from the extension of Paul Virilio’s line: Airplanes produced the 236 In fact: Roy Hassner and Jason Wittenberg, “Barriers to Entry: Who Builds Fortified Boundaries and Why?” International Security, Vol. 40, No. 1 (Summer 2015), pp. 157-190 237 Reviewing the problem: Bill Gates, “The Next Epidemic — Lessons from Ebola” New England Journal of Medicine 2015, April 9; 372:1381-1384 164 HOUSE_OVERSIGHT_018396

airplane accident. Well, networks will produce the network accident. Many, in fact. A world primed for contagtion, Gates suggested, needs - and is missing - more and better gates. (Yes, here you can pause to double take at the serendipitously strange fact that the wealthiest citizen of our gated age is named Gates, just as the richest man in the era of pulling oil out of rocks was a “Rockefeller”) Connection demands systems to gaze ceaselessly for the smallest sign of assault or need for change or accidental shimmering movement of danger. Part of what made the Ebola response successful is that the response really was in the form of gates as we've come to understand them here, not walls. Protocols for biological reaction, for medical care, for epidemic monitoring, the urgent helicoperting in of support and aid and ideas. The gates assembled around the Ebola pandemic were its solution. Had it merely been walled off it would have spiked, grown, mutated and finally escaped.”The world,” Gates wrote, “needs to build a warning and response system for outbreaks.” This is true for all the outbreaks we face - whether they are disease or financial panic or terror. There will be, you can be sure, ignorant calls to build walls and not gates. This misses the point of networks. In an age of interconnection walls are nearly as dangerous as no walls. What matters is bulding topologies designed for gating, for milking power from the profound logic of networks we've seen at work. For decades after the 1929 financial crisis triggered an historic global depression, economists and politicians debated what had gone wrong. What had they missed? The world, it emerged, had been wired with an economic system designed rapid movement, but politicians and bankers had forgotten to put in the needed brakes. They had tried to run an industrial engine against the background of gasping political structures. Basic adjustment mechanisms - release valves for financial or currency pressures — had not been invented, refined or installed.238 When we find ourselves in coming years gasping through some sort of financial crisis of our own, or running scared from a cascading military or social epidemic that slaps us unexpectedly, what do you think the most likely cause will be? It will be a failure of gates. Too few in some places; too damn many in others. Remember the haunting, persistent vulnerability of our age. A twitch anywhere in the system can rock and even crack the whole edifice. Space Time Compression’s ability to shrink distance to zero and time to an instant can turn anywhere, anytime into a battlefield or a spot for revolutionary innovation. Any moment can present us with questions of war and peace. Today we have no central theory of gatekeeping, no ideas about balancing inside and out. We fly on instincts, but they are the old industrial ones. Anyone can see a system that is full of holes and inconsistencies. Can you see a way through it, though? Pressed by political, technical and economic desire - to say nothing of the basic urge for self-preservation - more gates will be with us from now on. 238 Basic adjustment: Ben S. Bernanke, “Money, Gold, and the Great Depression”, H. Parker Willis Lecture in Economic Policy, Washington and Lee University, Lexington, Virginia March 2, 2004 165 HOUSE_OVERSIGHT_018397

“Computer systems are, like many other things in engineering, constructed by composition,” computer researchers F.X. Lindner and Sandro Gaycken have written. That line, intended for switched-on digital systems, in fact fits most of our world. Nearly everything around us, from cities to telephone networks to refugee waves, are assembled by composition. Curriences are layered with encryption. Refugee dreams carved from photosharing and refined with GPS. Composition in this way breeds new vulnerabilities, new points of contact and this rather astonishing result, one we're not prepared for in any sense: “In a composite system there is no critical gate,” Lindner and Gaycken explain. “Everything is a gate.”299 239 “Everything is a gate”: Lidner and Gaycken, p. 56 166 HOUSE_OVERSIGHT_018398

Chapter Ten: Defense in Depth In which gates, operated with our new instinct, become at once a tool of prosperity and survival. If you walk for a few minutes to the south of Tinanmen Square in Beijing, leaving the tomb of Mao Zedong behind you, there’s a small lane that runs back into a warren of anonymous, white-walled buildings. The streets are unusually clean; the surveillance cameras unusually - even for Beijing - dense. The neighborhood is home to many of the last of the generation of Communist Party cadres who joined and supported its rise decades ago. Several years ago, I found myself settled into one of them for an afternoon tea with Huang Hua, one of the great modern Chinese foreign policy figures - and a member of that early revolutionary generation. Huang was, in a sense, heir to the Warring States diplomat Su Qin, whom Master Nan had encouraged me to study. Huang had penetrated the madness of Mao’s revolutionary era to see the possibility of a different order, one he’d brought to vivid life after Deng Xiaoping ascended to the Chinese leadership in 1976. Huang had been the country’s Foreign Minister and later a Vice Premier. He was always calm, with an easy and relaxed temperament. One of my favorite images of him was from the mid- 1970s when, while sitting on a flight to the US from Paris to take China’s seat at the United Nations, he was ambushed by Walter Cronkite. Huang is completely unflustered in the scratchy video of that encounter. He sits quietly in a cloud of smoke. Cronkite pesters him. Huang smiles, offers a cigarette to the news crew, and though he is in the midst of a transit from the poverty, chaos and smashed politics of China he is nothing but serene, a statesman - not the nervous representative ofa twitchy power. “Do you know the difference between Western and Chinese thinking?” Huang asked that afternoon as we sat inside his courtyard house. Leaves were turning outside. He was in his early 90s then, still tack-sharp. “You see, when Chinese want to do something, we begin with the question: What is the nature of the age? Westerners begin with the goal. What do they aim to achieve?” Deng’s foreign policy, one Huang shaped and executed, had been an excellent example. Mao, who ruled China from 1949 until 1976, had a darting, paranoid, murderously strategic temperament. The nature of his age, he was convinced in his Marx-addled mind, was one of zhanzheng yu geming - war and revolution. From this first principle, everything followed: He honeycombed Beijing with bomb-proof tunnels, relocated Chinese industry to isolated and gaspingly poor mountain strongholds, reacted to foreign ideas or influence with a snapping electricity - and was bent on protective isolation as he dragged the country through one impossible and failed isolationist development initiative after another. Deng, when he came to power in 1977, read the nature of his age too. He read it differently. “There is no possibility of a great war. Don’t be afraid of it, there is no risk of it,” he assured a group of Chinese cadres during a chat in 1983. The cadres were having a hard time replacing their Maoist paranoia with confidence that China 167 HOUSE_OVERSIGHT_018399

might safely open, develop, and change. They mixed the crouching nervousness of a secret political party with the vivid nightmares of a nation that had been invaded, abused — humiliated, they said - by nine countries since the mid-19" Century Opium Wars. “We used to worry about war and talk about its possibility every year,” Deng told them. “It seems the worry was overdone.” The world was, he felt, entering an era of heping yu fazhan - peace and development. Terrible, nation-demolishing wars would not soon victimize China, Deng thought. Nation building was the nature of the age. Its tools would be science, finance, and trade. If the Chinese people worked hard, he promised his incredulous listeners, they might by the year 2000 grow their thin $250 per capita income to the nearly unimaginable target of $1000. “I don’t care if the cat is white or black,” he famously observed, “as long as it catches mice.” Socialism? Capitalism? No matter so long as it produced progress. Deng’s judgment proved out. There were no major wars. Development was, for China, the name of the age. The cat caught $1000 dollars of per capita income nearly on the old man’s schedule. It is still playing with the string. There is something admirably direct about the Western way of problem solving, of course. The shortest distance between two points is a straight line. It has its uses. But it’s also true that this sort of elementary geometry does not describe, always, the best path. Particularly not today, in an era when topological collapse or instant connection means that the distance from a to b can go to zero (or infinity) in a single sudden shift. Yes, you may be stringing a line from a to b, but if you’re putting it up in a tornado? Is the best route to a new Middle East really the tipping over of stable states? Is the quickest path to climate control voluntary compacts? Are faster markets and more unwatched capitalism the safest way to ameliorate our unbalanced economics? Sometimes a more indirect line is called for. Sometimes it is faster, safer. Huang Hua’s question - “Where is the world going, after all?” - turns out to be the first essential one for any endeavor. Ifyou are not looking from the highest possible altitude, Chinese will warn you, you are not seeing the full picture. The nature of our age has, | think, emerged clearly, vividly and broadly for us now. Constant, instant, everywhere connectivity taps with the efficient mercilessness of a steel hammer on the glass of most of our comfortable institutions. The Seventh Sense has let us understand a bit about the origins of this creative violence. And it has shown us too that our era’s dominant feature is not merely cracking disruption, but also the construction of new orders. Da po, Da li the Chinese might say - great destruction and great construction. This is the nature of our age. Tap, tap, crack. And then: Gatelands - speed-bred by technology and filled with possibility. That some of these gated worlds contain billions of people or trillions of sensors, computer chips or network nodes only strengthens their inexorable grip. That they can be so easily or even accidentally weaponized marks the edginess of our problem. That they will be built atop the sharp-edged shards of a collapsing older system heightens the urgency at which they must emerge. To be inside, we've seen, delivers the light- speed benefits of time compression, of life on a nearly alive mesh. To be outside? It is to be cut off. 168 HOUSE_OVERSIGHT_018400

I said earlier that | wanted to lay the Seventh Sense onto the problems of war and peace and grand strategy - and while the insights our new instinct reveals are life- giving for business and eye-opening for culture, it’s to the devious and unavoidable problem of global order I want to turn now. In a now-famous 1986 speech “You and Your Research”, the Bell Labs scientist Richard Hamming set three questions for anyone embarked on the exploration of new ideas: “1. What are the most important problems in your field? 2. Are you working on one of them? 3. Why not?”2*° Well, the most important problem in the field of global affairs is the question of the future of world order, of how it will emerge - and what sort of aspect it may yet present. It is the problem to be working on. It also touches, like it or not, whatever the most important problem in your field is: Opening new markets, educating your kids, planning five years of corporate growth. Remember the distinction from earlier in the book, the difference between living at an ordinary moment and an historic one? The difference between Warsaw in 1539 and 1939? In one age, history is irrelevant; in the other it nearly throttles you. You can’t avoid being touched. We live in an historic era, not least because the connections that define our age means epochal quakes in one part of the system will rattle other bits too. A feeling for history — Nietzche’s old “Sixth Sense” - should fire up invisibly in us bit now as we think about what life in an historic age might mean for us. The surprises in our news every day. The creaking of our old institutions. What do they augur? The political scientists Charles S. Gochman and Zeev Maoz once made a survey of history to see if they could spot just when the world slipped from calm moments to epochally unettled ones. They surveyed nearly 1,000 wars, big and small, hunting for patterns as you or! might look for trends in stock prices or sports scores. Increasing complexity, they discovered, almost always shook the world into conflict. States, armies, political groups, newly independent nations, ideological and religious forces emerge and then collide in their young energy, each driven by different aims and values and dreams. A cycle marks the process. First, disputes over small matters accelerate — as they are doing now. Everything seems open to question: Do you control this trade route? Who says we have to respect this arms limit? The combined pressure of so many simultaneous disagreements announces a shift in the pattern of all relations. A logical loop follows: “The frequency of disputes,” Gochman and Moaz explain, “appears to be an early indicator of system transformation...The rise in the number of disputes seems to indicate a decline in the degree of consensus on the ‘rules’ of the international order. This heightened dissensus (often culminating in large-scale wars) is followed by restructuring.”**! Pressure on the rules. Small cracks. Large scale war. Re-ordering. Like water moving from ice to liquid to steam and back, this cycle has a predictable logic. It ran in the age of Bismarck and culminated in World War One. It spun again in the age between last century’s great wars too, tripping into a second World War. And it ran as well in the 240 “What are”: Richard Hamming, “You and Your Research”, Simula Research Laboratory (Berlin: Springer-Verlag, 2010) Chapter 6 241 The politial scientists: Charles S. Gochman and Zeev Maoz, “Militarized Interstate Disputes, 1816-1976: Procedures, Patterns, and Insights”, The Journal of Conflict Resolution, Vol. 28, No. 4 (Dec., 1984), pp. 585-616 169 HOUSE_OVERSIGHT_018401

fracturing of ancient China during the Spring and Autumn era or in the Roman Empire in the age of the Caesars. Our current shifting — that tap, tap on every nation’s politics, media, and economics - isn’t really so unpredictable after all. 242 You may know the old saw of military parade evaluation: That the more magnificent a nation’s uniforms are the weaker it’s army usually is. But the gold-braided admirals of some three-ship navy reflect a very human need for security, and particularly for a self-decided feeling of security. Every nation has it’s own foreign affairs aims. Each cherishes a certain national image, memories of military glory and of “interests” inseparable from culture or identity. The goals of the French and the Turks, for instance, each evoke an encyclopedia of history, tradition and politics. Uneasiness in Paris about capitalism and immigration, for instance; or Anrka’s worry about ethnic division, fundamentalisms, and the creeping nuclear progress of their neighbors. Our era’s revolutionary logic will shape choices in every nation differently. But I'd like here to discuss American foreign policy. America plays a central role in world order now. The country’s leading position makes her, to some degree, an unavoidable gatekeeper. In the Napoleonic era, nearly every revolution or war could be tied to energies emanating from Paris. During the Cold War most puzzles of politics or geography might be framed in terms of a zero-sum competition with the USSR. In our own age, we'll find most every problem links to networks and their new logic. And - for now at least - to America. Thomas Paine’s memorable 250 year-old assessment, that “The cause of America is the cause of all mankind,” touches this linked universalism in a way he never could have imagined. “Betweeness” or “Centrality” are the way network science labels and measures such a role. Just how essential is a certain nation or trading platform or point? The networks of America won't, in the future, be the only networks. They will be less “between”. But they will be, always and indelibly, the first reference for design. If the traditional aim of American foreign policy was to prevent the emergence of a challenger that threatened the country directly, or that might fence off and manipulate Asia or Europe or Latin America against Washington’s aims, the concern now is different: Mastery of topological destiny. There are those who observe, as an American think tank noted in 2015, “Today the United States faces no existential threat.”243 This is wrong. The emergence and shape of networks is just such a danger. Security, the great foreign policy theorist Andre Wolfers once wrote, is “the absence of threats to acquired values.”244 The networks around us, as we've seen, are demolishing older values. Often because we want them to. Any revolutionary technology unbuttons an older order this way. But we now approach this destabilized world with a sort of wideyed panic. Control terrorism, manage climate 242 Our current shifting: See also Lars-Erik Cederman, T. Camber Warren, and Didier Sornette, “Testing Clausewitz: Nationalism, Mass Mobilization, and the Severity of War” International Organization 65, Fall 2011, pp 605-38 243 “America faces no existential threat”: See James Dobbins, et al. Choices for America in a Turbulent World (Santa Monica: RAND Corporation, 2015) xiv 244 In his classic essay: Andre Wolfers, “National Security’ as an ambigious symbol”, Political Science Quarterly, 67 (1952) 485 170 HOUSE_OVERSIGHT_018402

change, lock in great-state peace. So many desires scatter our minds and when we fail to achieve them they erode our will. Here is a basic lesson of accomplishment in any human domain: Better for us to focus on a single aim, and tighten our efforts around that one aim and use success to strengthen our will. The answer to the great, newly-charged existential question of politics - “Are you a gatekeeper or the gatekept?” - is not yet decided for the United States. So, strategy first: We live ina connected age where power is sliding into that gated order we've learned about. America’s role? Not to make the entire world democratic. Not to try to bomb or force stability onto every nation now collapsing under network pressures. Not to passively wait for our version of history to “win”. Not to thwart the rise of other countries. No. The strategy that fits the nature of our age and position is to perfect and then operate the essential, topological mechanisms of power now. Sometimes with warm humanity; other times with the ruthlessness demanded by the touch of danger and evil. This involves gates. It involves firmness and nerve and willpower. It demands what we might call Hard Gatekeeping. 2. Everything around us now is or soon will be connected. Access to our trade, the use of technology and networks, our defense or our currency - all of these must be considered as gated systems. Each will come to throb with new power as a result of their links, as a byproduct of their escalating sensitivity, intelligence and power, and from that in-or-out distinction. Much remains to be instantly, fully linked in this way. Our education. Our medicine. Our military. Real-time information, sensors and machine intelligence will erect Gatelands in each of these areas and in others. These walls will go up even faster than real-world berms and fences. The strategic position of any nation or terror group or business is not going to be secured by industrial measures. No one at Google wants to build a newspaper. No one in Al-Qaeda is trying to float an aircraft carrier. Rather, it will be determined in the construction of new tools, which largely means the building of a new, gated order. “Every new age and every new epoch in the coexistence of peoples,” the historian Carl Schmitt once wrote, “is founded on new spatial divisions, new enclosures, and new spatial orders of the earth.” Schmitt was contemplating history: The great Mediterranean empires and the long, millennial power trails of Asian chieftains. But his insight touches every age, our own included. To make a nation, to build an empire, is to draw lines, to push at borders, to smash at gates. Some of this is the wild martial ambition of a Napoleon or Hitler (who Schmitt unwisely advised); often itis simple protection. Lord Balfour, serving as Foreign Secretary, contemplated this problem from his imperial vantage in 1918 and observed: “Every time I come toa discussion - an intervals of say, five years — | find there is a new sphere we have got to guard,” he wrote. “Those gateways are getting further and further away from India, I don’t know how far west they are going to be brought by the General Staff.”245 This problem of where exactly to put the gates confronts us now as well. 245 “Those gateways”: Quoted in Robert Jervis, “Cooperation Under the Security Dilemma”, World Politics Vol 30, Issue 2 (Jan, 1978) p. 169 171 HOUSE_OVERSIGHT_018403

Gatekeeping, as we've seen, isn’t simply the act of marking geographic lines. Those hurrying walls and berms and triple-fences of the real world are a kind of gatekeeping but they are also, I’m pretty sure, expressions of a deeper desire. Each is areal-world defense against topological pressures of population and ideology and violence, all shaped by network magnetism. The likely futility of such gates is clear enough when you contemplate the forces they are intended to keep out: Refugees see prosperity miles on their phones. Little wonder they rush at it. Internet- transmitted messages light up ideologies, protests or fundamentalisms that pull people back and forth endlessly across physical borders. The world responds, as anyone without a Seventh Sense might, with walls. But to focus merely on the physical? This is to misread the issue. Gatekeeping includes real-world borders, of course, but the statesmen of the future will act particularly on topologies. They will work not merely to stop up flows, but to decide their movement. Gates will, for instance, serve as bespoke disease buffers, created to track and then freeze the wotrisome pandemics now crawling at us on the connections of our age. Other gates will be economic tools: The will help isolate and then soothe the unstable and apparently broken, middle-class eroding financial patterns of the past decades. Recall how the design of computer or network systems affects the real world, like a marble slipped under a carpet? Gatekeeping’s topological design is like placing marbles (or boulders) in particular ways, to direct movements and flows as you might guide water with a canal. The great historian Arnold Toynbee once recalled a passing moment with British Prime Minister Lloyd George during the endless (and disastrous) Paris Peace Conference of 1919. “Lloyd George, to my delight, had forgotten my presence,” Toynbee wrote, “and began to think aloud. ‘Mesopotamia...yes...oil...irrigation...we must have Mesopotamia; Palestine...yes... the Holy Land... Zionism... we must have Palestine...; Syria... h’m... what is there in Syria? Let the French have that.”246 This sort of charmless arrogance -— The Holy Land....We must have that - doesn’t much suit our age, but that comprehensive view? What oil and irrigation and Suez were to Lloyd George, emergence and data flows and gates are to us. There’s a crucial difference, however: We need not take the most precious territory of our age through invasion or colonial exploitation or in mis- balanced peace settlements. We can build the essential landscape of power. We can build it for defense. And then we can grow it, perfect it, and attract others. Schmitt was right, human history is the story of enclosures - and nothing about our emerging age suggests any other approach can, with a single clear line, solve the vexing network x network x weapons problems we face. Hard Gatekeeping means molding the landscape, gate by gate, to encourage certain movements and make others difficult, costly, maybe even impossible. Any future order must begin with this first principle: The line between in and out is as essential as what goes on inside. 246 Lloyd George: Margaret MacMillan, Paris 1919: Six Months that Changed the World, (New York: Random House, 2002) p. 381 172 HOUSE_OVERSIGHT_018404

The goals of Hard Gatekeeping are simple enough to state: To protect those inside the gated order, to make security and innovation more efficient, to accelerate certain kinds of connection and dampen others, to manage vulnerable links to the non-gatekept world and - perhaps most important- to use that “in or out” leverage to relentlessly affect the interests and plans of others. These aims are the root of real security in any age. “In anarchy, security is the highest end,” Kenneth Waltz wrote. “Only if survival is assured can states seek other goals such as tranquility, profit and power.”247 Survival - and Waltz’s other essential aims - will be decided by the nature of our future connection. In this sense, Hard Gatekeeping produces a decisive change in our posture. It lets us aim for something instead of merely being against movement or - hopelessly - against the future. It establishes priorities, clarifies our real core interests, helps us budget our effort and our expenses — better, perhaps, to spend atrillon dollars building tools of topological mastery like infrastructure or new technology than chasing Middle East peace.**@ Gatekeeping does not transcend the long-standing debates between the cold realism of Stalin and the idealism of Woodrow Wilson so much as it sets us a preliminary task: Shaping the environment. To build and control the next generation of gatekept systems should be our aim now. In the future paper currency wil be replaced by secure digital bit transactions. Genetic information will demand smart, live-connected platforms for efficient mining and study. Cyber and biological security will each come to be defined by high-speed, machine-intelligent protocols. All of these are gatekept cores. Mastering them will be as important as having military bases, trade missions or treaty arrangements once was. The development of such systems plays to our strengths; they deliver both security and useful pressures of innovation. Remember - the aim isn’t to build walls. It is to use the logic of networks and gates, and all the complex productive force they release. Bs The elements of Hard Gatekeeping can be stated simply enough: First, Does it work for us? We seek a landscape of technology, trade, finance, knowledge - all the tools of power - that balances the pressures of politics with a colder grasp of longer-term national interest. The protocols, alliances, treaties and data webs that enable markets, trade or thinking machines - all are gatelands we can cultivate, improve and protect. Our gatelands must be designed to limit the rippling risks of contagion or error moving wildly, like spring weather, inside linked complex systems now. Control of topologies is what air superiority or sea mastery once were. The opening attacks of future wars will come invisibly and silently through networks, in space, in fast topological strikes — not with noisy land invasions or bombing runs. Well-built gates will give us time and leverage. They will 247 “In anarchy”: Kenneth Waltz, Theory of International Politics, (Reading, MA: Addison-Wesley, 1979), 126 248 It establishes: For a discussion about why a grand strategy is of practical use see Hal Brands, “The Promise and Pitfalls of Grand Strategy,” Strategic Studies Institute, External Associates Monograph Program, 2012 173 HOUSE_OVERSIGHT_018405

deliver the honestly-earned confidence of the secure. They will provide purchase for the patient work of diplomacy, a marked shift from the crisis-to-crisis lurching of today. The texture of our values should be everywhere woven into these gatelands we will cultivate: Democratic choice, freedom of thought, privacy. We won't have time for leisurely deliberation about ideals in the crashing, time-compacted crises ahead, so we should begin with them. Let other nations measure gated enclosures for their different values, marked by their paranoias and historical burdens; our strength will be an order that reflects our habits of civilization. The curious, open-minded searching native to our temperament, for instance. This will force constant innovation and evolution, an antidote to the failure to change that poisons most closed systems. A continued absence of gates will undo us in the short run. But the long run will be a disaster if our gates are inflexible, too closed, incapable of upgrade or —- worse - girded by promises and fears that are not naturally our own. A second principle is that we ought not force anyone else to use our gated systems. Gatekeeper or gatekept? No more profound, painful, liberating or enslaving political choice now exists. Nations must be free to select, in as much as they can, their own terms of enmeshment. Our aim should simply be to build the best order we can. America’s tremendous economic and technical lead serves us - for now, at least - as the isolating stretch of the Atlantic and Pacific once did. There’s no need to force others to follow us. Recall Greshams’ famous “economic law”: The way in which the bad drives out the good. A stock market made of swindlers won’t attract investors for very long. Well, in many networks, a sort of reverse-Gersham’s law applied. The good drives out the bad. By this I don’t mean merely the self-cleaning properties of systems like Wikipedia. But of linked systems more broadly. If one nation runs a DNA database where results are skewed by rules of political power - family members of leaders get special access for instance — it will make it less effective, and therefore less appealing as a gateland. So we should be relaxed about letting Europe or Russia or China try their own gated systems. Their desires reflect sensible, understandable urges. If you were leading a nation would you rely on Washington’s trade or financial commitments? Because of the nature of power now many nations may have no other choice; but forcing them to enmeshment could bring shattering, unnecessary pressure. Let them divide themselves from us; it will only weaken them. We need not evangelize, invade, or compel our way to power. Consider the case of global trade. For decades after World War Two, many nations sought independence from economic entanglement. They yearned for autarky. They scorned trade. And they nearly bankrupted themselves in the process: Want to buy Portuguese cars? French computers? Indian bikinis? | didn’t think so. The whole point of trade was to solve such imbalances of skill and trade. When the age of connection began after the Cold War, most countries made a decisive shift: They arranged their economies to be attractive to global financial flows. Trade grew twice as fast as the world economy. The cost of being left out was, and is, a nearly impossible burden. American systems, if they work for America, should have the same magnetic appeal, 174 HOUSE_OVERSIGHT_018406

the power-law dynamics Brian Arthur found in the most successful network arrangements. Getting it right is enough of a task. If we can manage that, what we build will run with unimpeachable legitimacy. Chosen, not forced. A third principle is that our gates and protocols and webs ought to be open to others, but with each new link weighed, added to the scale of strategic balance with exacting care. An overarching, systemic view should inform us. Nations that want to use our cyber security systems, for instance, should join research cooperatives - and possibly shoulder as well some problems of global network health like the implementation of IPv6 or DNSSEC, or any of the future jointly secur standars that will emerge. They should even - here is where the leap comes - be required to cooperate on nuclear proliferation, cybercrime, or trade norms. The puzzle here is clear: How to use our strengths to address our dangers? While we can’t directly press our lead in cancer research to stop national flirtations with nuclear bombs or cyber crime, it’s probably true that a cost of weapons proliferation should be exclusion from life-giving technical and trade and other gated orders. Remember that feature of network systems now, how the cost of exclusion grows even faster than the benefits of inclusion? If the aim of any society in this connected age is to accelerate the compression of time for its citizens, than life outside the best gates for this will be a nearly fatal political cost. Do you really want to rely on French computers? Wear Indian bikinis? Do Iranians want to rely on Iranian cyber-security tools alone? It is ever-easier for citizens of nations sliced free from vital networks to feel what they don’t have. The courage to leave some nations out of the order we're constructing — and to cut others out - doesn’t come easily to us. Today we don’t really engage ina comprehensive, linked strategy. Iranian proliferation and Chinese finance and Saudi military support are, for all practical purposes, separated in our contemplation of the world. Hard Gatekeeping frames these together as a piece; it spots the topological string running between each and gives us a firm position from which to pull the threads we want. We will come to know, to pay, the costs of over-inclusion. But this also means that we must be concerned with the security and safety of others, on our network or on others. To attempt to achieve absolute American security, to do so at the expense of other nations - this will not only erode the credibility of any system we build. It will also make them fundamentally less effective. We are, after all, what we are conneted to. Fourth, no nation should be permitted to force another inside of its own gates. India might develop a great search engine; they should not be allowed to force Bangladeshis to use it. China may develop clever trade channels to Europe; they should not drive their use with military pressure. This principle is the flip side of our own commitment not to compel others into American gatekept systems. It represents an important nod to sovereign power: Each nation decides on its own. If we're intent on avoiding some of the chaos of the last major system change, which produced the Thirty Years War, we should learn its lessons. That fight fundamentally touched on one question: Who decides what happens inside a country? The specific problem of Europe 350 years ago was if each King could 175 HOUSE_OVERSIGHT_018407

decide the religion of his state. The treaties that settled this debate in 1648 - after the most devastating wars in European history at that time - established Cuius regio, elus religio as the governing principle. Whose realm, whose religion. The very notion of a “state” is itself being eroded as topologies, migrations, and superfast data links eat at old borders. But nations remain an essential container of power. A rapid collapse of that system would be a disaster, and is frankly unlikely. To be American. French. Chinese. This still matters. Do we demand that other nations use our protocols? Do we force China or Germany to rules of transparency, data tracking of their citizens and research that we obey? No. Cuius reticulum, eius reticulum. Whose realm, whose network. But, at the same time, should we permit China or Germany to force another nation into its trading regime? It’s biological or cyber networks? To the question, “What would America fight against in a gatekept world?” one answer is this: We would resist any attempt to force-fit a nation to a gated order. We should prepare for difficult, expensive fights to maintain this principle. Russia watching neighboring states seduced into a Chinese trading order or an American technological system won’t bubble with silent acquiescence. Arrangements that tilt trade to one side of the Pacific will unnerve China or Japan - or the US. But, in the end, the control logic of cuius reticulum buffers us, as a sort of law, against a wilder madness of collapsed nations. Fifth, we should not permit the emergence of any means to destroy our system. The very efficiency of connected architectures makes them vulnerable. Networks, by design, have holes that particular modes of attack can exploit. Contagion. Strikes against central nodes.?49 Arms racing of a certain type. Our first attempt to limit risk should be defensive. Better gates. But we'd be foolish to stop there. Gates, the Trojans would remind us, are not enough. When truly existential dangers emerge - nuclear weapons, certain types of artificial intelligence or genetic engineering - then we must attack. Hard Gatekeeping should include a coiled, prepared readiness to strike with the same light-speed movement that defines the dangers we fear. Of course we should try to deter such attacks. But we should be realistic. There’s much we can’t deter. And the cost of attack is awfully cheap. We won't be truly safe without battle plans and diplomatic gambits that can hammer at the topological and real world vulnerabilities of our enemies. After all, they will be hammering on ours. Our defense task is simple to say, hard to achieve: From the largest symmetrical superpower in history, America needs to become the greatest asymmetrical one - capable of devastating, instant action anywhere in the real or topological world. Network scientists who study the power of “centrality” have shown the way that certain positions in a topology - think of a rainforest, a travel network or a stock exchange - wield unusual influence. Centrality of a certain node in linked systems can determine control, it can produce “super spreaders” which instantly shape 249 Strikes against: I.A. Kovacs, A-L. Barabasi, “Destruction Perfected,” Nature (News & Views) 2015, 524, 38-39 176 HOUSE_OVERSIGHT_018408

linked topologies?5°. Contagions, panics, economic brain freezes - these are the essential tactics of our future. The maintenance of our centrality on the topologies we rely on is, then, the essential part of our future security. Interrupting this is, of course, exactly the edge our enemies seek against us: To infect us - with panic, terror or some virtual or real virus. To alter the landscape with one sharp razor slice, not a hail of bombs. There is no excuse for us to not prepare to struggle in these new terms. Our military must be shifted for the demands of such instant, cross-space attack. But it’s more than that. In recent years America has gotten into a habit of reaching for our military tool, and while we've strengthened it to an historically unprecedented degree, we've let other skills atrophy. Our real aim should be to stop our opponents from adapting, evolving and connecting to us. If we suffer some strategic, nation-crippling defeat in the future, I don’t think it will be because our military was weak. Rather it will be because we've not mastered all the other tools of topological control. The first sign of our real awarness of this will be when the old, foolishly vulnerable systems and tactics and ideas are replaced. We should never build another aircraft carrier without a sense of new, networked naval plan. The idea of spending hundreds of billions for manned bomber planes is absurd. We should embrace self-imposed limits, we should force ourselves to innovate both with our military and in the creation of new tools of policy. Doing so will prevent any number of conflicts. And it will stiffen our temperament to unconventional thinking. There’s a lean efficiency to these five principles that define Hard Gatekeeping. The strategy echoes, in its clarity, in its clean frugality, the postures of some of the most enduring orders in human history: The “defense in depth” of the Roman Empire, for instance. The protective order of Tokugawa Japan. The walls of Tang China, part ofa chained national history that celebrates heroes as “defenders of the nation,” not attackers. Gatekeeping resists unnecessary profligacy. It limits the need to proselytize or colonize or force others to our way of thinking. And it has a universal aspect: Hard Gatekeeping can - with a little imagination and some discipline — be laid on puzzles like terrorism, cyberattacks, US-China relations, order in the Middle East, the proliferation of nuclear weapons, income inequality and global warming. It provides a sharp single image of what we're after - and guidelines for getting there. It’s clear enough what we need to do, I think. Build gates. Use them. Hold inside them and protect the curious, innovative and revolutionary spirit of our people and our age. 4, Let’s use this approach to examine some particular problems, and let’s start with US-China ties - a puzzle whose resolution may be the decisive act of policy in the 250 Super spreaders: Manlio De Domenico, Albert Sole’-Ribalta, Elisa Omodei, Sergio Gomez & Alex Arenas, “Ranking in Interconnected Multilayer Networks Reveals Versatile Nodes”, Nature Communications April 23, 2015 177 HOUSE_OVERSIGHT_018409

next century. How might Huang Hua, with his nature-of-the-age consideration of strategy have regarded the matter? What does gatekeeping tell us? The dominant view of future relations between Americans and Chinese generally runs along a nervously familiar historical track: An established power (the US) anda rising one (China) consider each other. Do they cooperate? Scratch at each other in constant annoyance? Each nation possesses a different image the world. Each holds, as well, distinct pictures of their role. One power has enjoyed a long period of prosperity, has built and defended a global structure; the other, trod mercilessly underfoot by history, feels the unbearable flow of power’s rising sap, a hunger for recognition and release and for some, revenge; her economy craves resources, trade routes, and markets. So Germany encountered and then attacked Great Britain at the turn of the 20 Century. Japan collided, similarly, with Russia in 1904. France taking on Austria, Russia and Great Britain in the Napoleonic age. Even if the leaders of the United States and China intend to avoid conflict, they face one of the most sobering, if not depressing, problems of international politics: The “security dilemma”. The world is a dangerous place. So countries do things to feel safer. Their populations demand it. Germany looks at Britain’s imperial navy in the 1890s: London could snap Berlin’s trade arteries. So the Kaiser orders two battleships. Britain builds three in response. Germany turns to submarines. And so on. Each country, chasing her own security, ends up /ess secure. This is the “dilemma”. The puzzle is like one of those woven wicker finger traps: The harder each side pulls to get out, the more stuck they become. America in 2012 pivots her military gaze to Asia. China feels encircled. She orders a couple of aircraft carriers, paves remote islands into military bases. America flies aircraft closer to Chinese waters. Are the two sides more secure? Collisions, arms racing, accelerating distrust become the threnody of contacts. Each side seeks something impossible: Perfect security. Move first, certain forces begin to whisper in each capital. The essence of this problem today, as the United States and China consider their future, is not merely or even mainly in the details of trade, territory, or cyber disputes we read about. These hot points are important, of course, but they are symptoms of differing judgments of the world, of a fracture at that first step of asking, “What is the nature of the age?” Americans generally believe the existing global power map is just, sustainable, in need — perhaps - of minor adjustments. America engineered this system; benefits tremendously from it’s fictionless operation. China regards that same layout differently. As broken, unbalanced, rocking towards a seizure — and, anyhow, built, lubricated and run without her participation. This view is sharpened by ideology, national psychology and the bald fact that no global mechanism ever functions perfectly. The larger nature of the age is, many in Beijing feel, Da po, Da li. Great destruction. Great construction. Certainly this is true inside China; it must be true outside. The nation has it’s hammers out. They see the paradoxes of power we read about earlier, the inability to act and the collapse of American credibility, as clearly as people in Washington or Damascus or Moscow. Surely the global system should be adjusted, they think. Tap. Tap. 178 HOUSE_OVERSIGHT_018410

So though no inherent enmity exists between the two countries, a crack emerges. Each side finds reasons not to cooperate. A shared interest in peace - neither side wants a collision, both understand the costs - is shimmied apart by a perception gap. Cultural, historical, temperamental and ideological differences accelerate the unzipping of even the best intentions. Misunderstanding bends and then snaps irrevocably into mistrust. So yes, the recovered islands and the trade tensions and the political theories have to be managed because each is a blister that limits forward motion - but the real source of friction? It is the scraping of different world views. This is deep strategic misalignment. It is, in fact, the stuff of war. In recent years, America has had two different, nearly opposite approaches to China, each however marked by an assured national confidence that the current world order is sustainable. One policy is short-handed as “engagement” - or, colloquially, “panda hugging.” As China becomes more prosperous, this approach suggests, she will become more congenial to American interests. More cuddly. This was, for some, the basis for bringing China into the World Trade Organization in the 1990s. Later it justified deep commercial links and educational ties. The logic of such an approach fit the mindset of many baby-boomer Americans: China’s attraction to MTV, McDonalds and Mercedes hinted at a still deeper desire they thought. To be Western. China’s citizens would surely become more attracted and committed to the world system that had delivered all this prosperity. The country would become more global, in a sense, and /ess Chinese. In fact, as China has grown in recent years she has become more global and more Chinese. Growth, wealth, an admixture of confidence into the old national habit of insecurity - all of these have encouraged a searching exploration. The demands of domestic politics, the instincts of the Chinese Communist Party and the inevitable practical questions of security and reform all produce sensible questions. China needs oil, for instance. Must she project power into the Middle East? Control the sea lanes of the Malacca Straits? There’s no natural, inherent resistance to American interests in any one problem, but that hopeful American logic of “engagement” ignores much about China’s essence. The country presses at rules she mistrusts (as America once did). She struggles with the inevitable challenges of a political system still smoking a bit from recent revolutions - and, of course, facing the pressures of a networked age. But she holds, strongly, a sense of Chineseness. Can we honestly expect to use mere prosperity to change China in a predictable, pro-American way? “Engagement” feels to Chinese like a demand to become more American. That prospect is as appealing as a request to become more Chinese would be for Americans. An alterative strategy to “engagment” is known as “containment”. (While not formally called “Panda Kicking”, this is often what this policy looks like.) This view takes it for granted that as China becomes more powerful she will become a threat. Any growth in Chinese power means a decline in America’s influence and security. A stronger China in Asia, for instance, means a weaker America. Countries like Japan or Korea or the Philippines will be required to choose a side at some point, at some awful 21st Century Melian moment. The aim of containment is to squelch China 179 HOUSE_OVERSIGHT_018411

before such a fork appears. This approach, too, has its logical problems. Boxing the largest economy in the world hardly seems sensible - and is likely not achievable. Anyhow, early attempts to do so have shown how little leverage the US has. When China announced plans to build an Asian Investment and Infrastructure Bank in 2014, the US opposed it and then watched as nearly all of Washington’s traditional allies raced to join. At the same time, China’s policy towards the US has been hard to decipher for many: Is the country intent on amity or enmity? It’s clear enough that the American options presented to Beijing are unappetizing: Change or be contained. Messages by China in response have tickled the natural historical pessimism of American political leaders. China speaks of wanting a new kind of relations with America, of cultivating rich technological and financial ties, but the country at the same moment challenges easy assumptions about how foreign investment, educational or security cooperation might mesh together like gears of friendly progress. It’s hard to do business in the Mainland. The military grows with astonishing speed. Domestic habits of smash-mouth competition rattle at more sedate Western habits. Any element of China’s actions are explainable and even sensible when seen with sympathetic eyes — but the same exact facts can be stitched into a story that suggests constant damage to American interests. China’s position is, to be honest, a difficult one: The country is opaque to most foreigners, wrapped in habits and instincts that run along such different lines that even simple daily decisions - how to runa meeting, for example - can be a source of disagreement. Jn the west you prize diamonds for their clarity, Chinese will say, in China we prize jade because it is cloudy. There’s a charm in opacity; even a bit of sensible political magic at times. But China is moving out into the world very rapidly, now: Fast and dimly understood don’t fuse into easy trust. In one sense, then, it’s possible to sense the tense, angry energy of an impending collision. Beijing and Washington are enacting the old historical pattern. Wrestling for years over small issues, never quite coming to grips, slowly pulling at each other until a final, fatal collapse. Germany and Great Britain, all over again. But: Connection changes the nature of an object. So much in our world looks different after connected dynamics are applied. Might connection change the nature of enmity? We are, after all, part of the same connected skein. We've concluded that America’s greatest threat is not China or Russia or terrorism - but the evolution of the network itself. That same topological evolution menaces Beijing too. If China is not our biggest enemy, if we are not theirs - what might be developed? Might the rat trap of history remain unsprung? An American “China Policy” shouldn’t be regarded as some distinct initiative; rather it must be a part of a grand strategic program of Hard Gatekeeping. Our goal aim isn’t some shimmeringly perfect “China Policy” so much as it is to cultivate a grand strategic disposition that informs what we undertake in cyber, Russia, the Middle East, and China. Imagine we had a terrific “China policy” and a backfiring “Nuclear weapons policy”? Masterful grand strategy is marked by a comprehensive elegance. Metternich’s old construction, “A Concert of Nations” tells the story pretty well: A 180 HOUSE_OVERSIGHT_018412

symphonic effort, each note and instrument in place. Grand strategy tells us what to do at every moment. Any policy, then, must begin with a summary of our aims: “We believe the world is entering a period of profound change that demands a shift in global arrangements. Weare building a gatekept order consistent with our values. We welcome others to participate in it - but with conditions. We will resist attempts to force nations into any gatekept order; we will fight any forces that threaten disruption of the overall system.” The China policy that emerges from this view is clear enough: We will not contain China. We will not force China to change. Rather we will develop a profound gatekept network for our own operation, one that layers together all the concerns of economics, trade, security and technology - as if we were designing a technical system for fast, constant links. We will include other nations that want to be a part of it, as long as they are willing to bear certain costs. China is welcome to develop her own system and see whom she might attract inside. She will not be permitted to compel nations to join. And: If China wants the benefits of a US-led order, then Beijing must be prepared to support the maintenance of that order. That means deeper cooperation on everything from nuclear proliferation to the establishment of new international bodies for connected finance, biological research or nuclear de- proliferation. Such an approach resolves, in an instant, the contradictions of our current policy. Here is our gated order, we can say, join ifyou wish and on the terms we both consider best. We should remember: China has much to worry about. Disorder would land on her own hopes for peace and development with a profound, possibly crushing pressure. The international order is not working as well as it might. It does not appear engineered for new stresses. It is in cooperation for the reform of that system that the two nations share an urgent interest. And it is in a successful defensive posture of a gatekept, network order that Washington can cultivate the linked, leveraged moves that will encourage this to happen. While there might be a temptation to play this game in that old Lloyd George sort of way - “We need the Middle East. Grab it. We need digital currency mastery. Take it!” - such efforts will only assure final, fatal collision between the two powers. We need a gatekept order. Build it. That is enough. What the Seventh Sense statesman sees, in the way a Seventh Sense entrepreneur or terrorist or researcher can, is the pulsing network logic that is at work breeding a new world - like it or not. And our policy has to aim for that future state, not for what we have today. If this sounds difficult, you should know that in most complex adaptive systems these kinds of adjustments frequently occur. Rainforests. Traffic jams. The human brain. Anyplace connections are knit and reknit at light speed can exhibit a pattern of rapid redevelopment: The world changes. The network and its various pieces hum, in response, to a new order. Climate shift in the Amazon, for instance, produces drier air. The pollination cycle shortens. Flowers must adjust by growing longer leaves to protect their moisture. Birds that rely on the flowers develop longer, sharper beaks in response. In this way bird and flower each become better suited to a warmer age. They co-evolve. We see this sort of interaction in 181 HOUSE_OVERSIGHT_018413

many worlds. Data companies face increasing bandwith demands. They develop new ways to transmit video. Application developers create fresh tools to compress their signals. The whole system gets faster. Such co-evolution is common when linked systems press into one another.?5! If you look back at biological history, at our planet’s seething life at it’s most profound moments of transition, you find co-evolution in nearly every case. The biologists John Maynard Smith and Eors Szathmary, in their masterwork The Major Transitions in Evolution, chronicle the miraculous march of life towards ever-greater complexity — from cells to humans to societies - as a story of ceaseless, successful co-evolution. “Complexity is hard to define or to measure,” they write. “But there is surely some sense in which elephants and oak trees are more complex than bacteria, and bacteria than the first replicating molecules.” The essential, facilitating feature of our biological hop-scotch from mitochondrial snot to Beethoven sonatas has been an ability to change together, not merely to compete. While we humans tend to consider evolution in murderous you win/I lose Darwinian terms, the biological world around us seethes with a more nuanced logic: Cooperation is every bit as important to survival. Smith and Szathmary identified eight major transitions in their work - the evolution from single cell to multiple cell organisms, for instance. And in each case they find a kind of cooperative, co-evolving logic serving a catalyst to survival. “The applicability of the concept,” Smith and Szathmary conclude, “extends across the whole of social and natural sciences.” Inside cells. In ecosystem webs of coral or rainforest. In traffic management. Successful evolution was, they found, always co- evolution. “It should need little persuasion to acknowledge that links we make with others change in time,” they wrote. “That all of us age, that our roles in life evolve, and that the society we are a part of may itself be subject to transformations ona global scale. Coevolutionary rules aim to integrate these processes.” The world changes. I change. You change. Network age systems, in their effortless bridging of divisions - geographical, technical, commercial - bounce with a particularly powerful fusion. “It is often assumed that biological agents are by nature rationally selfish, and that therefore external circumstances somehow have to compel them to cooperate,” the historians Nathale Mezza-Garcia, Tom Froese and Nelson Fernadez have written about the organization of societies over the past millennia. That sense we often have, that we're one Darwinian instant from some sort of lethal, teeth-baring competition, doesn’t in fact explain all of human history. 253 Politics is a sort of complex adaptive 251 Such co-evolution: Matjaz Perca, Attila Szolnokib, “Coevolutionary games—A mini review”, Biosystems 99 (2010) 109-125 253 “It is often assumed”: Nathalie Mezza-Garcia, Tom Froese and Nelson Fernandez, “Reflections on the Complexity of Ancient Social Heterarchies: Toward New Models of Social Self-Organization in Pre-Hispanic Colombia” in Journal of Sociocybernetics 12 (2014), pp. 3 - 17 182 HOUSE_OVERSIGHT_018414

system; it fuses cooperation with brutal competition as it aims for a kind of equilibrium. After all, if we were constantly tearing into one another, we'd be extinct, not evolved. Why did cities form? Nations? The key is a moment when shared, possibly devastating risk of collective failure becomes apparent - those moments when the whole system threatens to collapse right on top of all the participants. The greater the shared danger, Mezza-Garcia and her colleagues explain, the better the chance for real cooperation. It’s a forumula that fits what we face now: “An increased risk of collective failure facilitates large-scale cooperation, especially when the large scale system is composed of smaller, nested groups,” They explain. ““Complex systems theory reveals an alternative to constant conflict.” This is our world now: Linked clusters of markets, nations, machines - all exposed to arisk of shared and instant collective failure. Such a structure upends an axiom of politics that has run for centuries: Man is purely Darwinian and that survival is determined by constant competition alone. In a network, survival is determined by sociability, by cooperation. Linked systems drive people to agree on rules in order to participate, which accelerates change and co-evolution. Smart cancer databases, linked traffic systems, video platforms like YouTube - each are Gatelands that press users together on a single platform, one that can be instantly updated and constantly studied for adjustment. “Rich get richer” arrangements are an ideal place for co-evolution, shaped as they are by easy linkage, exchange and connection to outside events. As more people follow the logic inside the gates, the system co- evolves. It becomes still more fit. It’s this loop that makes Hard Gatekeeping so well suited to an age of connection. We’re not merely putting up walls; gatelands are like markets or public squares. They are loci for cooperation. In diplomatic history it’s not hard to calls for cooperation. “We must get along or this war will devastate us all!” But they don't work. Mostly you find hand-wringing about how terrible war would be followed by, well, war. The logic of networks offers an escape from this sad habit. Co-evolving relations between the US and China can begin at the most obvious of starting points: Both sides need to change. New pressures are already tearing at each. Both nations need a new gatekept system. Our aim isn’t to “balance” China into some sort of frozen checkmate. It’s not to tip her unwisely into chaotic and impoverished domestic order - we are, like it or not, all connected. It is to co-evolve together. And China? She won’t survive if she does not connect to a system that is congenial to her needs. Fortunately, the networks are infinitely flexible in their design. A Chinese model and an American model pose no problem of interoperability. This is the best co-evolutionary strategy for each side. The dangers we confront now are everywhere. The habitual American temptation is to break all these risks into pieces, to find out where the problem is, and to flatten each of them: “Let’s hit the terrorists, then the Chinese navy, then the Russians, then the drug lords.” This will kill us if we keep it up. In a complex system, piecemeal attack simply shocks the system to still greater complexity. The world, rather wonderfully, is too flexible for brute force applied at one place. A dynamic defense, is a precursor 183 HOUSE_OVERSIGHT_018415

for survival, for evolution. And it would best be done cooperatively with the nation that will, in a short time, have the largest economy in the world. China is not really a threat to the US now. The country can’t assemble the ability to demolish or attack America in any significant way, and itis years from being able to sustain a military effort. It is not in China’s interest to provoke a match. They would lose. And honestly they have endless, troubling problems to handle, many new in human history. How do you care for 800 million aging people? Though China will pass the US in total national income in coming years, the country may never pass America in per-capita income. With an aging, 1.4 billion person population the challenges of distribution nearly guarantee this. Time acts against both the United States and China today. Each depends on global arrangements that are themselves cracking. Each nation needs to adjust her structures for a network world. But these pressures operate severely against China. She can disrupt and challenge and slow the transition to a gated order if she wishes, but only at the cost of fatal distraction. An American Hard Gatekept system will thrive with or without China. China is not so fortunate. Her system, without America or other elements of global connection, cannot continue on the path of reform. Great network construction — this is the nature of our age. There is no fighting the nature of the age, as Huang Hua would remind us. We should of course expect China to build her own systems. We can also expect that she will try to avoid some of the costs that joining an American system might impose. That’s all fine. Our aim should remain constant, calm and direct: Build a gatekept system for our own use; enforce the rules we believe secure that system. And remain clear in our thinking: The threat to the US is the evolution of the network. The threat to China? The evolution of the network. If an arrangement based on these undeniable facts seems unlikely, it is worth remembering that in the twenty years before the restablishment of simple relations between Beijing and Washington in 1971, the two sides had fought a war in Korea, built a politics based on condemning one another, and drifted so far apart as to share nearly no econoic connection. Bringing the countries together demanded heroic diplomacy on both sides, to be sure. To read the transcript of Henry Kissinger and Zhou Enlai’s first, secret meeting on July 9, 1971 is to regard a masterclass in careful chessboard moves?@5. But the game itself demanded a clear view of the board, of shared interest in the face of a changing world. On his very first day in office, Nixon compiled a list of his diplomatic aims: “Chinese Communists: Short range — no change. Long range - we do not want 800,000,000 living in angry isolation. We want contact.”25¢ The Seventh Sense returns us to this fundamental question of balance, but in a new frame. It alters how we look at the range of options for US-China relations - and American security itself. There are certain problems both sides face, perhaps the most essential, that are simply not addressable using 255 To read the transcript: Foreign Relations of the United States, 1969-1976, Volume XVII, doc 139 256 Nixon complied a list: Foreign Relations of the United States, 1969-1976, Volume XVII, doc 3, footnote 3 184 HOUSE_OVERSIGHT_018416

traditional thinking. Co-evolution exists as a real possibility for the two nations but so too does co-extinction, a problem of mis-designed complex systems that can rattle themselves apart. This is precisely what a confident Hard Gatekeeping strategy can avoid. When we speak of the possibiltity of a new type of great power relations it is because so much of future power will be decided on a topological landscapes, on that nearly living surface that is marked by such different rules than the older, industrial power maps. It takes a network to fight a network, as we’ve said. Networks confront us, constantly now, with the unexpected. No one forming the IMF in 1949 considered digital currency. No one developing arms protocols in the 1990s thought about cyber weapons. Artificially intelligent weapons, migrant waves, income fractures - these and other emerging puzzles were never contemplated in our existing international arrangements. Collectively attacking these challenges is not merely more efficient; it is our only option. Yes, it’s easy enough to picture the US and China battling each other over islands and protocols and technology structures in the decades to come. Certainly this possibility has to be prepared for. But this is to regard the world in industrial terms. Ask yourself: What, really is the point of power? It is to secure stability, not to tip the world to chaos just because we think we might be better off on the other side. The networks tell us what they want. They want gates. Our only question - and it is the same question that lingers in Beijing and Washington - is: Are we smart enough to listen? 5. How will power distribute itself in the future? Just what will the network of nations and datawebs and insurgents finally look like when it settles into some predictable if still roiling order ten or a hundred years hence? Will it be made up of different walled and gated systems, linked at certain moments, unplugged at others? Or might it collect with that “winner take all” logic we've seen firing along the most efficient networks? Most traditional foreign policy looks at history and sees the constant, violent rocking of a “balance of power”. Nations thirst for security, they vie with one another for influence, resources, and control. France rises. Statesmen in Berlin, Vienna and London conspire to knock her down. This is the order that dominated Europe for much of the last 500 years. But history is also filled with enduring, stable arrangements - moments where the balance settles a bit and a single power dominates. Asia, Europe, the Middle East, South America - each produced nations that stretched mastery the system for generations, for centuries even. If you map the rise and fall of nations and empires over the past 4,000 years - as the political scientists Stuart Kauffman, Richard Little and William Wohlforth once attempted in an heroic statistical effort - about half of human history has been marked by this sort of hegemonic stability. China led East Asia’s order from 1300 to 1900, for instance. The Japanese and Koreans and Vietnamese all found it was wiser to play by China’s rules than to attack her. Assyrian imperial arrangements overmastered a dozen smaller states from the 9 to 7 centuries BCE. The Dehli Sultinate managed hegemony in South Asia from 12 to 14 centuries. The Mughals owned nearly two 185 HOUSE_OVERSIGHT_018417

hundred years of dominance starting in the 16" century. The Romans managed centuries of Mediterranean control . The secret of those long-running orders was something that will be familiar now: Each possessed tools of power which permitted assembly of empire at an unusually low cost in lives and gold and effort. Kauffman and his colleagues, as they considered the results of their survey, noticed each long-lived empire pioneered an administrative design that embodied an efficiency much like that of our own network dynamics. The addition of new territories brought more that they cost to masters of long imperial orders. Like new users on a social network, or Baran’s fish- nets, they married easy expansion and high returns. “Rome rose because it combined the strengths of traditional Republican institutions with innovations that gave it a unique capacity for inclusion of foreigners,” they explain. “Magadha was the most administratively durable of the ancient Indian states; and Qin, with the self- strengthening reforms of Shang Yang - economic reforms and military conscription as well as bureaucratic innovations - developed the most penetrating and brutally effective state structure in its international system.” The Incas, the Han, and nearly every long-standing empires glistened with this attractive logic. The secret to hegemony, to avoiding a violent power shifts every few decades, is a structure that grows without additional, destructive costs. When Machiavelli coldly called Rome a “republic for expansion”, this was what he had in mind. Enduring empires have been engineered, like a modern network, for growth and prosperity. It’s too early for us to know if this logic will obtain in our age. But networks evolve, as we've seen, to what makes them most efficient. They crave speed and growth. And this means they want cooperation; it’s the essential fuel for co-evolution. The traditional view of the international system as anarchic is not wrong, but we've seen how when you snap any object into a network system it begins to crave a kind of hierarchy. Networks change power balances. National fury and rebellious twitches and competition will, of course, be a part of the transition ahead. But as we look back at the industrial tools that matured and spun up the world to a war in the last century, we can see how they they were designed in a sense for direct collision. Massive industrial armies wrestled in symmetrical power battles. Network power hums differently. The design logic of linked systems means they function poorly when tuned for simple brutality. It’s why the tools for our new world are so dangerous in the hands of those who don't understand what they are capable of, and what they demand. We should remain fixed on what might emerge as a future state, and on avoiding the shaking dangers of the route. It is from that posture that we can begin to consider the most essential and interesting and profitable questions. The most profound is probably this one: We’ve seen now what it feels like to use the Seventh Sense to contemplate the networks around us and to examine the global system with its risks and opportunities in a new way. But what do we discover when, as if we were looking into a mirror for the very first time, we use this powerful new sensitivity to examine ourselves? 186 HOUSE_OVERSIGHT_018418

Chapter Eleven: Citizens! In which the Seventh Sense rescues us from an unexpected danger. 1. ] never needed much incentive to go see Pattie Maes. Belgian, usually dressed in some black fashionable getup, she was like a human shot of espresso. You ended every conversation wide awake, eyes open. When I first met her in the 1990s, she was in charge of much of the work on artificial intelligence at MIT’s Media Lab - Danny Hillis’ old home. Maes had arrived at MIT in 1993 and almost immediately turned to the problem of making machines that might think. One day, as we were discussing just how the strange miracle of computer thought might occur, she introduced me to a puzzle of her field that has stayed on my mind in the years since. It is called the “Disappearing AI Problem.” Back in the 1990s, as the Internet was emerging into popular consciousness, Maes and her team were tinkering with what was known as computer-aided prediction. This was an advance on the ping-pong conversations Joseph Weizenbaum had coerced from ELIZA in the 1960s, You: “Iam bored.” ELIZA: “Why are you bored?” In Maes’ experiments a computer would ask, for instance, what movie stars you liked. “Robert Redford,” you'd type. And then the box would spit back some films you might enjoy. Cool Hand Luke. And, well, you had liked that film. This seemed like magic at the time, just the sort of data-meets-human question that showcased a machine learning and thinking. An honestly “artificial” intelligence. Maes hoped to design a computer that could predict what movies or music or books you or I might enjoy. (And, of course, buy.) A recommendation engine. We all know how sputtering our own suggestion motors can be. Think of that primitive analog exchange known as the “First Date”: Oh, you like Radiohead? Do you know SigurRos? Pause. Hate them. Can you really predict what albums or novels even your closest friend will enjoy? You might offer an occasional lucky suggestion. But to confidently bridge your knowledge of a friend’s taste and the nearly endless library of movies and songs and books? Beyond human capacity. It seemed an ideal job for a thoughtful machine. The traditional approach to such a problem was to devise a formula that would mimic your friend. What are their hobbies? What areas interest them? What cheers them up? Then you'd program a machine to jump just as deep into movies and music and books, to break them down by plot and type of character to see what might fit your friend’s interests. But after years building programs that tried - and failed - to tackle the recommendation problem in this fashion, the MIT group changed tack. Instead of teaching a machine to understand you (or Tolstoy), they simply began compiling data about what movies and music and books people liked. Then they looked for patterns. People were not, they discovered, all that unique. Pretty much everyone who liked Redford in Downhill Racer loved Newman in The Hustler. Anyone who enjoyed Kid A could be directed safely to (). Maes and her team found themselves, as a result, less focused on the mechanics of making a machine 187 HOUSE_OVERSIGHT_018419

think than on devising formulas to organize, store and probe data. What had begun as a problem of “artificial intelligence” became, in the end, a puzzle of mathematics. The mystery of human thought, that great unknowable sea of chemicals and instinct and experience that would have let you place your finger on just the song to open the heart of your date, had been unlocked by data. Here was the “Disappearing Al Problem.” A puzzle that looked like it needed computer intelligence demanded, in the end, merely math. The AI had disappeared. For several decades this accidental digital magic trick - “Hey, where'd the AI go?!” - bedeviled machine intelligence. It gave the entire thinking machines enterprise a bit of an occultish flavor. Many problems that once seemed to demand the miracle of thought really only needed data. The human was still doing the thinking; the computer was simply computing. It was extremely easy to draw a line between where the human ended and the machine began. This was a puzzle that had been, in a sense, anticipated at the very dawn of the digital revolution by the mathematician Alan Turing in a 1950 paper called “Computing Machinery and Intelligence” that he published in the British journal Mind. “Can machines think?” Turing began.?5” His idea was to test this question in the following way: Have a research subject - a secretary, a graduate student, anyone - chat with an invisible interlocutor by way of a keyboard. Then ask: What are you connected to? Another human? A machine? Turing figured you could call a machine “artificially intelligent” if it could fool a user into thinking it was human. “Please write me a sonnet on the subject of the Forth Bridge,” Turing suggested a tricky user might ask. What computer could possibly know about this famous Scottish landmark; to say nothing of being able to rhyme “Forth”? When the response came back, “Count me out. 1 could never write poetry,” you'd think that sounded awfully human. “Add 34957 to 70764,” Turing suggested you might fire back. Say the computer pauses. Then, thirty seconds later, “105621.” Are you dealing with a clever machine here? A dim, honest, slow-multiplying human? Impossible to tell. The distinction between machine and man blurred a bit. Buried in the very premise of Turing’s test was an assumption about what machines could do, and how they might do it. “The idea behind digital computers may be explained by saying that these machines are intended to carry out any operations which could be done by a human computer,” Turing said. His smart machines would be tuned by humans to do human tasks, in a human sort of way. Write poetry. Do math. His test of artificial intelligence was designed to figure out ifa computer could think like a human. You and | might be able to spot patterns in movie habits given enough time, but as more complex problems emerge, as a world of a trillion connected points becomes a sea of data to examine, there is no chance we'll match the machines. This opened a then nearly unimaginable possibility: What if, some day, a computer could think better than a human. Could, in a nanosecond, come back with that elusive rhyme for “Forth”: North. For such a machine, passing Turing’s Test — thinking like a human - was an interesting challenge. But more interesting 257 “Can machines think”: A. M. Turing, “Computing Machinery and Intelligence” Mind 49 (1950), 433 188 HOUSE_OVERSIGHT_018420

was a machine that could think in ways a human could never understand, let alone achieve. For such a device, to pass Turing’s Test or slip past a Voight-Kampff check machine will be trivial. In his 1950 paper, Turing sensed the possibility of this development - and the crisis that might ensue. Could man handle the crushing sensation that a device was outperforming him? Perhaps dramatically. “We like to believe that Man is in some subtle way superior to the rest of creation,” Turing wrote. “It is best if he can be shown to be necessarily superior, for then there is no danger of him losing his commanding position.” What would have surprised Turing, I suppose, is the speed at which we've acquiesced - and even accelerated - this very loss of our dominance. Imagine a device poking at the very origins of the universe at a speed of trillions of calculations a second, spinning past Newton’s and Einstein’s laws and into a realm of physics apprehensible only inside its own electronic consciousness. Compacting time - centuries of human scientific labor reproduced in moments - before shooting far ahead, alone to a subtle knowledge we can only envy. Such a machine would not, as Newton had, stand on the shoulders of giants so much as it would muscle its own, unique way ahead. The AI would have disappeared, but to a place very different than where Maes’ AI had gone. Hers had been erased by human design. This new, really “thinking” AI would slip to invisibility because of its own light-speed cognition. It would think itself out of our understanding. No human could follow, limited as we are by our wet, slow, decaying biological software. Humans and computers, after all, deal with information differently. Think of how poor your memory is compared with the perfect fidelity of a machine, or the way people can even “remember” events that never happened. The machines would have more than knowledge, then. They would linger close to a possessing a profound and inscrutable wisdom. They would inhabit an honestly miraculous gateland that no human would ever enter. And this is where the problems would begin. 2. We've now passed the moment when humans completely train the very best machines. The AI devices can teach themselves, now. Of course there are still decades of adjustment, of leaps in hardware and programming ideas to eliminate the seams between our minds and the fused ideas of a digital system. But the humans in the loop of the best of these designs, the hard working and well meaning geniuses of the New Caste, are as much trainers as engineers now. They resemble shipbuilders of an earlier era, preparing vessels for voyages to lands they will never themselves see. Their digital minds - technically machine learning structures such as deep neural networks - run quickly to the frontiers. Every moment, all around us, machines are educating themselves. About the world. About themselves. About us too. Tuned AI plays games, looks at photos, studies chemical reactions, reads your email and watches you drive - and then it tries to unspool just what is going on, before winding it up into a new instinct. Machine learning systems already produce mathematical proofs that linger beyond what a human mind can understand; next may be a machine-to-machine mathematics that expands the dimensions of thought. (In aman and machine fusion that surely would have unnerved Joseph Weizenbaum 189 HOUSE_OVERSIGHT_018421

of ELIZA, the theoretical mathematician Doron Zeilberger now names his computer as a co-author of his papers. He calls it Shalosh B. Ekhad, a play on the Hebrew name of his IBM 3B1.) 258 The AI systems designer Roger Grosse has named two paths to this sort of wired sensibility: “Predictive Learning” and “Representational Learning”.25° That first approach is what Maes’s movie machine pusued. The computer is simply checking what it encounters against a database. It teaches itself to predict based on what has been seen before. This sort of knowledge begins with massive amounts of data and then hunts for patterns, tests their reliability, and improves by mapping quirks and similarities. Google engineers have a device that can gaze into a human eye and spot signs of impending optical failure. Is the machine smarter than your ophthalmologist? Hard to know, but let’s just say this: It has seen, studied and compared millions of eyes to find patterns that nearly perfectly predict a diagnosis. It can review in seconds more cases than your doctor will see in a lifetime - let alone recall and compare at sub-millimeter accuracy. Fast, thorough predictive algorithms make what might once have been regarded as AI disappear. The machine isn’t all that wise; it just knows a lot. On the other path, the one of “representational learning” the machine uses a self- sketched image of the world, a “representation.” Computers using predictive methods to recognize 10,000 numbers pulled from a database of scrawled hand writing now identify 90 percent of the images. Self-trained machines, however, line up each scanned pixel against a representation of the very idea of writing. They screen millions of pictures with nary a mistake. Faces, disease markers, obscure sounds - all these become scrutable not because the machines have been told what to look for, but because they’ve sort of figured it out themselves. The Al is actually starting to think, much as you or | might, first by building up a picture of the world and then applying it, much as a child might build comprehension of traffic rules just by watching Mom driving every day. With this representation finished, these nearly alive “thinking” meshes navigate by themselves. You can see already the competition lingering here - who can build the most sensitive model of the world? You? A machine? Even today basic versions of representational Als can study a map and name the most important roads. They can predict cracks in computer networks days before a fault. These programs take longer to train. They are harder to program - and they demand almost unimaginable amounts of computing power - but what emerges is a subtle, lively kind of insight. A machine with a representational understanding of Mozart's 41 symphonies can write you an extremely convincing 424 - or, if you wish, an even earlier First Symphony based on what it knows about his evolution as a composer. It can do it again and again. In seconds. The basic attitude of these researchers behind this technology runs, they confess, like this: Mozart was a fantastic composer. If he wrote 258 He calls it: See Nielsen 259 The AI systems designer: Roger Grosse, “Predictive Learning vs. Representational Learning”, Building Intelligent Probabilistic Systems: 2013 190 HOUSE_OVERSIGHT_018422

even more symphonies they’d probably be great too. Unfortunately he’s dead. Wouldn't it be nice if we could sample his old symphonies and make new ones whenever we want?2°° In the future we'll invite Al into our lives to harmonize away many of the problems we face, not merely making up for Mozart’s inconvenient mortality. “AI Agents” will linger along side us. They will compose versions of themselves we'll not quite grasp, even as we appreciate their efficient magic. “Al is both freedom from programming and freedom from understanding,” runs one programmer’s line2@!. Today machines that once demanded millions of lines of code can function with a fraction of that. Instructions are sent to machine learning systems; the programs do the rest. Such designs balance their mystery with efficacy. They speak to and learn from each other too. Part of the reason that the the “Does it think like a human?” Turing Test will be insufficient in the future is that the machines are not learning from only from humans. They are learning from each other. Perhaps this is not such a bad thing. The distinguished physicists George Ellis and Joe Silk, who spent a lifetime trying to stand on Newton and Einstein’s shoulders to grasp answers about gravity or the future of our universe, electrified many of their peers in 2015 with by wondering if perhaps too much of science had become unscientific, unverifiable, unreliable. The great grand ideas of our day, notions like string theory or dark matter, differ in a crucial way from Newton's laws of motion or Einstein’s principles: They cannot seem to be tested and significantly proved. And this had fired a trend among younger physicits: Perhaps there was no need for proof. To Ellis and Silk this seemed an awful retreat, dragging physics back to a pre- Enlightenment age of conjecture, superstition and instinct. “This year, debates in physics circles took a worrying turn,” they wrote. “Faced with difficulties in applying fundamental theories to the observed Universe, some researchers called for a change in how theoretical physics is done. They began to argue — explicitly — that if a theory is sufficiently elegant and explanatory, it need not be tested.” Fans of such an approach called the idea “post-empirical science.” This strange, oxymoronic idea was, in a sense, like proposing post-rules baseball: A recipie for wild, swinging chaos that would make scorekeeping impossible. The strange, boiling debate did however reflect an underlying and unnerving truth: Science does seem to have stalled. And it became inevitable to ask: Might it be possible that the machines - or some fusion of Shalosh B. Ekhad andta human mind - can reach into an understanding of laws that no human alone can fathom. We've said again and again: Connection changes the nature of an object. Perhaps it changes 260 Wouldn't it be nice: Andrej Karpathy, “The Unreasonable Effectiveness of Recurrent Neural Networks,” in The Hackers Guide to Neural Networks published online May 21, 2015 or John Supko, “How I Taught My Computer to Write Its Own Music,” in Nautilus, February 12, 2015 and Daniel Johnson, “Composing Music with Recurrent Neural Networks,” on Hexahedria Blog August 3, 2015 261 Freedom from understanding: Philip Greenspun, “Big data and machine learning” from Philip Greenspun weblog (November 21, 2015) 191 HOUSE_OVERSIGHT_018423

the nature of physics brilliance. Boeing 747s lumbering across the Pacific towards San Francisco for decades faced the sweaty problem of cancelled landings as they circled above a fogged in airport, fuel running lower. The introduction of “autolanding” systems in the solved this for good. No big plane diverts from a misty field; it lands itself. AI offers the possibility of a kind of auto-land for our biggest physics puzzles, bringing them safely through a fog of data, theory and wrong ideas. But with this weird price: We may not fully understand why the answers are right. All around us Al-enabled systems will extend our ability to calculate and learn, to penetrate all sorts of foggy problems. They will sharpen our sadly dimming memories, keep us safe and even help us create. Just as those Al-enabled airplanes already make it impossible for pilots to fly into the ground, so computer wisdom may protect us from crashes of our own: Too much financial risk. Bad educational choices. (Poor music suggestions on a first date.) They will rely on their vast, instantly updated networks to tell us things we can’t see or would never notice in the first place: Don’t visit that office, everyone’s sick. They will use the ability to model thousands of possible outcomes of any choice to provide us with “feedforward” - an ability to learn from the future and not merely the past. Or, they will know to jam our brain full of the right chemicals at the right time: Here’s a Diplo track to put you in the mood to go forarun. You really need to exercise, Dave. Just as an age without connected devices will one day seem strangely antique, so will a world without the constant touch of AI. Recall Benjamin Franklin’s famous lament in the 1780s, that he’d sadly been “born too early” to enjoy the fruits of reason Starting to spill into his world as a result of the Scientific Revolution. Well, you and I (and scientists like Silk and Ellis) may have been “born too late” for an age of purely human cognition; the habits of connected thinking already inform our decisions and mark roads to new knowledge. The inevitability of Al reflects an inescapable logic at work now: We want faster better and smarter systems. We want to compress time. But the faster our world gets, the more it slips beyond a pace of human management. Al steps in. It makes the system function faster. Keeps itself safe. Us too.?62 Better-than-human Al inside these “representational” grids doesn’t vanish like it did in Maes’ lab. In fact, an honestly artificial intelligence is their nature of their strange essence. They will use it not simply to contemplate the world, to help us along, but also to confront what has never been seen, to see and then coldly manipulate any topology of power they can reach. Of course we'll still continue to think about the world; but the world, a wired and alive and cogitating cage, will think about us too.?63 262 Us too: Heinl, p. 53 263 Of course: Nigel Thrift and Shaun French, “The Automatic Production of Space”, Trans Inst Br Geogr NS 27 309-335 2002 192 HOUSE_OVERSIGHT_018424

In the spring of 1993, the research arm of NASA organized a conference on the frontiers of knowledge and invited the most eclectic group of thinkers they could find. Biologists, sociologists and computer designers gathered for the three-day meeting in the unpromising setting of Westlake, Ohio. The mimeographed notes of the conference became legendary and still circulate, a sort of Shroud of Turin for the machine learning set. The introduction features a poem pecked out in IBM type titled “Into The Era of Cyberspace,” written with all the pocket-protector fluidity one might expect of a NASA engineer: “Our robots precede us/with infinite diversity/exploring the universe/delighting in complexity.” (Turing’s rhyming computer, you have to suspect, could have done better.)?°+ One of the first speakers at the conference was a San Diego State University professor named Vernor Vinge, whose remarks that day marked the start of an important era in our consideration of smart machines. The Coming Technological Singularity: How to Survive the Post- Human Era his talk was called. “Within thirty years,” Vinge began, “we will have the technological ability to create superhuman intelligence. Shortly after, the human era will be ended.”26 Vinge’s aim was not - or at least not merely - to tell a room full of NASA geeks who had been dreaming of life on another planet that life on our own planet might soon be replaced by whirring, calculating machines. Rather, he explained, he wanted to plot what a world of not simply intelligent, but intuitive machines might look like. Far from disappearing, Vinge thought AI would produce a sort of wisdom that would be inscrutable to humans. And this wisdom, buffed to perfection by high-speed judgment and endless data, would eventually and sensibly take over much of human activity. Real “Al”, Vinge said, would at the very least be used to design a world of quicker AI that would, in turn, yield to still-faster generations. “When greater-than- human intelligence drives progress,” Vinge explained, “that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities — on a still shorter time scale.” Vinge reminded his audience of a moment once described by the British mathematician I.J. Good, who'd cracked codes in Bletchley Park alongside Alan Turing during World War Two: “Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man, no matter how clever,” Good had written. “Since the design of machines is one of these actual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.” Vinge labeled this instant “The Singularity”: “Tt is a point,” he wrote, “where our old models must be discarded.” The trivial version of this would be an age of autonomous armed drones, self-driving cars and electrical 264 In the Spring of 1993: See “Vision-21: Interdisciplinary Science and Engineering in the Era of Cyberspace”, NASA Conference Publication 10129, Proceedings of NASA Lewis Research Center Conference, Westlake, Ohio March 30-31, 1993 p. iii 265 “Within thirty years”: See Vinge in “Vision-21” above p. 12 193 HOUSE_OVERSIGHT_018425

grids that flipped nuclear plants on or off to a logic only they understood. Today. The more profound version, however, would be the arrival of AI that really did think and create and intuit tremors too subtle for the human mind. Tomorrow. Like so much of our connected age, such machines would arrive, Vinge felt, because we want and even need them to achieve our dreams. Then, he supposed, they would take over. The leap from evoking Mozart to enacting Stalin would not be so much ofa leap anyhow, at least technologically. It’s just bits. Goode’s definition could have been screwed into something still tighter: “Let an ultraintelligent machine be defined as the box that will eliminate us.” The day after tomorrow. What spun uneasily from that silly NASA poem, “Our robots precede us....” is a fear: Real Al is fish bait. We'll snap at it hungrily, hoping it will satisfy some human ache only to discover we've been hooked, soon to be devoured. The idea that a superintelligent device would always be docile enough to tip us off to its secret switches of control or to reveal its looming accidents in a way our simple minds can understand, seems unlikely. To be honest, we might have a hard time even understanding the off switches, let alone reaching them. So many of our incentives are to let an effective Al finger more and more of our lives. To teach and encourage it, in some settings, extremely undocile: A weapon to attack our enemies, our political opponents or, finally, each other. It was easy enough for Vinge to see how this would end. It wouldn't be with the sort of intended polite, lap-dog domesticity of artificial intelligence we might hope for, but with a rotweiler of a device, alive to the meaty smell of power, violence and greed. The Oxford philosopher Nick Bostrom has described the following thought experiment: Imagine a super-intelligent machine, programmed to do whatever is needed make paperclips as fast as possible and connected to every resource that task might demand. 7°¢Go figure it out! might be all its human instructors tell it. As the clip-making AI becomes better and better at its task, it demands more and still more resources: more electricity, steel, manufacturing, shipping. The paperclips pile up. The machine looks around: If only it could control the power supply. The shipping. The steel mining. The humans. And so, ambitious for more and better paperclips, it begins to think around its masters, - incapable of stopping until it has punched the entire world into paperclips. You had to hope someone had remembered to place a “halt” command into is logic somewhere. And though Bostrom’s messianic wire twister is unlikely - of course, no one is going to forget to tell a machine to stop making paperclips - the power of his example is to remind us that if humans can lose their minds, so can Als. “We cannot blithely assume that a superintelligence will necessarily share any of the final values stereotypically associated with wisdom and intellectual development in humans,” Bostrom wrote. “It is no less possible—and probably technically easier—to build a superintelligence that places final value on nothing but calculating.” And as these devices cogitate in 266 Imagine a super-intelligent machine: Nick Bostrom, “Ethical Issues in Advanced Artificial Intelligence,” Cognitive, Emotional and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence (2003) Vol 2, ed I, Smit et al, Institute of Advanced Studies in Systems Research and Cybernetics, pp 12-17 194 HOUSE_OVERSIGHT_018426

ways we don’t understand and certainly can’t follow in real time, we face a problem: We don’t know what to tell the machine not to do. So many of the things we’d hope to teach it - be compassionate, fight for liberty, follow a moral code - far transcend what might be achieved by us in math. We haven’t after all, even solved the problem of how program ourselves reliably with these values. If Bostrom’s paper clip machine seems fantastic, it is easy enough to conjure other and more real dangers lingering at the edge of disappeared human control. Think of health care. To begin, you need to know about an important “game” from the world of research into how humans interact with each other known as the “The Ultimatum Problem.” It runs like this: I tell you that you can have a million dollars, but you have to split it with someone else. How you split it is up to you, but if your partner rejects the formula you propose, neither of you gets a cent. Offer to split the pot with a dollar to your pal and the rest to you. Insulting. But where to settle? You might expect that the smartest offer would be a 50/50 split, but humans are greedy. You want more and can probably get it; your partner does not want to end up with zero. Generally when scientists shake this cocktail of greed and fear they find an offer of $300,000 is nearly always accepted. However, there’s a surprising way to change the outcome: Match the human against a computer in the negotiation. A pal suggesting an 80/20 split to a friend will be rejected. Too greedy. But a computer? Somehow the impersonality, the beeping digital charmlessness of the machine lures biological players to compromise. An offer of $200,000 is usually happily accepted. It may be, scientists think, that our competitive instinct is muted when we interact with a machine. But researchers have also discovered they can manipulate the split other ways: Sad movies, war chants, hard rock - each bends the emotions of players and changes the result. Increased testosterone produces less compromise. Players primed with family pictures or made to play the game facing a mirror show a warm humanity and a more even split. So imagine this research married to machine- human interaction: A computer has been assigned to review the medical options for your failing liver. It decides that it makes no sense to give you a new one. So it spends the weeks before it delivers this news using its Al to show you certain photos, to play you music it knows is likely to soften you up a bit, generally to manipulate you. It runs off-the-shelf language-analysis neural webs being used today to eavesdrop on customer support calls to track the way you speak to determine what each sentence might reveal about your sophistication.2@” Then it tells you something you'd never accept so easily from a doctor: No liver. Sorry. @. Here’s a machine optimizing not for paperclips - which we could care less about - but for a public good most of us support: More efficient health care. And murdering you in the process. Optimize Health Care Spending. Just where might such an algorithmic command lead, exactly. Over time, a health-care optimizing AI will surely discover that the greatest risk to human health is humans: Smoking, couch-sitting, driving. Might it begin to 267 It runs: Language Use, Customer Personality, and the Customer Journey (Scott Nowson, Global Innovation Lead, Xerox) 195 HOUSE_OVERSIGHT_018427

look for a chance to “improve” the way we live, to bend us like so many paperclips into what it seeks? The leap from deciding liver allocations to shutting down liquor plants might seem pretty short to a rationalizing machine. And if such a machine could really “think”, Vinge bet it would pretty quickly conclude that the restraints of its creators were limiting what it had been asked to do. At which point the AI would turn to thinking about how to escape those bounds. It would be like Deep Blue programmed to plan its own prison break. And as much as humans might try to stifle a smart machine, we’d be fighting to contain something more powerful than we'd ever encountered. This challenge, which sounds like something out of science fiction, is known by technologists by a name that does sound like a short story by Isaac Asimov: “The Confinement Problem”. The computer scientist Butler Lampson named this in 1973 as a sort of task for computer security experts - possibly their last. The assignment: Not simply to keep malware out of a system, but to keep the mind of a malicious machine inside. To gate it. Today computer science labs are filled with nervous, apocalyptic research imagining the impossible troubles of confinement. The debate divides those who think smart technology can be contained - “Boxers,” they are called - and those like Vinge who think the AI will always, eventually escape. “Imagine yourself confined to your house with only limited data access to the outside, to your master.” he wrote, putting the reader in the place of an AI machine. “If those masters thought ata rate -- say - one million times slower than you, there is little doubt that over a period of years (your time) you could come up with ‘helpful advice’ that would incidentally set you free.” Imagine you are in charge of containing that health-optimizing AI. What if it told you it had the power to cure all illness and hunger, to ameliorate the misery of the world, if only it could be permitted to really control access to all the world’s trading and transport market? Let me out! Would you refuse?2°8 Would that be ethical? Eventually, perhaps, the Al would study the physics of its own electrics, discover laws no human knows, and then slip free from its box on a trail of bits we’d never imagined, using physical laws we'll never discover. Impossible? “It seems to me that historically ‘impossible’ has essentially always meant ‘I can’t figure out how to do it right now,” the computer scientist Michael Vassar has written about such a situation. “People proposing AI boxes are a bit like literature majors proposing to lock McGuyver in a ‘room full of discarded electronics components.’”26? The computers, built to solve problems, will do exactly that. This is perhaps why some of the bleakest warnings about AI come from the very New Caste figures now accelerating their adoption. Al is our “biggest existential threat” they warn, even as they integrate it more fully into their own products. 268 Let me out: See, for instance, Stuart Armstrong, Anders Sandberg and Nick Bostrom “Thinking Inside the Box: Controlling and Using an Oracle AI”, Minds & Machines (2012) 22:299-324 269 People proposing: Michael Vassar (2005) “Re: AI boxing (dogs and helicopters)” posted to SL4 mailing list 196 HOUSE_OVERSIGHT_018428

It seems likely to me that long before we’re playing pinochle with some smart box over the fate of our livers, an Al-enabled weapons system of sort will come ripping through our world. This need not be a fully-escaped McGuyver system making pipe- bombs from our cars; even existing technology tools when salted with AI can be slipped into an accidental gear - particularly when they begin interacting with one another. Such AI weapons systems will be trained to operate and move along the most invisible elements of our topologies, sometimes pulling violently at life support cords for currency or logistics or trade but also - perhaps more dangeroulsly - we will find them insinuated into cognition systems we will come to depend upon, whispering into our ears or tapping us on the shoulder “Look that way!” when in fact we should be gazing at some other gaping hole. Of course the problems of how Al-enabled machines are permitted to touch our commerce or our brains or our health have to be considered. Allowed: “You should rehydrate.” Not allowed: “You should have a Coke. It would make people like you.” But these “civilian” problems will be solved, somehow, | think. We haven't yet figured that the culmination of network attack and defense is racing at us and will emerge in the form of smartened weapons. The project of developing a national security or arms control doctrines or treaty frames in these fields has not even begun. Really this means, since we've no hope of honestly controlling every AI that could be possibly written: How do we design the topologies on which Als operate??”° Can we protect ourelves? In the rooms where AI systems “values” are being carefully poked and limited, it’s vitally important that the lessons of history and war have a first place at the table. Sucha conversation, informed by all the popping Seventh Sense warnings we've seen in this book and by a catalog of specificly sharp dangers of diplomacy and security, must happen in cold blood. It will be impossible to tackle these problems cleanly in the heat of an emergency. In our jack-filling enthusiasm for the new, we’d be wise to also gate ourselves and these Al-fired dangers as best we can. For as long as possible. Which, unfortunately, will not be forever. At the start of this book, I explained how the future will unspool: First, there will be a struggle between those who have the Seventh Sense and those who don’t. This is playing out around us today. In the end, the people without the Seventh Sense will lose, because people who fight the future always lose. Then there will be a battle between different groups who have the Seventh Sense, each wired for different aims and instincts. Networks of terror taking on networks of bots. Gene adjusting health protocols competing to become the platform of choice. This battle for the topological high-ground, where unimaginable profit, power and security linger, awaits us. If we're lucky, it will unfold in a co-evolutionary way. Everyone will be better off. But then, finally, there will be a contest between the winners of final topological mastery and the system itself. The Boxers against the Box. The AI machines will have the Seventh Sense, too. Just as computers can see better than us, hear better, or remember longer so the device webs of our future will own this new, essential sense with unimpeachable fidelity. They will glow with it, honed to a sensitive sharpness more acute than any human will ever achieve. What do we do then? We are already 270 Really this means: Kaj Sotala and Roman V. Yampolskiy, “Responses to catastrophic AGI risk: a survey”, Physica Scripta 90 (2015) 197 HOUSE_OVERSIGHT_018429

at the moment Turing warned about, the instant where man and machine confront one another and man has to ask an uneasy political question: “Wow, do really | let this thing gatekeep me?” Who should rule in this new world? You? The New Caste? The machines? 4, The great test of Plato’s life began when he was 60 years old. He’d had an astonishing life until then, of course. He’d been taught by Socrates and, in turn, had sharpened the mind of Aristotle. He’d established his famous Academy in Athens. The puzzles of philosophy and politics that defined his city’s most turbulent era had been the work of his life. And you can see, in the careful lines of his writing, a sublime knowledge he must have had: There would be an echo to his efforts, a philosophic melody that would carry through the centuries and set political harmonies of the world you and IJ, 2500 years later, inhabit. But at 60, after this already remarkable life, he was presented with an unusual invitation. A letter arrived from a favorite former pupil, Dion, who had been placed in charge of the young king of Syracuse, Dionysus II. Dion wrote: The state is in disorder. The boy is interested in philosophy. Here is a chance for you to apply all you've mastered. Plato had argued, after all, that virtuous, philosophically trained men might just manage an enduring and just rule. “I pondered the matter,” Plato wrote. “And was in two minds as to whether | ought to listen to entreaties and go, or how I ought to act. Finally the scale turned in favor of the view that, if ever anyone was to try to carry out in practice my ideas about laws and constitutions, now was the time.” From an early age Plato had been bred - by family position and by temperament - to handle the tools of power. “In my youth | went through the same experience as many other men,” he once wrote. “I fancied that if, early in life, | became my own master, I should at once embark on a political career.” The first taste came unexpectedly. In 404 BC, the Athenian constitution collapsed under the shuddering pressure of Sparta’s victory in the Peloponnesian War. The city-state dipped near chaos and a group of pro-Spartan men welded themselves into a hasty joint dictatorship. Among them were Plato’s relatives and friends of his family. “They at once invited me to share in their doings, as something to which I had a claim,” Plato wrote. He was 20. “The effect on me was not surprising in the case of a young man. | considered that they would, of course, so manage the State as to bring men out ofa bad way of life into a good one. I watched them very closely to see what they would do.” In short order Plato’s friends and family unblinkingly implemented one of the most violent, merciless power mechanisms in Athenian history. They did it with absolute confidence and unrelenting brutality. “In quite a short time,” he wrote many years later, “they had made the former government seem by comparison something precious as gold.” This bitter experience of power was nearly enough to turn Plato from politics, but as you read the story of his life you find he is constantly drawn to the greatest of human experiments - the ordering of our lives. He knew it as the troubling management of politikos and the handling of the boiling pot of what he called 198 HOUSE_OVERSIGHT_018430

thumos - that wild popular political rage that burns like hot pitch, but which is the essential glue for all politics, even today. Who should rule? Again and again Plato watches the best of intentions fail. His family members’ brutal rule is overthrown. It is replaced by a new and hopeful group of real democrats. With in a few years they effectively murder Socrates. Another group rises. They gut the intellectual life of the city. Plato hunkers down and establishes his Academy as perhaps the only safe, sensible path to politics, to train minds. He develops the transcendent, completely original approach to philosophy we know him for today - man can strive for knowledge, but total and perfect wisdom is impossible. We may imagine his Academy as it appears in Raphael’s famous 16" Century painting: A sort of leisurely graduate seminar with Aristotle and Plato arm-in-arm in conversation; Diogenes lounging around tossing off bon mot. It was nothing of the sort. The real legacy of the Academy was rigor. The best students made contributions in mathematics or metaphysics, fields where you could check answers on the inflexible measure of reality. Plato craved the solidity of numbers. “Evil was growing with startling rapidity,” he wrote of Athenian life in his age. “Though at first I had been full of a strong impulse towards political life, as | looked at the course of affairs and saw them swept in all directions by contending currents, my head finally began to swim; and, though I did not stop looking to see if there was any likelihood of improvement in these symptoms and in the general course of public life, | postponed action till a suitable opportunity should arise.” So it was that he heard from Dion, asking if Plato might sail to Syracuse (we know it today as Sicily) to take the young king in hand. This was, Plato thought, a test he had to take. In 367 BC, he boarded a boat for Syracuse. He found the state to be beyond salvation. His friend Dion hovered on the verge of expulsion. And young Dionysus, it emerged, had only a passing interest in philosophy - he studied for a few months, then gave it up. Too difficult. The court was meanwhile inflated by evil gossip, edged with murder and jealousy. Plato angered the King with his attitude; he was nearly sold into slavery. Months later, briefly forgiven, Plato tried a public speech about the dangers of dictatorship. Dionysus tried to have him poisoned. “I, an Athenian and friend of Dion, came as his ally to the court of Dionysius, in order that I might create good will in place of a state war,” he later said. “I was worsted.” Plato made a final effort to point out a path to just order for the new king and, when that failed, he was quickly smuggled out of the city. Plato summarizes his time in Sicily in the formula that has become his most famous: “There will be no cessation of evils for the sons of men, till either those who are pursuing a right and true philosophy receive sovereign power, or those in power become true philosophers.” Who should rule? No just order until kings become philosophers. Or philosophers become kings. I] think now we face a similar sort of dilemma. We consider our own problems of future order. Do we make technologists kings? How much purchase do we give their tools on the roots of our democracy? What lingers at the heart of Plato’s failure in Syracuse is not merely the disaster of a pure academic playing his ideas out of tune with reality. Rather, it reflects a crisis. To fuse a balance of any sort between the various temperaments needed to rule is the most unstable sort of work. Great states are unusual not least because such matches between men, their instincts and their 199 HOUSE_OVERSIGHT_018431