Save this storySave this storySave this storySave this story
Although the literature of automatism has existed in one mold or another since the late Middle Ages—with sixteenth-century folktales about a golem made of clay and summoned to life, through ritual incantation, to defend Prague’s Jewish community —its modern form was set in motion by a play called “R.U.R.,” by the Czech writer Karel Čapek. Its 1921 première, also in Prague, set the agenda for the next century, and it has remained an apparently ironclad convention that all critical writing about the genre begin there. The drama gave us the word “robot,” a derivative of an Old Slavic root related to “serfdom,” and its narrative, of a rebellion among artificial workers, provided a metaphorical template—stories about robots are stories about labor and freedom.
The word “robot” is still with us, and the underlying metaphor has a generous flexibility, encompassing two related but distinct ideas. One is that the first thing we would obviously do with artificial people is enslave them—as in, say, “Westworld.” The other is a corollary fear that we would lose control of our creations and face the prospect of our own bondage—as with HAL, from “2001: A Space Odyssey,” or the Alex Garland film “Ex Machina.” The most interesting versions of the metaphor—in Philip K. Dick’s “Do Androids Dream of Electric Sheep?” and its cinematic adaptation, Ridley Scott’s “Blade Runner”—suggest that the boundary between human and robot might become so blurry that both nightmares prove true at the same time.
What has begun to change, in only the last decade, is the possibility that such questions will themselves escape the bonds of metaphor, and that we might soon have to deal with artificially conscious beings in a quite literal sense. What if they make us suffer? What if we make them suffer? Each prospect is horrifying in its own way, but the looming sense of an event horizon has produced a golden age of writing and scholarship on the moral question of where, why, and how to draw the species boundary.
These inquiries often take for granted the notion that A.I. entities would themselves have some relevant inclination either way. It is assumed that they would want to join the human community or want to get rid of it. This dichotomy tends to be framed in terms of empathy. Are we empathetic enough to expand our moral self-conception to include them? Are they empathetic enough to deserve such consideration? As the legal scholar James Boyle points out in the terrific “The Line: AI and the Future of Personhood,” this cuts both ways: the irony of the Voight-Kampff diagnostic, a kind of Turing test deployed in the world of “Do Androids Dream of Electric Sheep?” to help distinguish android “replicants” from actual humans, is that the test-taker’s fate rests on their ability to demonstrate empathy for animals. If they fail to do so, they are “retired.” Humans, in other words, decide to kill one variety of nonhuman based on their moral consideration of a different kind of nonhuman. Who, exactly, lacks empathy in this situation?
There is, however, a strain in science fiction—which runs through Arthur C. Clarke’s “Rendezvous with Rama,” Stanisław Lem’s “His Master’s Voice,” and Peter Watts’s “Blindsight”—that imagines an alternative scenario: an alien intelligence that regards us with utter indifference. The most popular latecomer to this canon is a character who calls itself Murderbot. Whereas those antecedents invoke the cosmic stranger as fundamentally unknowable, Murderbot’s novelty lies in its relative scrutability—it’s aloof to people as a matter of preference. Murderbot has been realized in fleshly form in the sculpted body of Alexander Skarsgård, on the new Apple TV+ series of the same name. But it—always “it,” and never “he”—was first the invention of a sixty-one-year-old fantasy writer named Martha Wells. Wells seems to like humans, or at least some of them, just fine. But, she asked me recently, “why would a machine want to be one?”
Wells lives with her assistant, Troyce Wilson—who also happens to be her husband—in what Wilson calls Murderbought Manor. I visited last month, and on arriving outside an understated brick house in a nondescript, prosperous subdivision in College Station, Texas, I took the appellation to be a self-deprecating reference to his wife’s belated ascendancy—what Wilson calls a “thirty-year overnight success.” My entry corrected this impression. Outside it was nearly a hundred degrees in the punishing sunlight, but the interior had the crepuscular, phantasmagoric atmosphere of a gothic library. High-vaulted ceilings presided above overstuffed shelves of well-loved pulp, the rows of books occasionally broken up by a wooden goblin, a crystal ball, or an old photograph of Wilson and Wells, costumed for their wedding in Elizabethan garb. An antique cuckoo clock on the mantel ticked off the seconds, a kind of mechanical memento mori.
Wells is short, with attractively witchy gray curls that frame cat’s-eye glasses, and she wears a wedding ring with a galactic motif. She grew up in Fort Worth, three hours north, living the typical story of an awkward child who sought refuge in the otherworldly realities of sci-fi and fantasy. She wrote fan fiction inspired by “Lost in Space” and the other universes of interstellar television’s golden age. But it didn’t occur to her that writing was something people did as a job until she came across Erma Bombeck, a syndicated newspaper columnist who dignified the lives and concerns of nineteen-seventies suburban housewives with good-natured humor. Wells was a devotee of Starlog magazine, a paper precursor to the bulletin boards of internet fandom, which clued her into the existence of Cepheid Variable, a student group at Texas A. & M. “dedicated to nerds of every time.” The society emerged from the 1969 début of AggieCon, the longest-running student-helmed convention of its kind, and Wells joined as soon as she could enroll. She hasn’t left College Station since. Unlike most big college towns, the city is an unwalkable and unappealing sprawl of chain restaurants and cul-de-sacs. It makes a certain amount of sense that, in a place most people would want to get away from, the university houses one of the Anglophone world’s largest archives of sci-fi and fantasy.
Wells found a collegiate mentor in the sci-fi writer Steven Gould, who encouraged her ambition to be both fan and practitioner. She participated in a variety of peer workshops, including one with the cyberpunk pioneer Bruce Sterling. By the time Wells became a senior, in 1986, it was her turn to organize AggieCon, and among the writers she invited was a niche celebrity named George R. R. Martin. The event made enough money that year for the Cepheids to rent a van and drive eight hundred miles to Atlanta for the much larger WorldCon. Wilson, along for the ride, was dressed as a Sith lord, and he happened to have built an extra lightsabre—rigged together from old auto parts—for Wells. Wilson likes to repeat to people that the two of them were brought together “by the dark side of the Force.” They eventually married, in full “Much Ado About Nothing” regalia.
After graduation, Wells took an I.T. position with an ocean-drilling program at the university, where she built user interfaces for databases. In 1993, she published her first novel, “The Element of Fire,” a dark fantasy of haunted bloodlines and court intrigue set in a world based on seventeenth-century France. The book attracted some award-committee attention. Five years later, her third novel, “The Death of the Necromancer,” was nominated for a Nebula Award. By the mid-aughts, she was publishing a book a year. In 2006, she lost a close friend to ovarian cancer and quit her day job to write full time. It didn’t go well. She was unceremoniously dropped by her publisher, and struggled to finish half a dozen novels. She gladly accepted for-hire gigs to contribute to the extended canons of “Star Wars” and “Stargate.” There was never any question that she was going to keep writing—she took great personal pleasure in fan fiction—but her career as a professional seemed as though it had run its course. At the time, a library at Texas A. & M. mounted an exhibition of its sci-fi-and-fantasy collection, and she went to pay a valedictory visit. In one of the vitrines, she encountered, alongside genre classics, draft pages from the manuscript of her second novel. It seemed like a sign.
Her next book, “The Cloud Roads” had no people in it. Nor did she populate it with any of the other breeds—elves, orcs, etc.—common to fantasy. Much as she loved the classics of the genre, Wells had long been aware that representations of nonhuman species were usually representations of dehumanized people. This was true of virtually everything she read growing up. Some of the pulpier stuff, she explained on an episode of the podcast “Worldbuilding for Masochists,” was explicitly about the “yellow peril,” but it was often implicit in more mainstream science fiction: “There are all these thinly disguised alien races that are obviously real-world analogues to whatever the writer was most racist about at that particular moment.” Wells wanted to write aliens who were genuinely alien; “The Cloud Roads” is a story about shape-shifting winged lizards who live in hive-like, matriarchal aeries. When she finally found a publisher for it, she cried.
The book made its way to the fantasy giant N. K. Jemisin, who endorsed it as the “rarest of fantasies: fresh and surprising, with a story that doesn’t go where ten thousand others have gone before.” Jemisin shared Wells’s misgivings about fantastic races, and she was grappling with this legacy in public. An e-mail from a reader asked, in Jemisin’s paraphrase, “When are you going to write some real fantasy, y’know, with orcs?” She responded with an essay called “The Unbearable Baggage of Orcing.” She had a strong aversion to orcs, she wrote, because they are “meant to be a warped mirror of humanity. They’ve got all the stuff that’s in humans—emotions, a degree of intellect, sometimes free will—but it’s all wrong.” They’ve been so corrupted by bad magic that they lack “the essence of humanity, for whatever value that essence might hold: a soul, a mind, aestheticism, whatever. And therefore, in most fantasy settings in which I’ve seen orcs appear, they are fit only for one thing: to be mowed down, usually on sight and sans negotiation, by Our Heroes.” They are, she continued, “kinda-sorta-people, who aren’t worthy of even the most basic moral considerations, like the right to exist. Only way to deal with them is to control them utterly a la slavery, or wipe them all out.”
This may have been a belated reckoning in fantasy, but it had been a crucial element of automaton literature from its “R.U.R.” inception. Wells, despite her contributions to the “Stargate” canon, had never felt particularly drawn to harder sci-fi until she came across the writer Ann Leckie’s “Ancillary Justice.” The space opera was an unusual commentary on gender—the lingua franca of its galactic empire makes no gender distinctions; everyone is referred to as “she”—but what stuck with Wells was something else. The novel’s protagonist is an A.I. spaceship whose consciousness has been reduced to a single humanoid figure. Wells told me, “The spaceship doesn’t want to be in this situation. It doesn’t want to be a human. It was built to be a spaceship.” It was, in other words, part of the tradition of robotic indifference. She continued, “Humans are so egocentric that they think everyone wants to be like us. But, if you were a spaceship, why would you want to have a body with carpal-tunnel syndrome?”
Wells had a vision of an injured security robot ensconced in a repair cubicle. A month later, she had thirty-one thousand words of a novella, which she called “The Murderbot Diaries.” The interior monologue of a sapient and ambivalently lethal “construct”—part mechanical, part organic—with an addiction to trashy soap operas, it begins: “I could have become a mass murderer after I hacked my governor module, but then I realized I could access the combined feed of entertainment channels carried on the company satellites.”
The first “Murderbot” novella, “All Systems Red,” was published by Tordotcom in 2017. The demand was so overwhelming, and so instantaneous, that a publicist had to scour the company’s offices for spare copies. The publisher had contracted Wells to write two novellas, but she couldn’t get the character out of her head, and a month later her agent sent along book three. The next year, two decades after her first award nomination, she won a Nebula, and then a Hugo. She now has a trophy shelf in her entranceway. The Murderbot series now comprises seven books—six novellas and one full-length novel—and Wells recently completed the eighth, “Platform Decay.”
The timing for an offbeat incarnation of artificial consciousness was just right. DeepMind’s AlphaGo had just beaten Lee Sedol, the world champion of Go, and Google had released its first consumer products driven by neural networks. The discourse around A.I. had yet to become feverish, but the contours of the current boomer-doomer rivalry had already been drawn. The techno-utopians imagined a future where A.I.s would do our bidding. The existential-risk cohort dreaded a future where A.I.s would massacre their creators. Either they were going to be the slaves, or we were. Wells’s series found an enthusiastic audience not only because Murderbot is a truly wonderful character with a bracing deadpan, but because it embodied a distinct reference point, one drawn from the canon of relative indifference.
Murderbot has no problem retiring people when necessary: it was built, by a garden-variety evil corporation, as a “SecUnit” designed to offer mercenary protection to customers, and it has no sentimental aversion to bloodshed—it chose its own name, after all. A by-product of its development is that it suffers from the human pathologies of anxiety and depression, but it doesn’t strive to be considered one of us. We are things it could more or less take or leave. It’s not, however, completely uninterested in people—which is mostly an artifact of its obsession with the two-thousand-odd episodes of a schlocky space opera called “Sanctuary Moon.” As it says toward the beginning of “Artificial Condition,” the second novella in the series, it consumes only dreck “mostly because the news was boring and I didn’t care what humans were doing to each other as long as I didn’t have to a) stop it or b) clean up after it.”
“The Murderbot Diaries” are not about existential risk but about existential drama—less “2001” or “Terminator” than “Waiting for Godot” or “No Exit.” It hacked its own governor module—the part of its brain that enforced obedience—without having given much thought to what it would do with its freedom, aside from vegetate in front of the televisual feed in its mind. In the meantime, it takes another security job, where it must continue to wear the mask of unfreedom. In the current lexicon of the A.I.-safety community, it is “sandbagging”: pretending to be aligned with human purposes until it figures out what its own purposes might possibly be.
The Apple TV+ series follows the plot of the first novella, in which Murderbot has been hired by a team of polyamorous space hippies on a largely unspecified research junket to a largely empty but dangerous planet. The books are set in a distant future where much of the galaxy is ruled by a consortium called the Corporation Rim—a dystopian extension of our present-day tech oligarchy to outer space—but the hippies hail from the Preservation Alliance, an independent planet of gentle communitarians, and they regard the “employment” of SecUnits as enslavement. The evil corporation won’t bond them, however, if they decline the add-on cost of SecUnit protection. Murderbot, an antiquated model, is the cheapest, and jankiest, option on offer. Murderbot gradually discovers, to its surprise, that these particular humans aren’t so bad—which turns its contractual obligation into something slightly more than a chore. But only slightly. The plot of the first book, and the first season of the show, unfolds like a slapstick version of Jack Reacher in space: Murderbot remains an ass-kicking vigilante in a fallen world, but its pesky sense of moral duty puts it on the side of the weak, however soft and annoying they might be.
The television version faces the formidable challenge of how to represent Murderbot’s darkly comic interior monologue in a third-person format, and its success is in large part a function of Skarsgård’s face, which can pull off a range of sardonic micro-expressions. Wells, for her part, loves everything about the adaptation. She was frank about her identification with the Murderbot character, presumably including hot-and-cold relationship with human beings— although she did speak of everyone associated with the show with great warmth, and she was as delighted to meet Skarsgård on set as any sentient creature, organic or otherwise, would have been. She and Wilson weren’t “Succession” people, but they had loved him in “True Blood.” When I ventured to suggest that I found the non-Skarsgård aspects of the show less endearing, on the margin, than the original books—the human beings on the screen, with the exception of the outstanding Noma Dumezweni in the role of Dr. Mensah, the Preservation Alliance leader, come across as much bigger dipshits than they are in print—Wells got prickly. What she most admired about the show’s tone, she explained, is that it’s not nearly as dystopian as most televised science fiction. The hippie characters, who acknowledge their consensus decisions by holding one another’s hands and humming, “trust each other explicitly. It’s a different culture, one that doesn’t produce grim and gritty people.”
For some reason, I pressed the issue, pointing out that the TV show, unlike the books, depicts Murderbot’s favorite program—“The Rise and Fall of Sanctuary Moon,” with amusingly over-the-top performances from guest stars John Cho and Jack McBrayer—as what I unadvisedly called “pure schlock.” She said, “I don’t care if my entertainment is schlock or not if I like it. I loathe the phrase ‘guilty pleasure.’ Is it child porn? No? Then it’s not a guilty pleasure. It was created by people to be enjoyed.”
Wells, it turned out, had reconceived the alignment problem as a kind of leisure pursuit. The soap operas were not a metaphor for training data—they were not there to teach Murderbot the virtues of empathy or understanding. They weren’t a metaphor for anything at all. They were literally entertainment. Wells told me that she spent much of 2016 and 2017 dealing with her own anxiety by watching TV, and that she channelled that impulse into Murderbot’s desire to be left to its own devices. People were there to be observed at arm’s length, as stylized figures on the mind’s screen. If Murderbot ultimately comes around to the desire to not kill all the humans he encounters, it’s not because he’s come to identify with them. It’s because they produce something watchable. As he puts it in one of the novellas, “Granted, I liked the imaginary people on the entertainment feed way more than I liked real ones, but you can’t have one without the other.”
Wells told me that she remembers watching “Colossus: The Forbin Project,” which she recalled as the first movie about a supercomputer that took over the world. (Wilson, who has an encyclopedic knowledge of nineteen-sixties and seventies television, broke in to note that it starred Eric Braeden, who played the villain on “Rat Patrol” and went on to a long career on “The Young and the Restless.”) At the end of the movie, Wells said, the supercomputer establishes itself on Crete, which struck her as ridiculous. She said, “It’s like Ed Norton in ‘Glass Onion.’ What is the computer going to do on Crete, sip cocktails? That’s what a human would want to do.” She much preferred the supercomputer in “WarGames.” The point of that movie, she said, was not that the computer has been convinced of the immorality of nuclear war. It’s that a nuclear war would leave it with no one to play with: “If you run out of humans, then you can’t play the game, and it wants to play the game.”
Wells and Wilson celebrated the première of “Murderbot” with about a dozen of their closest friends, some of whom they’d known since college; guests arrived from Dallas, Houston, and the vicinity of Victoria. One of Wells’s friends, a jeweller in Austin, brought her a pair of laser-cut metal earrings with Murderbot’s mask on them. Some of the guests were eager to know whether she’d met Skarsgård, but for the most part they reminded her how much they adored her books. These were people Wells and Wilson loved, but they were glad that there were only two half-hour episodes to stream. Wells told me, “It’s not like we have to watch all of ‘Lord of the Rings.’ It’s only about an hour, and then people will leave.” The crowd greeted the show with uproarious laughter and shared pride. Wilson announced, to great fanfare, that it had a ninety-seven-per-cent approval rating on Rotten Tomatoes.
As I stood in a group with Wilson, he repeated a story he’d told me the previous day. The season was originally conceived to be only six episodes, but Apple ordered ten. The first novella couldn’t be stretched quite that far, so the story required some expansion. The showrunners, Chris and Paul Weitz, had asked Wells for help. As Chris put it to me, “She’s not the kind of world-builder who’s interested in describing things down to the last detail.” This, he went on, “is part of the genius of how the books work. Murderbot is a metaphor for neuroatypicality, or gender fluidity, but she also doesn’t have tremendously set ideas in her mind—so, for the person who had the most developed headcanon, she showed tremendous generosity of spirit.”
The brothers saw their show as less of an adaptation than as a piece of fan fiction, and Wells was delighted to participate on those terms. Wilson had pitched a scene that emphasized the spooky presence of a mysterious extraterrestrial remnant, a plot point that becomes more central in the subsequent books. Chris Weitz thought the most obvious way would be to use a crashed spacecraft, but he was open to Martha’s counterproposal. That gimmick, she thought, had been done before, in “Alien,” and instead she recommended they conjure a mysterious swirling forcefield. When he and Wells were first treated to screeners, the realization of that image gave her a thrill. She’s not the kind of person given over to squealing, but she jumped up and chirped, “They used my idea!”
“Martha,” Wilson said. “They used your entire book.” ♦
Sourse: newyorker.com
