The future is augmented

You are midway through the fourth book (of seven) in your favourite magical teenage warlock series. On her way to a first confrontation with the saga’s principal nemesis, our hero sprints down a bustling medieval thoroughfare, past apothecaries and creature ‘shoppes’, through a chaotic market heaving with alien fabrics and bottled oddities, until…

Wait a second. This place is interesting. You like it here. You’d quite like to postpone the showdown with Evil Whatshisname for a while and have a bit of a look around.

On the margin of the page you see a little QR Code, a scrambled chess board of tiny squares.

You scan it with your smart phone, and the micro-world of Potion Quest™ blossoms open on your screen like a flower: cobbled lanes, a starry, supernova-streaked sky, and - oh look! - that furry flying reptile sidekick from Book 2. A glowing arrow on the ground invites a point of entry, like Google’s Street View: and you tap the screen to step towards an animated vulpine vendor, who directs your gaze to the assorted bric-a-brac on his table. Potion Quest™ phone cases in a range of colours - just £11.99! Click?

Extra- (or intra-) textual experiences are nothing new; the book has long been a doorway to narrative experimentation in time or space. Children of the 70s and 80s will remember the choose-your-own-adventure stories driven by a rudimentary but addictive branching storyline mechanism (turn left? = go to page 85 = get slowly eaten alive again by the ravenous ghoul). Of wilder ambition was Kit Williams’ Masquerade, a million-selling storybook of clues to the location of a genuine 18-carat gold jewelled hare buried in a park in Bedfordshire, a folly that nearly ruined the life of its author when hint-hungry hare-seekers began converging on his doorstep. ‘Readers’, in the broadest sense, crave these giddy compulsions: Masquerade’s spiritual descendent is the immersive experience now widely offered by theatre companies such as Punchdrunk, or the LARP (live action role-playing) game worlds now staged anywhere people care to assemble, at varying degrees of sophistication and expense (the Potter-themed, scandal-hit "College of Wizadry" is its most infamous incarnation).

So while books are sometimes enough, in and of themselves (have you read Piranesi yet?), they often nowhere near enough, acting, as they do, as a kind of appetizer for a more gigantic imagined world, where the story’s most memorable ingredients might be reconstituted into a heartier, more satisfying meal. This happens already, and prolifically, in fan fiction, that sickly postmodern smoothie of pop culture hybridisation (Doctor Who Versus My Little Pony, anyone?), or in the deluxe-format, nested storytelling of supplement-loaded ‘event novels’ such as J.J. Abrams’ and Doug Dorst’s S: Ship of Theseus, with its pull-out maps, postcards, and assorted ‘hand-written’ documentary ephemera.

Might the humble book’s next evolutionary leap be the ‘augmented’ novel, a symbiotic marriage of the physical (or digital) book with the uncanny, extra-textual sublime of the video game? The ‘open-world’ game genre, a sizeable chunk of the global £100 billion video games market, allows a customisable character to explore the Wild West ("Red Dead Redemption 2"), mythological Scandinavia ("God of War"), or a buggy, generically dystopian near-future ("Cyberpunk 2077"). These worlds are not ‘open’ in any meaningfully infinite sense (think lots of locked doors, empty drawers and indifferent non-player characters), but they can be explored at leisure, and at their best they awaken the dormant visceral responses articulated by the 18th century philosopher Edmund Burke in his Philosophical Enquiry into the Origin of Our Ideas of the Sublime and Beautiful:

“Whatever is fitted in any sort to excite the ideas of pain, and danger… or operates in a manner analogous to terror, is a source of the sublime; that is, it is productive of the strongest emotion which the mind is capable of feeling.”

In other words: it doesn’t feel like you are only playing a video game when you are trying not to get stomped by an ice giant, or hiding behind a flimsy wooden barrel while a rival moonshiner is trying to mow you down with a Gatling gun. You are there, in the thick of it - as an avatar, yes, but your heart is thumping regardless.

The authentic sublimity of these encounters depends on the visual integrity of environments created in game design software such as Unreal Engine, which allows a character to move through and interact with a gameworld, while acted on by an approximation of the physical laws of our reality (walk into a wall and you’ll bounce off, fall off a tall building and you’ll violently die). Unreal Engine, when combined with textures and lighting imported from other applications, can now produce such high-fidelity sensory experiences that the technology is now utilised in top-tier television entertainment such as the Star Wars property "The Mandalorian"; much of the series is filmed on a wraparound stage set called a ‘volume’, with an environment projected from Unreal onto a 360 degree background that can be virtually lit and re-lit in real time (a burgeoning cinematic art form known as ‘machinima’).

Anyone with a gaming PC, and the patience for a brutal learning curve, can now learn how to create convincing, explorable 3D virtual worlds. And anywhere timid humans are still reluctant to go, artificial intelligence has already reconnoitred: the algorithms of the Open AI project can not only synthesise passable prose in a variety of genres, but are now also able to generate 2D and 3D images, across a spectrum of artistic styles, via simple text prompts such as “an armchair in the shape of an avocado” or “a snail made of a harp” (cue everything from the whimsically banal to the genuinely horrifying).

So if you don’t fancy a stop at the Potion Quest™ market just yet, you may be happy to let the text just prompt itself into spawning a virtual environment that gradually colonises the computer screen next to your reading chair, waiting for you to engage it at your leisure. Or if you feel there is a better seven-book warlock series to be written, it might serve as a tool for strategizing your own spatial imagination (I once asked the novelist Lionel Shriver if she had physically mapped out the school execution scene in We Need To Talk About Kevin, but she artfully tiptoed around that question).

How readily will book publishers embrace this augmented future? To Burke’s “pain, danger and terror”, add “the bottom line”: how many £11.99 phone cases would you need to shift in order to hire a brilliant computer arts intern with a high-end laptop?

Alastair Hagger is a PhD researcher at the University of West London in screenwriting and adaptation studies, focusing on the 18th century outlaw Dick Turpin.