Today at Facebook’s F8 developer conference, CEO Mark Zuckerberg laid out the company’s vision for the future. The short version: augmented reality blanketing a digital layer over everything you see and touch. The long version? A potentially thorny tangle of interactions in which your smartphone offers not just access to the world around you, but becomes the main window through which you experience it.
Augmented reality itself no longer feels like a great unknown, thanks (really!) to Pokémon Go’s glorious few weeks as mid-summer obsession. Facebook’s ambitions outstrip spurts of Snorlax-hunting, though. The company imagines a world in which AR doesn’t just enhance experiences, it defines them.
The result appears to be a world of technological marvels that both add to your immediate environment and put you at a remove from it. It also raises a question: Does filtering the world through your smartphone work to augment reality, or supplant it?
Facebook’s augmented reality dreams have a name. In fact, they have a whole platform: Camera Effects, tools available to developers that work with the camera function across all of Facebook’s apps.
“We’re making the camera the first augmented reality platform,” said Zuckerberg at the F8 conference.
The possibilities Facebook teased Tuesday have limits defined only by one’s imagination: A bowl of cereal surrounded by tiny, leaping, animated sharks. Artwork splayed on walls that, in base reality (reality prime?) are blank. Leaving a virtual note for a friend next to a menu at a local restaurant so that you can recommend the corn fritters.
On the giant display that backstopped Zuckerberg’s presentation, these interactions all seemed vibrant, and innovative, and downright cool, a special-effects version of everyday life. What the demo neglected to include, though, was a pair of hands, holding up a 4.7-inch display, swiping away notifications so that you can just watch the dang breakfast sharks. Pulling off convincing augmented reality on smartphones takes remarkable know-how. It’s also potentially a pain.
First, the more mundane but very real hardware issues that inevitably spring from having billions of users with all manner of devices.
“There are a couple of issues,” says Alan Craig, a human-computer interaction expert at the University of Illinois’ Institute for Computing in Humanities, Arts, and Social Sciences. “One is that phones are small, in terms of computational power, memory, things like that.” Facebook has also chosen not to offload those computations to far-flung servers, because the potential latency can disrupt the effect. That also, though, puts a lot of demands on hardware that's not necessarily up for it.
Consider, too, that not all smartphones are manufactured equal, especially where it counts most. “It is known that wide ranges of mobile-phone-quality cameras are much less accurate in estimating the 3-D geometry of the real world,” says Allen Yang, who heads up UC Berkeley’s Center for Augmented Cognition. “If AR is to be used in more demanding applications ... low-quality cameras to support AR might run into problems.”
For the near-term, that may not prove much of an impediment. Facebook’s immediate focus appears to be on small-scale effects; think more ambitious Snapchat filters. As its augmented reality vision scales up, though, it threatens to leave a chunk of users behind, or with stuttery experiences.
That’s also secondary, though, to what may be the bigger long-term Facebook AR ask---that we absorb a new version of the world through handheld OLED displays.
“Augmented reality is going to help us mix the digital and physical in all new ways,” said Zuckerberg at F8. “And that’s going to make our physical reality better.”
The first half of that quote is inarguable. The second half? Well, that probably depends a lot on how one defines better.
If it means “more information-dense,” then sure. In Facebook’s vision of AR, you’ll have all the information you can hope to handle. See a wine bottle on a table? AR could serve you up details about the vintage, find some tasting notes, and potentially even provide a link to purchase yourself a bottle. You deserve it!
In small doses, it makes all the sense in the world---especially if you’d just be spending the same amount of time Googling that same wine bottle on your smartphone anyway. On a global scale, though, one in which every storefront and every street corner gets stocked with digital signage and surprises, it could easily overwhelm, especially once Google and Apple and everyone else introduces their AR solutions as well.
“It’s pretty easy to imagine this getting overwhelming,” says Blair MacIntyre, an Georgia Tech augmented reality researcher currently working on Mozilla’s Emerging Technologies team. “Picture a group of college students at a popular off-campus meeting spot,” where the AR interactions created by your friends would pile up nightly.
Facebook actually may be better suited to solve that problem than anyone else, thanks to its extensive social graph that contains minutely detailed data about its billions of members and beyond. “They can use the same techniques they use to curate your feed in the first place to curate your AR feed,” says MacIntyre. It could show you not only just AR experiences created by your friends, but the ones you’re most likely to care about.
Regardless of the merits of AR itself, using a smartphone to access it has some serious drawbacks.
“The phone has generally sucked for AR because holding it up and looking through it is tiring, awkward, inconvenient, and socially unacceptable,” says MacIntyre. Adding more of it doesn’t solve those issues. It exacerbates them. (The exception might be the social acceptability part; as MacIntyre notes, selfies were awkward until they weren’t.)
Even Facebook knows that mobile handsets represent an AR stopgap, at best.
“We all know where we want this to get eventually,” said Zuckerberg in his keynote. “We want glasses, or eventually contact lenses, that look and feel normal but that let us overlay all kinds of information and digital objects on top of the real world.”
Despite the flaming wreckage of Google Glass, a face-worn solution done right potentially solves so many problems of mobile AR. Eyewear would free up both of your hands. You could take in the world on its own terms, without reaching into your pocket every block. And it would make for a more immersive experience, with AR effects moving with you as you swivel your head.
The optimistic timeline for that sort of tech, though, stretches out to five or 10 years. In the meantime, then, an imperfect solution takes the stage.
That carries its own risks though. “People tend to, in my experience, judge something new by their first experience with it,” says Craig, who notes that lukewarm reception to virtual reality in the '80s effectively back-burnered development until recently. “If what they see is not compelling to them in their life, they’re likely to dismiss it and miss what comes along later.”
If Facebook’s mobile AR play succeeds, it glues us even more securely to our smartphone screens. If it falters, it turns people off to the next big tech evolution. In the meantime, get ready for a fresh patina on the world, whether you choose to see it or not.
