Atman Binstock was working late one night in the summer of 2015 when he saw a door open that shouldn’t have been open. There were only two keys to the room, and for good reason: That’s where the Oculus team kept the Toybox demo.
The Facebook-owned VR company had just come back from the E3 video game trade show, where it had used the demo to show off the capabilities of its new handheld controllers. In Toybox, you could build a house of blocks, set off mini-rockets, even play ping-pong, just by reaching out and using your hands the way you normally would. Perhaps best of all, you could do all those things with another person. Toybox showed not only that VR wouldn’t feel like playing a video game, but that it wasn’t going to be isolating—that, as WIRED wrote around that time, VR could allow people to be alone, together.
After E3, Binstock’s team had rebuilt a demo pod back in the office so that more Oculus employees could try it, but there had been … incidents. Hence the locked door, and hence the two keys. Can’t have a free-for-all. But now that door was open. Aw, man, Binstock thought. I’m going to go ruin somebody’s night. He poked his head in the room, ready to drop the hammer, and instead found his boss. His boss’s boss, really. There was Mark Zuckerberg, who the year before had (in)famously bought Oculus for around $2 billion.
“Oh, hi, Mark,” said the chief architect of Oculus. “Need any help?”
“No, I’m good,” said the chief executive of Facebook.
So Binstock watched as Zuckerberg practiced hosting a Toybox demo with a prototype headset and prototype controllers.
“You’ve got to remember,” Binstock says now, “these things are cranky. It takes forever to even start them up and debug what’s going wrong.” But as he watched, it became clear that Zuckerberg wasn’t there to try the demo; he was there to practice. He had a routine. He had a patter. Binstock realized that the man who had once said VR would “change the way we work, play, and communicate” had spent hours getting good at this, just so he could be able to share his vision of VR personally.
To say that a lot of things have happened since then—to Oculus, to VR, to Facebook, and to people’s trust in all three—would be an understatement on the order of “2020 was weird, huh?” All of Oculus’ original founders have moved on, a scrappy team giving way to Facebook Reality Labs, a massive AR/VR division that may constitute as much as 20 percent of Facebook’s entire workforce. The Oculus Quest 2, VR’s multimillion-selling device of the moment, is half the price and far more powerful than the Rift, the company’s first mass-produced dedicated headset. Facebook has waded deeper into the hardware space with the Portal video-call device, and a year of pandemic lockdown has been very kind to both. What time has been less kind to is public sentiment; between its complicity in the disinformation campaigns of the 2016 election, privacy issues that arise from its ad-driven business model, concerns about AI bias, and other issues, Facebook has found itself on defense far more often than any company would like.
Yet all that change has made this week in particular a good time to take stock: It just happens to be the five-year anniversary of the Oculus Rift. Over those five years, despite everything, Facebook has solved an astonishing number of problems. And as the company looks ahead, those issues—as well as ones yet unsolved—figure prominently. From its Luxxotica smart glasses coming later this year to the far-flung future Facebook is imagining in plain view, Zuckerberg has maintained his convictions about AR and VR’s inevitable ubiquity. The technology has survived its initial lean years, but going from a few million users to a billion means far more than just adding a couple of commas. The question is if the bet pays off.
Think back to those first few years of the current age of virtual reality. The first Rift prototype showed up behind closed doors at E3 in 2012. That fall, Kickstarter users ponied up nearly $2.5 million to get their hands on the first developer version of the headset. Headquartered in Southern California at the time, Oculus began to grow. Fast. 2013 brought nearly $100 million in funding. As it grew, it started working out many of the kinks that had plagued VR the first time around in the ’90s. When the Rift finally came out (with the HTC Vive and PlayStation VR not too far behind), the headset managed to do something no predecessor had: deliver stable and comfortable virtual reality for the price of a game console.
But that delivery wasn’t easy. The headset needed a high-end gaming PC to power it, and its cables snaked everywhere. It needed external sensors to track its position in space, which added yet more cables and hardware. It wasn’t uncommon for early adopters to encounter driver updates and USB port errors that demanded a monk’s patience to figure out. Just because some problems had been solved didn’t mean the solutions weren’t stopgap measures, and there were miles to go before VR headsets would be as intuitive and turnkey as a smartphone.
So the work continued, as it did at other companies. But acquiring Oculus had only been the beginning; Facebook had also begun pouring resources into supercharging the company’s internal research pipeline. “It felt like if we put a lot of time and energy behind it, we could accelerate this into something that could get wide adoption—because that's the only way we would all be interested in it,” says Mike Schroepfer, Facebook’s CTO. “If it was some super-niche high-end very expensive toy, it just wouldn't fit with what Facebook was trying to do. So from the very beginning, it was, ‘Can we take this thing and turn it into something that everyone can have?’”
Something “everyone can have” had been a priority for Zuckerberg since well before he acquired Oculus in 2014. “I talked to Zuck in 2012, when I was originally recruited to Facebook,” says Caitlin Kalinowski, who’s now head of hardware for Oculus, “and he understood already where the company would need to go in terms of owning a hardware portion of the next platform. I don’t think he knew what it was yet, but he really understood VR’s potential.” First had come a dedicated team in Seattle, where chief scientist Michael Abrash and Binstock had begun digging into VR’s thorniest problems; later came facilities in nearby Redmond and across the country in Pittsburgh, where an ever growing phalanx of PhD-level specialists sought to untether VR and push immersion as far as possible. Back in Menlo Park, Kalinowski and her colleagues worked to turn the emergent technologies into product form.
As money flowed in, progress flowed out. First came the Oculus Go, in May 2018. It was wireless but couldn’t track itself in space, constraining the experience to something more like a cell-phone-powered device. (Remember Samsung Gear VR? Google Cardboard?) A year later, though, the Quest fixed that too; the company had figured out how to integrate outward-looking sensors, finally getting over the hurdle of “inside-out” tracking. Then, in December 2020, a sequel followed: the Quest 2. In the space of five years, Facebook had increased its annual R&D spending from $5.9 billion to nearly $18.5 billion. It had also turned its flagship VR headset into something that was a big step closer to a mainstream device, at half the price of the Rift.
Perhaps more significant than the headset itself, though, is the financial potential that the VR ecosystem has begun to realize on the software side of the equation. In 2016 a game called Raw Data became the first VR title to bring in $1 million in revenue. By the beginning of 2020, more than 100 others had joined it. And that’s across all VR platforms; on the Quest line specifically, fully one-third of titles for sale have done the same.
That means breakout games like Beat Saber and Onward, but it also means two of VR’s most interesting use cases: social worlds and fitness apps. Rec Room, a multiuser social platform that lets users build their own worlds (and even get married), was recently valued at $1.25 billion after sextupling its user base in 2020 alone, making it one of VR’s first unicorns. Facebook's own social app, Horizon, is getting closer to a wide release—"Now we have enough people to really bring those communities out and populate those worlds," says Meaghan Fitzgerald, head of product marketing for AR/VR—and the company recently announced a new avatar system that uses your own speech to drive your avatar's mouth movements and expressions. (It’s almost a given that eye- and face-tracking will be in some future version of VR headsets to improve that further. "If you don't have eye-tracking so you can make eye contact with someone, and if you don't have face tracking so you can naturally emote," Zuckerberg says, "it's not going to be the best social platform.")
Exercise-focused titles like Supernatural and FitXR are seeing impressive growth in both audience and results. Supernatural—which charges users a Peloton-like $20 monthly subscription fee for the privilege of coach-led cardio classes, lunging and swinging at multicolored orbs to the beat of curated playlists—boasts a thriving Facebook community where users upload videos of their daily workouts.
“We continue to see a really wide spread in our demographic, not only in terms of age and gender, but also in terms of fitness ability,” says FitXR cofounder Sam Cole. “We have people who say things like ‘I’m a bodybuilder and I hate doing cardio, but this is the way I get my fix’ through to people who are sedentary and have really struggled with fitness their entire lives.” Yet the app’s users average a ring-closing 35 minutes of activity per day. “One of our customers said to us recently that this feels like the best thing to happen to exercise since exercise,” Cole says.
For Mike Verdu, a games executive who came to Facebook in 2019 to head AR/VR content, that’s a telling inflection point. “I think we finally got a use case that can go broad and lends itself to sustained use over a long period of time,” he says of the fitness sector. “You weave it into the fabric of your life, you do it every day, and companies have to get good at delivering fresh workouts and music and content.” VR as a service, in other words. Verdu sees other use cases on the horizon like creative tools and productivity utilities—some of the teams in the org hold weekly meetings in VR, using an internal app that lets them gather in virtual conference rooms—but the important part for him is that content creators are finally seeing the fruits of their labor on a promising but untested platform. “I think we're just scratching the surface on what VR is capable of,” he says. “It’s just exciting to see developers leaning in on experiences that will be around for a long time.”
What “long time” means in the world of games and apps is one thing. What it means in the larger timeline for VR and AR, though, is another. Companies continue to sink money into the technologies; Apple in particular is the subject of another story seemingly every week detailing its explorations around a high-end VR headset or AR glasses. Facebook might have been a first mover, but it also wants to be the last mover, so it’s trying something different: It’s playing a very patient game, out in the open where everyone can see.
When the first Quest headset hit the market, the teams that were still grouped together under the Oculus umbrella had a new leader—and their old ones had long since departed. Boy wonder founder Palmer Luckey had left Facebook in 2017 under a company-imposed cloud of secrecy. Onetime CEO Brendan Iribe departed the following year, and VP of product Nate Mitchell moved on in 2019. By that point, Mark Zuckerberg had tapped Andrew “Boz” Bosworth, who was leading Facebook’s ads and business platform, to head the company’s larger AR/VR division. (Bosworth credits Michael Abrash, who he calls “the keeper of the flame” of Facebook’s ambitions, with helping convince him to take the gig.)
Under Bosworth, Facebook’s hardware ambitions have swelled. According to the Information, his division—which last year changed its name to Facebook Reality Labs—oversees nine teams that include VR, AR, Portal, and Devices. As many as one in five Facebook employees reportedly work on them. Some of the fruits of those teams are obviously already on the marketplace; the Portal family, now four items deep, has proven to be a favorite among the work-from-home set (read: everyone). Others … aren’t. Facebook’s first smart glasses, a Ray-Ban–branded collaboration with Luxottica, are expected later this year. A smartwatch reportedly may not be far behind. Zuckerberg has alluded to future iterations of the Quest.
And then there’s the Big Kahuna. The mythical augmented-reality glasses that represent Facebook’s endgame. (Let’s just forget about brain implants, shall we?) It’s a vision Abrash has spooled out at developer conferences for years, and one that the company has grown increasingly comfortable talking about. Imagine a piece of eyewear that can overlay virtual content on top of the real world. That could mean simple things like games or a keyboard, or it could mean the photorealistic avatars that the company has been working on in its Pittsburgh research lab. You know that scene in Kingsman: The Secret Service where they have a meeting with a bunch of people who aren’t even there? That.
Even better, imagine that the glasses’ computer vision capabilities—which will by then be generations past those used for the Quest’s inside-out tracking or the Portal’s auto-framing technology—can see the world the way you do, and utilize an unobtrusive but powerful assistant that can do things like reduce background noise in your earbuds or translate signs in other languages. That’s something that doesn’t just get you off your phone; it replaces all your digital devices. (“Why have a TV when the glasses can display whatever you want wherever you’d like?” Abrash asks me rhetorically in an email.)
By everyone’s estimate, that’s a long, long way off. “I knew if we were going to build this out as a long-term platform, that’s a 10- or 15-year commitment,” Zuckerberg says. But he’s talking about his thinking when he first met the Oculus folks. In fact, it’s probably another decade until we’re closer to something like Abrash’s goal—and the problems that need to be solved between now and then are some of the hardest ones yet. It’s not just the computer vision and general compute power capable of delivering lifelike AR effects on the fly, but how to shrink it down into a not-totally-unattractive pair of glasses. Which then has enough battery power to last all day. And doesn’t generate ridiculous heat. Oh, and you have to be able to manipulate all the virtual objects and information that are getting overlaid on your real-world surroundings without lugging around a game controller.
Many of those things might be possible—"We've got 'proof of experience' for a lot of things," Schroepfer says—but packaging it all down is where the work really is. Which makes the path from now to then somewhat of a shifting course. “I have a sense of what it's going to look like 10 years from now,” says Bosworth. “I have less of a sense of what's going to happen in those 10 years.” As evidence, he mentions a breakthrough one of his team had just two weeks ago. They’d spent a year trying to get a type of sensor package into a VR headset, only to realize that from a thermal and compute perspective it was too expensive—but then discovered that an alternative technology that they’d dismissed had had huge gains, so it became the front-runner. “The sequencing is going to vary based on trade-offs between form factor, cost, weight, and functionality, and those things all are very zero-sum today,” he says. “It's easy to keep your North Star. It's the middle parts that change the most.”
Two weeks ago, Facebook Reality Labs held a media briefing to show off its North Star. You’ve likely read the stories by now, but if not, the magic word is “wristband.” Specifically, it’s an electromyography (EMG) neural interface wrist device, meaning it translates the electrical signals your muscles make as you move. The hope is that it unlocks the ability to manipulate the interfaces of your decade-hence AR world with tiny movements of your fingers—or none at all. The FRL briefing also included footage of an employee playing a simple video game without moving his hands; the EMG device read the nearly imperceptible signals his brain sent when he thought about pressing the spacebar. (Before you ask: Yes, Mark Zuckerberg has tried it. “I talk to the people on the Labs team every week,” he says. “They send me pelican cases of different gear—I’m sitting in my office right now, and I have two on the floor next to me, and one has the wrist device.”)
That may not be invasive technology in the traditional sense—again, let’s just leave that whole brain-implant thing alone—but it’s yet another reminder that AR and VR’s power depends on data. Lots and lots of data. Where you’re looking, how you’re looking at it, what your face and others’ faces are doing. In VR, that's a fount of psychographic information that has in the past proven very attractive to companies like Cambridge Analytica. And when you can identify people by their movement patterns alone, anonymity dies.
In AR, the proposition gets even more fraught. When you leave a party or a store, you're likely to forget many more details than you remember; your glasses are picking up everything you are, and quite possibly much more. The result, Katitza Rodriguez and Kurt Opsahl of the Electronic Frontier Foundation wrote last year, can all too easily become a "global panopticon society of constant surveillance in public or semi-public spaces." And when the company that’s building those systems is the same company that hasn’t exactly inspired trust in the past, and tech-ethics bugaboos like facial recognition are still on the table, that’s all the more reason to cast a skeptic’s eye at the future.
Which is exactly why Facebook is showing off its research and trying to engage with those bugaboos many years before they ever make it into a product. Schroepfer points to FRL’s “responsible innovation principles,” the foremost of which is: Never surprise people. “The bar we’re being held to is very high,” he says. “That means we have to do an exceptional job in the details of how these products will actually work, and how people understand them. The reason the Portal took us so long is because the pose-tracking algorithm had to run locally on the device—because then we don’t have to explain that we’re processing the video on the server for pose detection, but not other stuff. What I’ve learned over the past five years is that it’s really painful to have a problem figured out after you’ve launched the product.”
Perhaps tougher than that is the narrative Facebook has found itself battling: While there may be good people doing good work trying to solve problems, the company, at its core, has lost its way. If all those people are going to realize this vision, this thing that led Mark Zuckerberg to Atman Binstock's demo room all those years ago, they're going to need to do it in a way that makes people trust again. And that's its own brand of work. “There's no magic bullet,” says Bosworth. “Trust is not a thing that you swoop in and solve with a great speech or a great product spec. It's a thing that you solve by setting expectations consistently and meeting and exceeding those expectations consistently over time. There's no shortcut—and I'm not looking for one.”
“People understandably have a lot of concerns on the internet today about how our data is used,” Zuckerberg says. “But at the same time, Facebook has probably built some of the most advanced privacy tools and controls for people and infrastructure of any company that's out there. We're going to take this extremely seriously as we're in the foundational stages of building up these next platforms. But I'm pretty confident, given the experiences that we've had, that if we do this in an open way and show our work along the way, that the solutions will create a lot of value for people.”
Granted, we've heard this before. And when the Portal first came out in 2018, you'd have been hard pressed to find a reviewer who felt good about recommending it without qualification. But fast-forward to now, and the refrain goes something more like this: "I will happily throw all my principles out the window if Facebook will alleviate the torture of long-distance grandparent hell." Proof of experience is one thing, but quality of experience—when coupled with a good-faith effort to rehabilitate trust, that is—can go a long way. Even if the distance is just between two avatars.
- 📩 The latest on tech, science, and more: Get our newsletters!
- Audio pros “upmix” vintage tracks and give them new life
- Why you stay up late, even when you know you shouldn’t
- How sea chanteys made me love video games again
- Apple bent the rules for Russia. Other countries will take note
- Want carbon-neutral cows? Algae isn’t the answer
- 👁️ Explore AI like never before with our new database
- 🎮 WIRED Games: Get the latest tips, reviews, and more
- ✨ Optimize your home life with our Gear team’s best picks, from robot vacuums to affordable mattresses to smart speakers
