Apple has major plans for augmented reality (AR). It started with ARKit, a new augmented reality platform, launched at its annual Worldwide Developer Conference this year, and then built into iOS 11. That means you can try out AR apps for yourself right now if you own an iPhone X or any other iOS 11-compatible device, but Apple intends to do something far greater with the emerging technology than simply facilitating a few apps.
“AR can be really great,” Apple CEO Tim Cook said when asked about Pokémon Go during an earnings call in 2016. “We have been and continue to invest a lot in this. We are high on AR for the long run, we think there’s great things for customers and a great commercial opportunity.”
Apple AR glasses, or an augmented reality headset, may be part of this, according to a growing number of rumors. Leaked by three alleged Foxconn employees, it’s apparently known internally as Project Mirrorshades; but this is unlikely to be its release name. Exactly what form it will take is also unknown. It may be an AR headset similar to current VR headsets, a pair of Google Glass-like glasses, more than one product, or something else entirely.
Here’s everything we think we know about it, and Apple’s AR masterplan, so far.
Apple is using its sizable bank account to acquire startups working in augmented reality. It’s through these acquisitions it will shape and perfect any future products. On November 21, it was reported Apple acquired Vrvana, a startup that hit the headlines a few years back with its Totem headset. Apple paid around $30 million for the Montreal-based Vrvana, two sources claiming to have knowledge of the deal told TechCrunch.
When the Totem landed on Kickstarter in 2014, it caught the attention of commentators for a number of reasons. The headset had onboard “pass-through” cameras to combine AR effects with the world around the wearer, head tracking, and the ability to show full color animations, separating it from other headsets. The effect combined both VR- and AR-style experiences. Despite its promise, the Totem headset has never been released.
Apple hasn’t confirmed the acquisition, but nor has it denied it, and some former Vrvana employees are already working for the Cupertino-based company. The startup’s website is still online, though its social media updates ceased in the summer, suggesting a timeframe for when the reported acquisition may have taken place.
In 2016, Vrvana was known to be “planning a business-to-business model wherein it will sell the Totem hardware directly to enterprise customers in small volumes,” according to Tom’s Hardware. Whether Apple sees consumer potential for Vrvana’s technology, or intends to also target businesses with any future augmented reality products and services, remains to be seen.
Over the past three years, Apple has acquired companies with an expertise in AR, 3D mapping, and computer vision — including the 2013 purchase of PrimeSense, which pioneered the depth-tracking technology behind the Kinect, Metaio and Flyby. In September 2016, Apple hired two VR veterans from Oculus VR and Magic Leap, companies with a pedigree in augmented reality and virtual reality technologies.
When is it coming and how much will it cost?
Apple’s augmented reality device is still in the very early stages of development. We don’t know what form it will take, or if there will be more than one device. Anyone getting excited is in for a long wait, as Apple apparently hopes to launch its AR device in 2019, with shipping to start in 2020, according to sources speaking to Bloomberg. Apple partner Quanta has said it’s working on an AR project for a major client that’s due in 2019. Analysts speculate the client is Apple.
Apple products are rarely cheap. If the headset described in the rumor originating from Foxconn is real, it’s claimed it will cost around $600.
Will Apple’s AR headset work with the iPhone?
Will Apple’s AR headset need the iPhone to work, like Google’s Daydream View requires an Android phone? Apple may approach its AR device like the Apple Watch, with a dedicated processor and software, that can link to the iPhone, but doesn’t require it to operate.
In November 2016, Bloomberg reported that Apple was working on a pair of glasses that blended an augmented reality display with a standard pair of glasses. It was said to have begun talks with potential suppliers for components like near-eye displays, with the goal of developing a pair of glasses that could connect wirelessly to an iPhone and overlay information on the wearer’s field of vision.
What will Apple’s AR headset be like?
Apple makes beautiful hardware, so we shouldn’t expect any augmented reality product to be an ugly mess of parts and cables — but what will it look like? Rumors are giving us a few clues. According to the leak from the alleged Foxconn employees, at least one pair of Apple’s prototype AR glasses have used near-eye displays from technology company Kopin, mounted in plastic frames that can hold “polarized or prescription [lenses],” potentially from lens experts Zeiss. Tiny projectors beam images onto a 428 × 240 glass prism in front of the lenses, and DC motors produce sound by vibrating through the small bones of the user’s ears, like Google Glass.
Controlling Apple’s AR glasses may also be similar to Google Glass. It’s said at least one prototype has a touch-sensitive strip on the arm to answer calls and control the volume, along with gestures to navigate through menus. A combination magnetometer and light sensor detects when a wearer shakes his or her head, and responds accordingly — a Tinder app user, for example, could shake their head for no, or nod for yes.
This is Apple we’re talking about, so any final AR device will need to be suitable for everyone. Rumors talk about two sizes for the headset or glasses, and at least three colors, with the potential of different colors being added seasonally. The shape is rumored to be based on a classic, round “P3” frame. Design may change before the headset or glasses go on sale though, so don’t treat this as final yet.
Corning, best known for its Gorilla Glass product used to cover smartphone screens, is said to have been working on specialized glass — specifically for AR glasses and headsets, Variety reports. Corning has been an Apple supplier since the first iPhone in 2007, and in 2017 Apple invested $200 million in the firm, specifically for research and development.
Another long time Apple partner, Quanta, which among other things builds the Apple Watch Series 3, has said it’s working on a AR project for a client suspected to be Apple. The company described the product as “headset-like gadget with a fully transparent lens that allows users to see through and interact with the environment.”
A recent report in Bloomberg states that Apple will develop its own chip for the headset instead of using an existing chip. The same report says the company will create a new operating system for the device, much like it did for the Apple Watch.
We’ll keep you updated with all the Apple AR headset project rumors right here.
Updated: Apple acquires Vrvana, an interesting augmented reality startup.