Why is Apple developing code for AR glasses?

Code that suggests Apple is developing sophisticated support for Augmented Reality (AR) glasses has been found inside iOS 13, a report claims.

When development stops, production begins

Apple says it is “high” on AR. Company boss Tim Cook has talked about it as a “profound” technology. “I don’t think there is any sector or industry that will be untouched by AR,” he also said.

Now it looks like one of Apple’s much speculated upon plans for the technology may be entering prime time. MacRumors claims an “internal build” of iOS13 it has tested contains:

• An app to switch into head mounted viewing mode.
• A document that may relate to stereo audio feedback in a headset.
• Code including “Starboard mode,” “views,” “scenes,” “ARStarBoardViewController,” and “ARStarBoardSceneManager”.
• Code names including Starboard and Garta.

This follows the revelation that Apple has put a new Matrix co-processor inside the forthcoming A13 chip that will handle AR and other maths-heavy computational tasks.

This will also work in conjunction with the depth of view capability of the upcoming triple-lens iPhone cameras.

Innovation takes time

Apple has been developing AR glasses for years. But a Digitimes report in 2019 claimed it had “suspended the development of head-mounted display (HMD) AR/VR headsets, and already disbanded its team.”

No one really accepted this claim then and it makes even less sense now, given the claimed existence of code to support such devices inside iOS 13.

Indeed, it now looks as if hardware development has reached a point at which software development is taking precedence.

Apple is also working to develop usage cases for such products.

Think about the significance of its recently announced [AR]T Walk sessions in San Francisco, New York, London, Paris, Hong Kong and Tokyo.

Or recent claims its purported Tile competing Find My tag will show people their lost objects using AR.

These are all signals that suggest Apple is working hard to create the perfect triumvirate of hardware, software and need.

But I’m not expecting these things yet

Apple will host a special event on September 10 at which it is expected to refresh its iPhones and Apple Watch ranges, with a second event (probably around October) anticipated during which the company may share new iPads, Macs and services.

These two events seemingly leave little space for any other news, making it less likely the company will surprise us with any additional products.

Or does it?

After all, Apple’s September 10 invitation promises “By innovation only” and its [AR]T Walk seems to provide a good opportunity to evangelize such devices.

But the lack of applications for these devices will be the bugbear for any launch.

Apple has certainly created an ecosystem for AR development.

ARKit is evolving rapidly and is created in partnership with AR industry leading lights.

What isn’t clear is how easily those ARKit experiences will migrate into eyewear.

Nor do developers know what they need to know in order to build those experiences effectively – what are the limitations of use? What kind of imaging assets are required? What might the refresh rate of the eye displays be, and how do developers avoid motion sickness? What happens to users with poor vision?

We also don’t yet know what the concepts and usage cases for such a product might be, though gaming, navigation, medical, retail, training and collaborative experiences seem plausible.

Apple needs time to make this happen

The bottom line?

Apple is going to need time to introduce this new product family.

That’s not at all unusual.

Apple is really good at getting people interested in a new product, delaying its release and finding partners to help bring the solution to market. It proves this at every WWDC event it holds.

Why would this new product (and new paradigm) be different?

We don’t know if any of this is genuine, of course.

Everything here is based on rumor and speculation.

But if Apple does want to nurture iPhone and services sales and also introduce a brand new computing platform, then we can already guess at a six-point plan it might use to do so:

• Get people speculating for a while.
• Then talk about the new solution(s).
• Give people limited chances to experience them.
• Work with others to build showcase apps and development tools.
• Release those tools in time for apps to be made available.
• Ship

I’m not anticipating any news on September 10 (though it isn’t impossible), but I do think that whatever happens, people should pay attention to what Apple may or may not say around AR.

It may well also be of interest to take a look at what games it introduces in Apple Arcade.

Please follow me on Twitter, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.