visionOS 27: Spatial Accessories framework for real-time physical object tracking

Apple introduces a dedicated spatial accessories framework that registers physical hardware system-wide on Vision Pro, letting any app track them without per-title integration.

A system-wide registry for physical accessories

With visionOS 27, Apple formalises the concept of a "spatial accessory": a physical piece of hardware — a tool, a medical device, an industrial controller — that is registered system-wide on Vision Pro via the new Spatial Accessories framework. The practical consequence is that any installed app can access the accessory without per-title integration; registration happens once.

API and development flow

Developers discover accessories via `GCSpatialAccessory` and manage tracking via `AccessoryTrackingProvider`. Accessories hot-swap without restarting the ARKit session, which matters in industrial and training contexts where tools change frequently mid-session. Reference kits are already available commercially: DFRobot with the seeMote Cap and MikroE with the Spatial Anchor R1.

Third-party engine integration

Unity PolySpatial, Unreal Engine, and Godot all receive spatial controller plug-ins in visionOS 27, meaning teams on established engines get accessory support without writing native Swift. This lowers the entry barrier for companies already building simulators on mature engines.

The emblematic use case cited by Apple at WWDC 2026 is a maintenance simulator tracking a physical torque wrench, overlaying digital step-by-step instructions anchored to the tool's real-time orientation.

← Back to home