AI Descriptions for Blind Users
Apple previewed the integration of AI-generated descriptions directly into VoiceOver and the Magnifier app. The stated goal is to reduce reliance on manually entered text descriptions from developers, making a broader catalog of visual content — including photographs, generic UI, and physical environments captured by the camera — more accessible.
Automatic Captions and a Sign Language API
Accessibility Reader is being updated to handle complex document layouts, while on-device video caption processing aims to improve privacy compared to solutions requiring a remote connection. On the visionOS side, Apple is adding Power Wheelchair Control via Vision Pro's eye-tracking, Vehicle Motion Cues for users in moving vehicles, and face gesture support for system actions. The new FaceTime API for sign language interpretation is a significant development for the deaf community: it will allow third-party apps to integrate real-time interpreting services into video calls. As reported by MacRumors, Apple officially communicated these features before WWDC, consistent with the practice established in recent years.