Screenshot as a starting point for actions
On macOS Golden Gate, the screenshot tool no longer merely captures a screen area: thanks to Visual Intelligence integrated with Siri AI, the image content becomes actionable material. The example shown during the keynote is emblematic: a user photographs a festival schedule, selects concerts of interest, and Siri AI adds them directly to the calendar. Engadget reported this demo as one of the most concrete use cases of the day.
The difference from iOS
On iPhone, Visual Intelligence operates primarily through the live camera or on already-taken photos. On Mac, the natural input vector is the screenshot, which allows working on any content displayed on screen — web pages, PDFs, presentations, third-party applications. This significantly broadens the scope: there is no need for the site or app to be native Apple; the content just needs to be visible.
Implications for productivity
The feature addresses a frequent problem: reading information on screen and then having to manually transcribe it into another app. With Siri AI mediating between visual content and system apps, the workflow becomes linear. It remains to be seen how this will work in non-optimal scenarios — text on complex backgrounds, languages other than English, non-standard layouts — but as a demonstration of vertical integration between computer vision and productivity apps, the principle is sound.