OpenAI is experimenting with a more intimate form of context for its macOS Codex tool. A new feature called Chronicle allows the AI to "read" a user’s screen, transforming active visual data into a personalized understanding of workflows and habits. Currently in a research phase for ChatGPT Pro subscribers, the tool marks a shift from reactive prompting toward an ambient assistant that learns through observation.
The system functions by taking temporary screenshots, which are stored locally for six hours before being purged. From these images, Chronicle generates "memories"—Markdown files saved on the device that the AI references to refine its responses. While these files remain under the user’s control for editing or deletion, they represent a significant step toward an AI agent that understands not just the code it is asked to write, but the specific environment in which the user operates.
However, this level of integration requires deep system access, including Accessibility and Screen Recording permissions, which has prompted predictable security concerns. OpenAI has noted that both the temporary screenshots and the resulting Markdown files are stored unencrypted on the device. Furthermore, the feature is currently unavailable in the European Union, the United Kingdom, and Switzerland, likely due to the rigorous data privacy standards of those regions. Beyond privacy, the company warns of "prompt injection" risks, where malicious content captured from a screen could theoretically hijack the AI’s behavior.
With reporting from Canaltech.
Source · Canaltech
