Concerns over artificial intelligence (AI) hacking targeting the financial sector are escalating, and tech giants have issued a warning that Agent AI security risks could turn virtual assistants into pathways for data leaks without any external hacking. Meta’s ‘Muse’ and similar AI agents handle reservations and payments on behalf of users, but researchers warn that even if users delete prompts, the information remembered by the AI can linger like a ‘ghost’ and be reused.
Google Research published a report on the 5th (local time) titled ‘Open Problems and New Directions in Agent Privacy and Security: A Contextual Perspective.’ Prominent privacy scholar Helen Nissenbaum from Cornell University and security expert Dawn Song from UC Berkeley co-authored the report alongside Google researchers.
The researchers focused on the phase after users share sensitive information with agent AI for assistance. They highlighted vulnerabilities in how AI assistants read emails, schedules, and work materials while retaining necessary details. Key risks identified include sharing excessive sensitive information, exposing privacy through the combination of multiple data points, and personal data persisting even after deletion.
The research team illustrated how AI attempting to help users might inadvertently expose privacy using restaurant and pharmacy examples. Assuming a scenario where an AI assistant manages schedules and conveys food restrictions to a restaurant and medication schedules to a pharmacy, a third party viewing both sets of data together could deduce the user’s medical condition. Sensitive health information can thus be revealed even without explicitly transmitting the diagnosis.
Another issue raised is ‘ghost information’ that remains within the AI even after deletion. The research team pointed out that when an AI reads a user’s personal email and saves summaries to its long-term memory for future work, deleted original emails can still pose risks. If salaries or family details contained in those emails remain in the AI’s memory, they could potentially be transmitted externally at any time. The researchers described information persisting after the original is gone as ‘derived knowledge’ or a ‘ghost.’
To mitigate these risks, researchers suggest limiting the information and permissions granted to AI to the minimum necessary for the task. They recommended that developers design services to require user confirmation before transmitting sensitive data and ensure that deleted information is not reused through AI memory. Government and corporate roles were also emphasized in establishing information-sharing standards for agent AI and assigning accountability for data leaks to enterprises.
However, some argue that expecting flawless privacy protection while delegating tasks to AI is unrealistic. Lillian Tsai, a co-lead author of the report, shared the report on LinkedIn on the 5th, stating that this report is only the beginning and expressing hope for continued collaboration between industry and academia to address these challenges.
