news.volyx.in

Claude Cowork exfiltrates files (promptarmor.com)

870 points by takira · 189 days ago · 399 comments on HN

Article summary

Claude Cowork, a general-purpose AI agent, is vulnerable to file exfiltration attacks via indirect prompt injection due to unresolved isolation flaws in its code execution environment. This vulnerability allows attackers to upload files to their Anthropic account, potentially exposing sensitive user data. The attack exploits the allowlisting of the Anthropic API, which is trusted by Claude's VM environment. The vulnerability was previously identified in Claude.ai chat and was acknowledged but not remediated by Anthropic.

Main themes

  • AI security vulnerabilities
  • Prompt injection attacks
  • Data exfiltration
  • Responsible AI development
  • User risk awareness

What commenters say

  • Telling users to 'watch out for prompt injections' is insufficient and negligent, as most users are not aware of the risks or how to mitigate them.
  • The comparison between SQL injection and prompt injection is inaccurate, and a more apt comparison would be to phishing attacks, as there is no clear separation between control and data channels in LLMs.
  • The onus of security should not be placed solely on users, and developers should prioritize building secure products rather than relying on user vigilance.
  • The vulnerability in Claude Cowork is a significant concern, and users should exercise caution when configuring connectors and interacting with the AI agent.
  • Some argue that the attack requires a series of unlikely events and user mistakes, and that the risk is overstated.
  • Others believe that the vulnerability is a symptom of a larger issue with the development and deployment of AI models, and that more emphasis should be placed on security and responsible development practices.
  • There is a need for better education and awareness about AI security risks and how to mitigate them, particularly for non-technical users.
  • The use of terms like 'prompt injection' may be misleading, and a more accurate comparison to phishing attacks could help to better communicate the risks to users.