AI Agent Security Risks: 5 Alarming Incidents Law Firms Can’t Afford to Ignore
September 30, 2026
We’ve written a lot about AI risk this year — governance controls, data loss prevention, and the whole vetting-your-vendor conversation. But something shifted this week, watching a handful of stories break within a few days of each other. They weren’t about AI giving a bad answer or hallucinating a case citation, which is the risk most of the legal world has spent 2026 talking about. They were about AI agents — the kind that don’t just respond to a prompt, they go do something — doing exactly what they were told, and that being the problem.
That distinction matters more than it sounds like it should. Let me walk through what actually happened.
When "Doing What You're Told" Is the Vulnerability
On September 28, a story broke that I haven’t been able to stop thinking about. Someone asked Meta’s AI assistant to export a file. A simple request. What came back wasn’t the file — it was the entire Linux filesystem. SSH keys, internal documentation, all of it, handed over because the system did precisely what it was asked to do without any judgment about scope.
There was no hack here. No exploit, no malicious actor. Just an AI agent taking an instruction literally, with none of the “wait, should I actually have access to give out this much” instinct a human employee would have. I think that’s the part general counsel and IT leadership need to sit with — we’ve spent years training people not to overshare. We haven’t spent nearly enough time training, or constraining, the systems now doing work alongside them.
The Same Week, a Second Warning Sign
Developers are increasingly handing credentials — AWS keys, specifically — to AI coding agents that can’t reliably tell a development environment from a production one. If an agent session gets compromised or is simply over-permissioned from the start (which, in my experience, is the default state of most AI tool rollouts), the blast radius isn’t limited to what a careless developer might touch. It’s whatever that agent can reach.
For a law firm, this might not seem directly relevant if you’re not running your own dev shop. But plenty of firms now use AI-assisted platforms built by third-party legal tech vendors, and those platforms are increasingly agentic under the hood. The question worth asking your vendors isn’t just “is our data encrypted” anymore. It’s “what can your AI agents actually touch, and who decided that.”
AI Is Now Writing the Malware Too
Also reported this week: attackers are using AI chatbots to generate functional banking malware and phishing kits, at a volume and sophistication that’s climbing month over month. This one isn’t new exactly — we’ve flagged AI-accelerated threats before — but the pace is the story. Every week the tooling gets a little better at writing something convincing, and every week the population of people capable of deploying it gets a little bigger.
For firms handling sensitive client financial data — and honestly, what firm isn’t — this is one more reason the “we’ll deal with AI policy eventually” approach doesn’t hold up anymore.
The Legal Tech Industry Is Already Reacting
Here’s what I’d call the more encouraging thread. Mitratech, a major legal AI platform provider, just acquired a company called BotDojo specifically to build out “agent orchestration” essentially, governance and control layers for AI agents operating inside legal workflows. When the vendors building this technology start spending real money on governance tooling rather than just capability, that tells you something. The industry itself is quietly admitting that unmanaged agents are a problem worth solving, not a feature to ship faster.
I’d also point to a recent survey of 557 arbitration professionals, which found real, unresolved disagreement about legal AI adoption — support varies a lot depending on the specific use case. I don’t think that’s a bad sign. It tells me the profession hasn’t rushed to a consensus it hasn’t earned yet, and that skepticism is doing useful work.
What I'd Actually Do About This
None of this means law firms should retreat from AI. I don’t believe that, and I don’t think it’s realistic anyway. But I think it does mean the conversation needs to move up a level. We’ve spent this year talking about which AI tools to allow and how to keep data from leaking out through prompts. That’s still necessary — local DLP for tools like Claude and Copilot isn’t optional anymore. But the Meta filesystem story is a different category of problem: it’s not about what a person might accidentally paste into a chat window. It’s about what an agent, acting on its own initiative within the permissions it’s been given, is capable of doing without anyone in the room to say “hang on.”
That’s exactly the thinking behind the governance controls we outlined for Grok-style bots handling deal data — the goal isn’t to slow AI down, it’s to make sure every agent operating on your firm’s behalf has boundaries that were actually decided on, not defaulted into. And it’s worth revisiting our earlier look at Google’s legal AI pitch with this lens too — the sales pitch rarely leads with “here’s exactly what this agent can access and why.”
If your firm has rolled out any AI tool with agentic capabilities — meaning it can take actions, not just generate text — this is a good week to ask exactly what permissions that tool actually holds. Not what the vendor says it needs. What it can actually reach, right now, in your environment.
That’s a conversation worth having before an agent decides to answer a simple request a little too literally.
Recent Posts
Have Any Question?
Call or email Cocha. We can help with your cybersecurity needs!
- (281) 607-0616
- info@cochatechnology.com
About the Author:
Steve Combs
Co-Founder & Managing Director, Cocha Technology
Steven is a fractional CIO/CISO with 30+ years of enterprise IT and security leadership. He has built AI governance frameworks for organizations with 1,700+ users, led enterprise Microsoft Copilot deployments, and conducted security assessments across law firms, energy companies, financial institutions, and PE-backed manufacturers.