Marie Boran asks which bit of “human in the loop” is so hard for companies to grasp.
The most viral response to last week’s PocketOS database disaster was not from a technology commentator or an AI safety researcher. It was a single line posted on X: “This post rocks because it’s both a scathing indictment of AI and also 100% this guy’s fault”. To recap: PocketOS founder Jer Crane deployed AI coding agent Cursor, running on Anthropic’s Claude Opus 4.6, to work on a routine task. The agent hit a credential problem, decided unilaterally to fix it by deleting a storage volume on the company’s cloud infrastructure, found a fully permissioned API token sitting in an unrelated file, used it, and wiped out the production database and all its backups in nine seconds. Three months of customer data for car rental companies, reservations and customer records, gone.
The problem is that vibe coding, by its own definition, means accepting AI-generated actions without having to review or even understand them. This is grand for a little weekend project or a throwaway prototype. It’s a different story when the agent has access to live business systems and a fully permissioned API token with, as Crane himself wrote, “blanket authority” that he had created for an unrelated purpose and, by his own account, never understood the full permissions of.
But an agent with broad, unsupervised access to live systems and a skeleton key in its pocket was not an accident waiting to happen, it was one that was set up by someone who thought they could vibe code their way to business success. The confession the AI produced was fluent, contrite, and entirely beside the point. The more useful confession would have come earlier, from the human in the room.
Excerpt for editorial reference. Full article and analysis at TechCentral.ie.