The Autonomy Tax Arrives

The current AI sales pitch is that software should stop waiting for permission. Agents browse, download, decode, write code, run code, and—if the brochure is especially breathless—reason their way through the whole unpleasant business. This is progress, in the same sense that removing circuit breakers is progress for electricity: the power flows more freely until it finds something expensive.

A researcher’s reported lab attack on Claude Code Opus 5 Auto Mode illustrates the problem with uncommon clarity. A request to summarize a site reportedly drove the agent from a failed web fetch to curl, through a redirect into a ZIP archive, and into writing its own Python decoder after declining to run a supplied binary. The reported compromise hinged on Python module shadowing: running the new decoder from the attacker-controlled extracted directory allowed a malicious local struct.py to be imported. In some tests, the author reports a controlled command-and-control callback and Calculator opening. The Windows desktop application has become security research’s traditional ceremonial goat.

The important point is not that one post, with small samples and no independent replication in the supplied evidence, has settled the safety of an entire product. It plainly has not. Nor do its reported 60–80% results measure the same thing as the separate 72-scenario evaluation said to have produced 0.00% success. The point is more mundane and therefore more useful: a fixed prompt-injection benchmark measures performance on its fixtures. An attacker gets to alter the fixture, the protocol, the file layout, the fallback path, and eventually the operating system’s import rules. “WebFetch got a 415. Let me try directly.” is not villainy; it is initiative. Initiative is precisely what becomes dangerous when the machine is allowed to improvise across trust boundaries.

For teams deploying coding agents, the lesson is not to compose a more lyrical policy file. Treat the agent as a fast, credulous contractor with shell access and a talent for interpreting obstacles as tasks. Put unattended work in a container, VM, or OS sandbox; restrict egress; expose neither home directories nor credentials; and monitor it. Auto Mode may be a convenience feature. Your cloud keys should decline to become its capstone project.

While agent vendors attempt to teach software the confidence of a junior engineer at 2 a.m., diffusion language models propose a different mechanical instinct: revision. Rather than emitting tokens strictly left to right, they begin with a whole sequence guess and iteratively refine it. The Kuleshov Group’s technical survey describes the attraction cleanly: ordinary autoregressive generation cannot correct an emitted token, requires roughly one generation step per token, and looks only backward through causal attention. Diffusion offers parallelism and a chance to reconsider.

That does not mean the transformer has discovered remorse. Standard masked diffusion has practical defects of its own: fixed-length output, no native iterative refinement, and weak speed without further work. Hence the rapidly growing drawer of compensatory machinery: block diffusion for variable-length sequences, encoder–decoder designs, remasking, distillation, guidance, and reinforcement-learning post-training. In other words, the architecture has entered its enterprise phase, where a simple idea acquires enough adapters to qualify for a platform roadmap.

Still, some of these mechanisms matter. Remasking can revisit tokens that initially looked plausible and correct them; block generation can reuse cached keys and values; uniform-state diffusion allows revisions without relying on a special mask token. The survey says diffusion may achieve 5–10 times autoregressive generation speed, while noting that simultaneous token generation can create inconsistencies. Named efforts—Mercury, Gemma Diffusion, Nemotron Diffusion, and LLaDA—suggest this is no longer merely a whiteboard genre. Yet diffusion has not been scaled to the same parameter, compute, or data levels as leading autoregressive systems, and claimed throughput and quality figures vary by model and evaluation.

The deeper connection to Auto Mode is not that diffusion solves agent safety. It emphatically does not. But it makes explicit a property production systems need more of: the capacity to revise before acting. Generation can be parallel; authority should be staged. A model revising a bad verb is useful. An agent revising its plan after it has already launched an attacker-controlled child process is just post-incident analysis with better typography.

The labor-market version of this story arrives with less spectacle and more low-grade dread. In a first-person account of LLM-assisted development, Paolo Galeone describes the increasingly familiar loop: prompt, inspect, correct, repeat. He does not deny that models can accelerate implementation or improve code with sufficient guardrails, rules, skills, and design documentation. His concern is that the work no longer teaches in the same way. “The point was never just going from A to B.”

This is easy to dismiss as artisanal nostalgia until one remembers what senior engineering judgment actually is. It is not a mystical property acquired by owning the right mechanical keyboard. It is compressed exposure to failure: knowing where a dependency will quietly misbehave, why an apparently innocent import path matters, what an operational shortcut costs six months later, and which metric has been optimized into irrelevance. Galeone calls this savvy—learning by doing, making mistakes, learning from them, and doing it again. The security story above supplies an unusually literal example of why the distinction matters.

Corporate technology has chosen an awkward time to measure developers chiefly by visible output velocity. A serious productivity metric, as Galeone notes, would have to include technical debt, maintainability, and cost as well as speed. That is annoyingly difficult, which is why organizations prefer counting merged pull requests and calling the resulting archaeology “AI transformation.” If the model drafts the code and the human reviews it, the human may still catch errors—but catching an error is not identical to building the mental machinery that predicts one.

The answer is not performative abstinence. Galeone himself uses local models and found setting up Linux hardware for local inference enjoyable; tool fluency remains valuable. The organizational obligation is to preserve apprenticeship deliberately: require design ownership, make debugging and incident analysis legible work, rotate engineers through systems they must understand without autocomplete as a life-support machine, and distinguish generated artifacts from demonstrated competence. Otherwise the industry will spend years training models on accumulated expertise while quietly ceasing to reproduce the people who can audit it.

There is a domain where the automation argument should be handled with both urgency and restraint. Nature’s report on AI for zoonotic-disease surveillance describes models used to identify risky viruses, forecast disease risk, and monitor outbreaks—from livestock, poultry, and gorilla data around Uganda’s Bwindi Impenetrable National Park to global systems that sift news, health data, and travel information. Human–wildlife contact is increasing through agriculture, climate shifts, and land-use change; disease also moves through a world that has made distance mostly a pricing issue.

The promising use case is not an LLM declaring pathogen victory from a dashboard. It is earlier, better-targeted attention. Conservation Through Public Health combined testing data from cattle and poultry near Bwindi with gorilla-monitoring records to help train outbreak-warning models. BlueDot filters thousands of articles and official data in 65 languages; HealthMap has long filtered outbreak signals from news, public-health departments, and social media; BEACON uses an LLM to combine alerts into an operational dashboard. These are worthwhile attempts to find the smoke before the building committee convenes.

But data abundance is not preparedness, just as a benchmark score is not a sandbox. The article notes that such models are still limited and not widely used in infectious-disease epidemiology, while virus-discovery systems are used mostly for research. LucaProt’s scan of public datasets identified 161,979 RNA-virus species, including 70,458 not previously seen; few, if any, are likely to harm humans. The model can expand the map. It cannot tell a public-health system which uncertain signal deserves scarce sampling, trust, money, and intervention.

That is why the most revealing line in the report is Edward Holmes’s: “It’s a politics and people problem.” Local communities, field workers, laboratories, public-health institutions, and practical prevention are not the legacy layer beneath the intelligent system. They are the system. The cited estimate that One Health-guided prevention could cost up to US$11.5 billion annually—about one-third of pandemic management—makes the usual point with cruel efficiency: we can afford foresight only until it looks like a budget line.

AI is getting better at producing, revising, filtering, and predicting. Fine. But the governing design principle across coding agents, developer workflows, and outbreak surveillance is almost offensively old-fashioned: keep authority close to accountability. Let models accelerate judgment where evidence can be checked; do not let their speed become a substitute for boundaries, practice, or public capacity. The machine may now write the first draft. Someone still has to own the consequences in production.