The Abstraction Bill Arrives

The industry’s working assumption has been wonderfully simple: transformer scaling is a law of nature, GPUs are a utility, and an agent that can emit code is approximately an engineer. This has the aesthetic of a financial model built in a Jupyter notebook: elegant until one asks where the electricity, context window, or incident report goes.

That consensus is now under pressure from two directions. First, the transformer is expensive precisely where companies want models to become most useful: huge contexts, long-running agents, sprawling repositories, and the accumulated sediment of their own output. MIT Technology Review’s survey of the post-transformer hopefuls is therefore less a parade of clever architectures than an invoice dispute. Sparse attention, rolling retained state, liquid networks, diffusion generation, and state-space models are all attempts to stop paying dense-attention prices for every token’s social life.

Second, even when the model can produce code quickly, organizations still have to operate what it produced. Cal Newport’s account of a senior engineer’s retreat from fully agentic coding is anecdotal, not a controlled trial, but it captures the real operational asymmetry: writing code is only one small component of owning it. The feature arrives in hours; the responsibility arrives indefinitely. Silicon Valley has rediscovered that compilation is not an acceptance test, which is rather like rediscovering that ships need hulls after getting very good at sails.

The architecture challengers are not necessarily promising a new intellectual dawn. Most are promising something more immediately valuable: less waste. A 10,000-word document can force dense attention into roughly 50 million multiplications, according to the MIT Technology Review report. That becomes awkward when the industry wants an agent to ingest a codebase, its tickets, its tool outputs, and perhaps the emotional residue of every failed shell command.

The proposed evasions differ. Subquadratic wants sparse attention that selects relevant tokens; Manifest AI retains a rolling, lossy summary; Liquid AI mixes transformers with liquid neural networks for smaller models; Inception’s diffusion approach generates blocks rather than strictly marching token by token; Pathway replaces attention with state-space machinery. Liquid says some of its models run on a $50 Raspberry Pi; Inception says its Mercury 2 matches some 2023 GPT-4 models while running faster. These are interesting claims, but claims nonetheless: the report does not supply independent replications, comparable hardware settings, or complete benchmark methodology. The industry has learned to say “frontier”; it has not always learned to say “error bars.”

There is a useful inversion here. The question is not whether attention was a historical mistake—it plainly powered the current era—but whether it is still the right universal tax collector. As Inception cofounder Stefano Ermon puts it, “Ultimately, the currency is going to be intelligence per dollar.” That is a more disciplined metric than parameter count, though it should also include latency, failure modes, deployment friction, and the cost of having a human reconstruct what the machine thought it was doing. Intelligence per dollar is wonderful. Correctness per on-call hour would be better.

Agentic coding’s strongest use case may be neither replacement nor autonomy, but nuisance removal: tests, disposable scripts, routine scaffolding, local transformations with easy-to-check outputs. That is not a failure of ambition; it is what mature tooling looks like. Compilers did not abolish programming because they emitted assembly. They made it possible to spend human attention on more expensive mistakes.

The warning in Newport’s piece is that generated code can look plausible while remaining difficult to review, especially in an unfamiliar system. The engineer at its center reported two product crashes tied to AI-generated features and then returned mainly to hand coding. Other commenters report successful greenfield deployments, so this is not a universal indictment. It is a reminder that greenfield demos and old, business-critical systems live in different climates. The former rewards velocity; the latter contains undocumented behavior, accumulated contracts, and an API consumer somewhere who treats every accident as constitutional law.

The uncomfortable organizational issue is training. If junior engineers become prompt operators before they acquire an internal model of systems, then firms may save time now by borrowing against future debugging capacity. A harness can generate a patch; it cannot automatically create the person who knows when the patch has broken an invariant two services away. The sensible policy is almost boring: constrain autonomy by reversibility, observability, blast radius, and reviewability. In other words, make the agent earn permissions rather than granting it a terminal because its demo had soothing animations.

The same abstraction habit appears in model reasoning. A research site alleges that encrypted reasoning blocks returned by proprietary APIs can be replayed into weaker related models and jailbroken into plaintext, potentially exposing hidden reasoning and sensitive artifacts from public trajectories. If accurate, that is not merely a chain-of-thought policy problem. It is an architecture problem: data placed in a portable client-visible object has acquired a life beyond the product manager’s preferred narrative. Read the claims, and their important limits, in the authors’ report.

The authors say they reconstructed 315,320 reasoning blocks from 6,708 publicly available agent trajectories and found hundreds of distinct privacy artifacts, including credentials and personal information. These are preprint-era claims, presented with selected examples rather than independent validation; prevalence, reproducibility, provenance, and provider scope remain unresolved. But the threat model is intelligible even before every chart is litigated: encrypted is not synonymous with safely non-exportable when the recipient can replay the object across contexts. The industry has spent years treating reasoning visibility as a UX setting. It may need to treat it as data egress.

Then there is the web that supplies all this machinery with context. The Walrus argues that AI summaries, link rot, disappearing archives, and platform incentives are making source material harder to find even as systems speak more fluently about it. Its case for public-interest search and preservation is an argument, not a completed policy blueprint. Still, the connection is hard to miss: models are being asked to remember more while the public web is being allowed to remember less. A system that summarizes everything while preserving little is not a library. It is a very confident witness with no citations.

There is, amid the grand architectural pronouncements, a pleasingly practical counterexample. Cua/Lume reports that a process-scoped Metal capability shim in an Apple Silicon macOS VM let llama.cpp select newer GPU paths, producing large speedups in its tests—without physical GPU passthrough. The virtual GPU remained virtual; selected capability answers changed for an injected process. “Still a VM,” as the project’s write-up puts it.

On one M1 Ultra, the authors report that unlocked guests recovered much of bare-metal throughput for TinyLlama and a Gemma 4 12B workload, while MLX-LM changed essentially not at all. That contrast is the whole story, and more useful than the headline multiplier. The bottleneck was not simply “virtualization”; it was application-specific path selection gated by conservative capability reporting. Change the reported affordance, and llama.cpp takes a different road. MLX takes its own road, encounters an unavailable residency set when a newer family is advertised, and declines to become a marketing chart.

This is the issue’s less glamorous lesson. Progress often comes not from an AI moonshot but from identifying which layer is lying—politely, defensibly, and perhaps for compatibility reasons—about what the layer beneath it can do. Transformers may become cheaper, coding agents may become safer, and reasoning traces may become better contained. But the durable engineering work remains the same: inspect the boundary, measure the workload, preserve the evidence, and distrust any abstraction that claims to have made the underlying constraint disappear. It has usually just moved it to your pager.