Post

My thoughts on the current state of AI driven software development, why I'm no longer concerned about a runaway intelligence explosion, and what I think is a far worse outcome we could be walking into maevyn.codes/decoherence

I've been wondering about the impact of AI dev tools on standardization for a while. A lot of what you talk about echoes the tensions between different pace layers in an evolving system.

It's pretty normal for fast-moving unstable systems to be built atop a slow-changing, more stable layer. Examples abound in economics, biology, and of course software.

AI looks pretty normal through this lens, with the main concern being that AI vacuums up investment from all over the economy, starving other industries.

Which recasts your question of the decoherence trap in economic terms: If the least coherent businesses absorb (nearly) all economic power, what then?

Normally they would become inefficient and get outcompeted, but we're entering a new market in which investor story time is often a better business model than selling products and services, since investors have most of the money.

Oh, this is _such_ a good way of framing it. I'm planning on exploring the impacts of decoherence on markets and economics in the next post, though in a bit of a different direction (essentially, why does pricing break down as a signal in highly technical markets) but I'll need to think on this

Looking forward to it! Feel free to ping me when you post it. Price signals aren’t magic, they all come from people spending money, and I’ve been thinking about how much of the money is controlled by a few tech billionaires with founders’ shares…who all know each other and bandwagon together.

Price signals don’t come from people spending money. They come from coherence—people paying the same price. The “tech problem” for markets is not in tech itself, it’s in the ability to usefully price discriminate. That’s the *why* of surveillance capitalism from the capitalists’ pov.

If you want to exclude the price system from “systems that can be coherent”, I’ll find another word. But prices are not “just” data. They’re strategic variables. For the price system to work *for* society, it’s really important that goods that are close substitutes be priced similarly.

Well, the price/financial system is indeed decoherent, but the claim was that coherence is everyone deciding what price to pay. Decoherence is conflicting or redundant systems. Prices don’t really conflict, there’s just a spread. Coherence is unified systems, and prices _are_ coherent on their own.

The distinction matters because there are many systems that influence prices, and all of them can be decoherent to different degrees. The details matter here, because in some places price based markets may be fine, and in others they definitely are not.

Historically, prices were useful for understanding what needed to be built, and then coherence came from trying to be more efficient in building it to make more profit. The issue in software is that price incentives encourage decoherent system design in a number of ways.