All Posts
Why the Web Centralised: The Protocol Gaps That Platforms Filled

Why the Web Centralised: The Protocol Gaps That Platforms Filled

Series: Networking Foundations | Interlude, between Posts 5 and 6 We have spent five posts on the plumbing. Packets, layers, addresses, names. Every one of them arrived at the same truth: the internet was built for connection, not control. This interlude asks what that truth actually explains about the web you use every day. The answer is bigger than I expected when I started writing it.

Five posts in, this series has been circling one idea from five different angles.

ARPANET connected four trusted institutions and needed no security, because everyone on it was known. Packets travel with their addresses exposed, like postcards, because routing demands it. The OSI layers each trust the layer beneath them to have done its job honestly. IP addresses locate a machine but say nothing about who is operating it. And DNS, the internet's phone book, cannot verify that its own answers are genuine.

Every post lands in the same place: the foundational protocols were built for connection, not control. They answered "how do we make these machines talk to each other" brilliantly, and left almost everything else to someone else.

This interlude is about that "someone else."

Because here is the thing I want to argue, and I think the evidence for it is strong: the web did not centralise despite its open, decentralised design. It centralised because of it. Every gap the original protocols left open was eventually filled by a company. And every company that filled a gap became a chokepoint.

That is not a betrayal of the web's architecture. It is the market completing an unfinished protocol.


The Gaps Were Real, and They Were Deliberate

Let us be precise about what the early web actually shipped, because the story only works if we are honest about the technical facts.

Tim Berners-Lee's web gave us three things: a way to name resources (URLs), a way to request them (HTTP), and a way to structure them (HTML). That combination was extraordinarily powerful and extraordinarily minimal. It let any machine serve a document and any machine request one, with no permission needed from anybody. That openness is why it won.

But look at what it did not include.

It did not include a way to know who you were talking to. As we established in Post 4, IP addressing tells you the address of a machine, not the identity of the person, organisation, or thing responsible for that machine. The protocol stack was built to locate endpoints, not to authenticate the humans behind them. The internet was built without any native way of knowing who and what we are connecting to, and because of that, everyone offering a service has had to cobble together their own workaround. That patchwork is why the average business user today juggles something like 191 passwords. Every site had to invent its own identity system, because the web never provided one.

It did not include a way to pay for anything. This one is my favourite, because the evidence is sitting in the specification itself, in plain sight, and has been for over thirty years. HTTP status code 402 is called "Payment Required." It has been in the HTTP specification since the early 1990s. And its official definition has always been, essentially, reserved for future use. The people who designed the web knew a payment layer would be needed. They left a placeholder for it. And then it sat dormant for three decades, because the web was never given a native payment primitive, and the core architecture of the internet has no way to send money the way it transmits information.

Think about what that means. There is a status code in every HTTP implementation on earth, waiting for a payment system that the protocol never delivered.

alt HTTP 402: The Placeholder That Waited 30 Years
alt HTTP 402: The Placeholder That Waited 30 Years

It did not include a way to establish trust. HTTP will faithfully fetch you a document from a stranger's server. It has no opinion whatsoever about whether that stranger is honest, whether the goods they are selling exist, or whether they will take your money and vanish. Trust was simply not in scope.

It did not include a way to find anything. The web gave us links, but no directory, no index, no search. If you did not already know a URL, or know someone who did, the resource was effectively invisible to you.

And it did not include a way to store state. HTTP is stateless by design. Each request stands alone, knowing nothing of the last. Elegant, scalable, and completely incapable of remembering that you are logged in, what is in your basket, or who you are.

None of these were oversights by careless engineers. They were the deliberate minimalism that made the web deployable. A protocol that had tried to standardise identity, payment, trust, discovery, and state in 1991 would have been so complex, so contentious, and so slow to agree that it would very likely never have shipped at all. The web won because it was small enough to adopt without asking anyone's permission.

But an unfinished protocol does not stay unfinished. The gaps get filled. The only question is by whom.


Every Gap Became a Company

Here is the pattern, and once you see it, the entire shape of the modern web snaps into focus.

The identity gap became login providers. Because there was no protocol-level way to prove who you are, every website built its own account system. That was miserable for users and miserable for developers, so a solution emerged: let one big platform handle identity for everyone. "Sign in with Google." "Log in with Facebook." Facebook's identity system was explicitly marketed to app developers as a generalised web identity system, and it genuinely improved usability, but at the cost of ceding developer and consumer control to the platform. A missing protocol layer became a product, and the company that provided it now sits between you and thousands of services, watching you arrive at each one.

The payment gap became payment processors. HTTP 402 stayed dormant, so commerce needed rails the protocol did not offer. Card networks, then PayPal, then Stripe stepped into that void. They did it well. But it means that value on the web does not flow peer to peer the way information does. It flows through gateways, each of which can charge a fee, set terms, and decide who is allowed to transact at all. The web transmits data without asking permission and money only with it.

The trust gap became platforms. This is the subtlest and most powerful one. Because HTTP could not tell you whether a stranger was trustworthy, buying from a random website was genuinely risky. The solution that emerged was to not trust the stranger at all, and trust an intermediary instead. You do not trust the seller on Amazon; you trust Amazon. You do not trust the driver; you trust Uber. You do not trust the host; you trust Airbnb. Each of these companies is, at its core, a trust layer that the protocol never provided, monetised. And notice what happens once a company becomes the trust layer for a market: it can set the rules of that market, take a cut of every transaction, and remove any participant it chooses. That is not a business model bolted onto the web. It is a missing protocol layer, privatised.

The discovery gap became search. With no directory in the protocol, finding anything required an index, and building a good index was hard and expensive. Google built the best one, and in doing so became the front door to the web. A gap in the protocol turned into the most valuable position on the internet: the entity that decides what is findable.

The state gap became cookies, then tracking. HTTP's statelessness had to be worked around for anything interactive to function, so we got cookies. Cookies solved a genuine engineering problem, and then, because they let a party recognise you across visits, they became the foundation of the entire surveillance advertising economy. A patch for a protocol limitation became the infrastructure of tracking.

alt The Five Gaps (the anchor diagram)
alt The Five Gaps (the anchor diagram)

Look at the pattern in aggregate. Identity, payment, trust, discovery, state. Five gaps. Five industries. And in every single case, the entity that filled the gap acquired enormous power, not by conquering the web, but by supplying something the web needed and had never provided.

This is why I think the standard story of centralisation is wrong. It is usually told as a moral tale: the open web was hijacked by greedy corporations. But the mechanism was simpler and more structural than that. The protocols left holes. Holes in essential infrastructure get filled. Whoever fills a hole that everyone depends on becomes a chokepoint, and chokepoints accumulate power automatically, regardless of anyone's intentions.

If this argument sounds familiar, it should. It is the same one I made in my post on AI infrastructure: control concentrates in the layers underneath, and it cascades upward. The web is that argument told over thirty years.


Web3: The Right Diagnosis, and What Actually Happened

This is where Web3 enters, and I want to handle it carefully, because it is a topic where most writing is either evangelism or contempt, and neither is useful.

Give Web3 its due: the diagnosis was correct. Its central claim was that the web's centralisation was caused by missing protocol layers, particularly trust and identity and payment, and that if you could provide those at the protocol level instead of the platform level, you would remove the need for the intermediaries entirely. That is a genuinely serious argument, and it is the same argument I have just spent this post making. Web3 read the problem correctly.

The bet was that cryptography and distributed consensus could supply the missing layers. A blockchain could provide trust without a trusted intermediary. A wallet could provide identity without an identity provider. A token could provide payment without a payment processor. Fill the protocol gaps, and the platforms that grew inside them become unnecessary.

So what happened? The honest answer is that it has largely not worked at the scale it promised, and the infrastructure lens explains why better than either the hype or the mockery does.

It re-centralised at the layer below. This is the most instructive failure, and it is exactly what this series would predict. In November 2020, an Infura outage disrupted Ethereum wallets, DeFi platforms, and exchanges. MetaMask users could not access their funds. But the blockchain itself was fine, still processing blocks normally. The problem was the access layer: too many applications depended on a single infrastructure provider to reach the chain. The lesson the ecosystem drew was blunt, and it is the thesis of this entire post: even a decentralised blockchain can rely heavily on centralised gateways.

It happened again, larger. When a major AWS outage hit in October 2025, it exposed how heavily Web3 depends on centralised infrastructure. Coinbase went down. Its Base layer-2 network went down. Infura was degraded. MetaMask front-ends faltered. An ecosystem built explicitly to eliminate central points of failure was taken offline by a central point of failure, because someone still has to run the servers, and it turned out to be the same handful of companies as everyone else.

That is the pattern this series keeps finding. You can decentralise a protocol, but the protocol runs on infrastructure, and infrastructure has economics. Whoever pays for the servers, the bandwidth, and the reliability tends to end up in control, no matter what the protocol above them says.

It fought network effects and lost. The centralised incumbents were not just holding a technical position. They were holding a network effect. A decentralised alternative that is technically superior but has a tenth of the users is not, for most people, superior at all. This is the same wall I discussed in a recent comment about DNS: you cannot flag-day the internet. IPv6 is technically better than IPv4 and, decades in, still runs alongside it rather than replacing it. Better does not win. Better-plus-deployable wins.

And the incentives pulled it off course. Much of the energy that could have gone into building the missing layers went into speculation instead. Serious critical scholarship has questioned whether Web3 genuinely distributes power or merely shifts influence to a new tech-savvy elite, and a fair reading of the last few years makes that question hard to dismiss. Meanwhile the practical gateways people actually used, the exchanges, the wallet providers, the RPC endpoints, consolidated into exactly the kind of intermediaries the movement set out to abolish.

alt Web3's Re-Centralisation
alt Web3's Re-Centralisation

So the fair summary is this: Web3 identified the right problem and demonstrated, at real expense, that fixing a protocol gap with cryptography alone does not fix the centralisation, because centralisation is not only a protocol phenomenon. It is an infrastructure and economics phenomenon. That is a genuinely valuable finding, even though it is not the one anyone wanted.

And it is worth noting the quieter, more interesting sequel. HTTP 402 is finally being activated, thirty years on, through efforts to build a real payment primitive into HTTP, driven largely by the needs of AI agents that cannot click through a checkout page. Whether that succeeds is genuinely unsettled, and I would not bet either way. But it is the clearest possible demonstration of the thesis: the gap was real, it was left in the specification on purpose, and the pressure to fill it never went away.


What This Means for Anything Being Built Now

The reason this matters beyond history is that the pattern is not finished. It is running right now, on the next layer.

Every emerging technology arrives with its own set of protocol gaps, and the same race to fill them. AI systems have no native identity layer, no native payment layer, no native trust layer. Right now, those gaps are being filled, and the entities filling them are becoming the chokepoints of the next era, exactly as Google, Facebook, Amazon, and Stripe did for the web. The specific technologies change. The structure does not.

So the useful question, whenever you look at any new system, is not "is this decentralised?" It is: what does this protocol not provide, and who is going to provide it instead? That question tells you where the power will accumulate, usually years before it becomes obvious.

If you want to feel this rather than just read it, a few small projects make the gaps tangible, and I would suggest them as things to try rather than tutorials to follow:

Run your own service end to end, on your own VPS with your own domain, as I covered in a recent standalone guide. Doing it once teaches you exactly how much the platforms are actually doing for you, and where you are still dependent no matter how self-hosted you feel.

Build a small site that handles its own authentication instead of using "Sign in with Google," and notice how much work the identity gap really represents, and why almost everyone hands it to a platform.

Try to accept a payment without a processor, and watch how quickly you discover that the web has no native way to move money.

Serve something over a peer-to-peer protocol like IPFS and then ask yourself, honestly, who is pinning the data and who is running the gateway you are reaching it through.

Each of these is small. Each one teaches the same lesson from a different side: the gaps are real, filling them is hard, and that difficulty is precisely why the companies that filled them became so powerful.

alt How a Gap Becomes a Chokepoint
alt How a Gap Becomes a Chokepoint


What You Now Understand

We took a detour from the plumbing, but it was not really a detour.

You now know that the web's foundational protocols were deliberately minimal: they specified naming, requesting, and structuring documents, and left out identity, payment, trust, discovery, and state. That minimalism is why the web was deployable at all, so it is not a design flaw. It is the reason the web exists.

You know that those gaps did not stay empty. Each one was filled by a company, and each company that filled one became a chokepoint: login providers for identity, processors for payment, platforms for trust, search for discovery, cookies and tracking for state. Centralisation was not a hijacking. It was the market finishing an unfinished protocol, and collecting rent on the parts it completed.

You know that Web3 diagnosed this correctly and still largely failed to fix it, and you know why: it re-centralised at the infrastructure layer beneath the protocol, it fought network effects without a deployment path, and its incentives pulled it toward speculation. That is not a moral judgement, it is a structural one, and it is the same lesson the whole series teaches. Control lives in the layers underneath.

And most usefully, you have a question you can now apply to anything: what does this system not provide, and who will provide it instead? That is where the power goes.

In the next post we return to the plumbing, and to a question this interlude has quietly raised. We have talked a lot about how data finds its destination. Post 6 is about the choice every application makes once it gets there: reliability or speed. TCP and UDP, the three-way handshake, and the tradeoff that shapes everything from your bank transfer to your video call.


This was an interlude in the Networking Foundations series, sitting between Post 5 on DNS and Post 6 on TCP and UDP. If it gave you a new way to read the web's history, share it with someone who thinks centralisation was just greed. New here? The series starts with Post 1, on what a network actually is, and this interlude leans on everything from Posts 1 to 5. Subscribe to our newsletter to get each new post as it publishes.

Enjoyed this post?

Get notified when I publish next.

No spam — only new posts on networking, security, DevOps and infrastructure.

Comments

Leave a comment