The Platform Socialist Calculation Problem

The Socialist Calculation Problem refers to the difficulties planned economies have in co-ordinating production and consumption in even moderately complex societies. The Platform Socialist Calculation Problem occurs when computational resources are cheap, but inputs to the calculation are of low quality because of poorly structured social co-ordination.

One of the clearest and earliest presentations of the original Socialist Calculation Problem is by Hayek. With striking foresight, and concurrently with Shannon’s invention of information theory, Hayek pointed out that the economy is an immensely complicated computational problem. In a market economy, this computation is massively distributed, and mediated through the price mechanism, with many different people adjusting prices based on local information, and thereby propagating information on supply and demand throughout the entire system. 

Every attempt at socialist planning we’ve seen in practice has basically fallen over, due at least in part to this problem. The calculation is more or less a  linear programming problem with a truly enormous square matrix, every row and column of which is a different product in the economy. Even though computational power has increased by many orders of magnitude since the 1940s, there’s good arguments that it doesn’t match the staggering scale that such a combinatorial explosion requires. Cosma Shalizi’s informal, but very well-informed, left-wing re-presentation of the problem in terms of algorithmic complexity is a good summary.

Soviet central planning was the largest scale demonstration of this failure. For a fictional, but historically well-grounded, exploration of it, you really can’t go past Francis Spufford’s novel Red Plenty. It is beautiful, and computational, and sad. It weaves through the multiple perspectives of bureaucrats, engineers, farmers and intellectuals as they try to make the experiment of central planning work, and how it fails, on both the primitive computational hardware available, and the social gaming of the system that happens so that people can survive.

And yet.

A number of recent books and articles have made the argument that however scary the Big O notation for calculating a modern economy is in theory, in practice, the problem is tractable to a good enough approximation. The theoretical argument by people like Cottrell and Cockshott is that most of the goods across an entire economy are not needed as inputs to the entire economy, so most of the entries in the enormous matrices are zeroes. This simplifies the calculation. Building on that, the authors of The People’s Republic of Walmart argue in cheerful, accessible prose that giant computational planned economies have grown up in front of our eyes in the last twenty years. The most obvious of these are Walmart in the US, and globally, Amazon. The close links between WeChat, delivery systems, and the Chinese state is also notable. The basic idea of Platform Socialism is that a government could take over these economies and run them not according to wealth but according to need.

(You might also be asking: Why does it seem now like I’m reading a god-awful, capitalism-fellating airport business book? Suck it up. Socialism is about logistics, comrade.)

Phillips and Rozworski, The People’s Republic of Walmart

I think platform socialists may be onto something. We have so much more compute power now, and so much more comprehensive, cheaply accessible inventory, that I think there is a good argument that the computational aspects of the Socialist Calculation Problem are solvable in many practical settings, through good approximations. My view has shifted over the last twenty years. But so has the tech, and the logistical power of these companies; and I’ve also become aware how even in WWI the House of Morgan played a central planning role in the simplified war economy. 

But there’s a second aspect to the the problem of socialist planning, and that’s of needing an alternative input to prices to signal and resolve production and consumption needs and desires. This is the problem of feedback against material reality. Amazon and Walmart don’t suffer from this, because as big as they are, they still sit within a wider market economy, its prices, and the purchase decisions of hundreds of millions of consumers. 

The Soviet, and traditional planned economist, substitute for external prices, is that work units and/or living units go through a budget planning process on some annual cycle, then there are iterations that resolve conflicts, etc. I don’t see how this would be any less painful and detached from reality than the bureaucratic budgeting process in large corporations or government departments. I’ve read or listened to a number of expositions of such planning systems, and they are universally dire and unconvincing, usually in turgid Marxist prose that sounds like it was emptied out of the oil trap of an undermaintained Soviet tank, an expression of the authors’ secret wish for death over the prospect of living through departmental budget mud-wrestling splayed through every level of social interaction.

The answer given by PR Walmart is more interesting. They propose that something like product reviews at amazon.gov can be a more dynamic and useful input to the planning calculation, and the shadow prices therein. This would probably supplemented with lot of voting and discussion, often online.

The last two decades of platform experience is terribly reminiscent of just this kind of interaction. Instead of getting free or cheap stuff from governments in return for votes, we get free and cheap stuff from giant tech platforms in return for having ads served up to us. Generally, as a social form, it kind of sucks. In many ways it is worse than the pre-platform internet. As a result I am more pessimistic about socialist ratings being a solution than I would have been fifteen years ago. Make that economic ratings platform pervasive without some better structuring mechanisms and you won’t get farsighted climate solutions, you’ll get TikTok mobs stamping on a human Boaty McBoatface, forever.

This is an institutional design problem, in the same class of problems as parliamentary government itself: the Platform Socialist Calculation Problem. Feedback on material and social reality has to be timely, robust, and not vulnerable to spam and dumb mobs.

Socialism isn’t the only system with a calculation problem. Benjamin Bratton calls the failure of market economies to include inputs not easily represented as agent-centric numeric prices the Capitalist Calculation Problem. Climate change is the most obvious and urgent example. No agent represents the damage of excessive atmospheric carbon as a price, except, crudely and after much kicking, the state. Even then political will to set an explicit price reflective of the damage still isn’t there after more than thirty years of trying; what signals do exist are sent through carbon taxes and centrally planned net zero targets. So capitalism has a problem with feedback from social and material reality as well, though it has to be said that it kicks in at a much higher level of complexity.

The institutional nature of the problem suggests there may be constitutional solutions. I don’t really see any Western lefties working on it, except maybe two Marxist economists, and a few Ethereum DAO people (Distributed Autonomous Organizations). In China you have serious smaller scale experiments with social credit, but not with more material economic planning, that I’m aware of. The attempts to redirect monetary capital to social ends, like KlimaDAO, are more convincing (and Klima basically died). At any rate, across this selection, I have not been impressed by anything but the experimental gumption of the DAO soviets so far. One thing in their favour is that spinning up a small platform business/commune gives immediate feedback on and experience with these ideas; unlike the usual socialist fairy dust of “first we capture the state, then we get rid of capitalism, then we can invent the socialist economy”. Platform socialism that doesn’t learn how to help people to thrive from the ground up won’t be worth a damn.

We are learning to think of democracy not in terms of the history of an idea or the emergence of a social movement, but as the assembling of machines.

Mitchell, Carbon Democracy

References

Cottrell and Cockshott – Economic planning, computers and labor values

Hayek – Economics and Knowledge

Shalizi – In Soviet Union, Optimization Problem Solves You

Spufford – Red Plenty

Phillips and Rozworski – The People’s Republic of Walmart

Waistcoatpunk

The supposedly definitive speech for hopepunk is given by Samwise Gamgee in The Two Towers: “That there’s some good in the world”, and so on.  It’s a fine speech, and it would be churlish indeed to pick on good-hearted Sam. But recently I read (or possibly re-read) Michael Moorcock’s old essay on Tolkien, as well; the one that pegs Lord of the Rings as an armour-clad Winnie the Pooh.

I was never quite the Tolkien-maniac so many teenagers are, though I won’t claim it was out of social competence or any foresightful discomfort with JRR’s racial scheme. It was probably because I’d read too much trash fantasy before I got hold of The Fellowship of the Ring, and was hoping for more of the same. Even though I knew it was unfair, it seemed like Tolkien was plagiarizing himself. 

Underneath, I think the structure of the world and the story might have also got up my nose. I always sympathized more with elves than hobbits. More fool me, perhaps. All of the hobbits have a bit too much Forrest Gump in them, a much worse work of art whose magic moron morality I will continue to hate until my dying day.

I was certainly up for Moorcock’s pulp literary experimenta. My taste was much more for Melniboné than Bag End. And he’s a pretty good critic. He gets the politics of Middle Earth and Middle Hopepunk bang on:

While there is an argument for the reactionary nature of the books, they are certainly deeply conservative and strongly anti-urban, which is what leads some to associate them with a kind of Wagnerish hitlerism. I don’t think these books are ‘fascist’, but they certainly don’t exactly argue with the 18th century enlightened Toryism with which the English comfort themselves so frequently in these upsetting times.

Epic Pooh

It’s not that hopepunk is bad. Like Tolkien, it’s deeply anti-fascist, and pro-community, in a rural traditionalist way. But it’s as punk as a hobbit in a four button waistcoat. It’s comfortable to wear, but I remain skeptical it will change much.

Every Fucking Platform

phone
Every Fucking Platform

We argued on the evening before lockdown
The lonely morning made it sweet again 
But by the time we needed food from Woolies
Everything I said was wrong and you cursed me in the rain
We split up for a while after a phone call
I pinged you six days later on WhatsApp
I was hoping that the break 
Would make things go a little better for us
And for a little while it almost did

Now I’m having drinks in front of YouTube
And I’m trying hard to forget your name
I’m staring at the label of a bottle of Corona
And every fucking platform feels the same

You said you had a question about Fleabag
But all of my replies were left on read
And when I checked on Facebook you were posting cats
And screenshots of your Duolingo French
Foolishly, I followed you to Twitter
Like a ghost I doomscrolled Trump and Kamala
And all the bright young things
Were throwing up their takes on Harry Potter
While I stared at your unfollowed avatar

Now I’m watching egirls dance on TikTok
And they’re singing that bloody sea shanty again
And I can’t believe I’m dancing to this but a cute girl liked my last one
And every fucking platform sounds the same

While waiting for a coffee with my mask on
An email from you brought me to Discord
But all the channels seemed to be for tankie poly rationalists
And anyway, it sounded like you weren’t alone
So I drifted till I ended up on OnlyFans
A horny platform for a troubled soul
And for a month I paid a woman far too much
To flash her tits and make me feel trolled

Now I’m sitting playing rounds of DOTA
And a Chinese guy wants me to know his name
And I can echo idioms
In seven different acronyms 
But every fucking platform looks the same
Cya l8r, 88, gg, ragequit, brb
Every fucking platform’s just the same

(With a debt to Paul Kelly’s original.)

Retooling the Social Media Contract

Why would Reddit spend developer time on cryptocoins?

I think to understand this, you have to see the current situation of a handful of highly centralized platforms, with centralized moderation, and enormous commercial and media power, as a kind of disaster. From that perspective, which I basically share, this is an attempt to make better technical infrastructure for communities of curation and moderation.

A recent symptom of this disaster: Novara Media, the independent leftwimg British media organization, had its entire Youtube channel deleted without notice or serious explanation. Novara is quite high profile, too: a number of its hosts, like Ash Sarkar, have regular columns and make appearances in more mainstream media. It was reinstated about a day later after various public figures across the UK political spectrum weighed in. Nothing but the most generic corporate response was offered. They had no previous strikes.

This is hardly the first example, and it is more related to bad platform design – technical and governance – than cancel culture or any primarily socially-driven phenomena. It wasn’t in response to any particular statement or social media outrage storm. The current platforms foster these problems the way giant monocropped fields invite plagues of locusts.

Facebook and YouTube are at one extreme because they are central platforms with few community or user controls for the bulk of the platform experience.

Take YouTube. How many interactions can you have with it? Like, dislike, watch, comment (requires semantic interpretation), report (requires human, eventually). You can search for specific videos or follow the feed. It is widely indexed and close-ish to the open web.

It’s also one flat user social structure, feeding into a corporate policing structure. The model there is deep learning trained filters feeding a large team of corporate censors, mostly not in-house, but outsourced to censorship specialists in the Philippines and other middle income countries. It’s essentially a design based on a chemical factory toxic waste pipeline. The whole focus on improving it is just about making a better waste filter, essentially out of deep learning tech.

Facebook is similar but not even on the open web, it’s a non-indexable enclosure. In many ways Facebook is best understood as an anti-website. In addition the ML-determined feed is even more dominant, making for even less user agency.

Twitter is largely flat as well, but has always been on the open web, had a usable API, and is open to bots and experiments. By being pseudonym and multi-account friendly it also makes separate curation across different accounts easier.

All of these platforms still have the factory toxic waste pipeline design. The also have always had some form of non-exchangeable social media currency. I’ve said previously that this makes them accidental reserve banks of sincerity.

The two main exceptions to the toxic waste pipleline design are Reddit and Mastodon. Reddit is much bigger, and for-profit, but is not FAANG (MANGA) big. Reddit is built around the idea of specialized communities built around shared interests. These communities come with community moderation and curation built in. It also has a pooled reputational currency (karma) and exchange currency support.

YouTube and now Twitter have got into the superchat / exchange currency pledges, but on a socially flat technical foundation. Reddit has a technical structure that includes community moderation and curation at a far more local level. Then when the platform police – or the meatspace police, for that matter – get involved, it’s rarer, and after a previous locally semi-autonomous policing and socialization process has happened.

Federation also gives more scope for individual communities to match the most appropriate national laws, and concentrates criminals and abusers in identifiable communities around the topics of their obsession.

The main platform built on federation is Mastodon, though it is of course re-applying patterns behind the internet itself. Mastodon is a social media protocol where people run nodes and choose which other nodes are on the network. Though fascist nodes exist, the main communities just refuse to federate with them.

Where do Reddit’s new community points fit with this? The write-up highlights autonomy, and developers speaking elsewhere have talked about federation. That contains much of the mastodon model. Perhaps with such a solution, Reddit could radically federate, to avoid the thousands of censors processing toxic waste that other platforms have, and emphasize their role as a platform at arms-length from content. That means expanding community self-management and making clearer organizational separation. But Reddit also want to make money, and they already govern two currencies. This leads fairly quickly to some form of tokens, and maybe some reputational stake or sponsoring system when communities federate or certain privileges are achieved.

There has been so much crypto hype, mostly without understanding the necessary latency and complexity tradeoffs, that I would start from a position of skepticism for deploying it as a technical solution on an existing platform. There have been a lot of tenuously useful blockchain projects. For Reddit, however, if they want a federal architecture and localize content management, the contract mechanisms in Ethereum already have a similar shape, and reinventing the wheel would be a waste. This is a real use case that may actually requires the power and cost of Ethereum smart contracts, or some more user-accessible extension.

Sinister Normative Compulsions

Michael Eby’s piece in New Left Review on Agile software development is interesting, and a little frustrating. I think it’s generally quite useful to get a left-wing, Marx-y class analysis on the structural of the software workplace. Eby argues Agile is a renegotiation within capitalism that by itself doesn’t challenge, and may indeed clarify, the executive power of the capitalist or manager, in the form of the Product Owner. This certainly seems true – it’s right there in the name.

I like the piece a bit more in concept than execution, though. There’s a few too many mistakes in the detail, getting the timeline of methods wrong, treating Scrum as separate to Agile when it’s the most popular method, talking about story points, and not lining up the jargon quite right in various ways. If I had to guess, I’d venture its a generalization from discussion with a few developers rather than a more careful reading of the key books and the c2 wiki. The full pedantic catalogue I will leave to the hacker news comments, while noting that process is more complicated to analyze than it looks, because software processes in practice rarely strictly follow a book. This is a point driven home by Alistair Cockburn’s description of developers always wanting to add stuff they don’t do to their description of their process – in Agile Software Development, as it happens.

Despite mixing up some of the details, Eby still has some sharp moments where he spots things you would never see in the usual software literature. Agile absolutely does have mechanisms of management discipline for workers, for instance:

It is clear that Agile dissolves many of the more visible features of hierarchical managerial control. But it does so only to recontain them in subtle and nuanced ways. For one, the self-organizing strategies of teams allow for certain workplace disciplinary mechanisms to take the form of normative compulsions rather than explicit instructions. 

The example that follows this quote isn’t great (again those details). I’d say instead that the most important disciplinary measure is frequent planning discussion and delivery – iterations and standups. It’s both a bug and a feature. Because they are the leverage points in this structure, iterations and standups are also where Dark Agile happens, if it happens.

But it’s still better. 

Agile work is more satisfying because a software worker has more control over the detail of what they produce. The owners and managers of the firm get more software delivery, through a change to internal communication and decision structures. To the degree society as a whole is helped by the software, it gets those helpful things sooner. It is too easy to forget how spectacularly wasteful waterfall development was. Enormous specification documents that were irrelevant by the time any code was cut. Months of kabuki theatre adversarial ping pong between development and testing teams. Years spent coding features and products never used. All of this was completely known and routine. Flows of waste and stupidity are hardly alien to software today, but at least in Agile a third of your projects aren’t thrown into the sea.

Agile techniques are more effective at delivering software, precisely by taking more technical decisions out of a bureaucratic org chart. Methodologies like Large Scale Scrum even emphasize that it is not a manager’s job to parcel out work, but to remove friction from the flow of money and production. This is surely fruit hanging low and ripe for a socialist critic to take.

Eby goes on to state:

[A] silent bargain between capital and wage-labour has occurred, with capital steadily shedding impediments to accumulation, and wage-earners forfeiting hard-won security in exchange for putative freedom.

This isn’t a terrible description of the new status quo, but again, the historical sequence seems off. The scythes of neoliberal deregulation had already been slicing into corporate and union bureaucracies alike all through the eighties and nineties. Downsizing, private equity buyouts and rightshoring were already well-rehearsed corporate practice before the Agile manifesto was signed in 2001. Agile has now shown itself so successful that it is propagated by management as corporate process. But you could also see it as skilled labour reacting to a disrupted expectation of long-term employment by taking control of low-level details of production, selling the change using increased labour productivity, and succeeding despite a lack of formal legal support. The lack of any explicit political theory (most of signatories of the manifesto are far from leftists) was probably helpful as well, in slipping under the radar, and avoiding the bureaucratic modes of 20th century unionism.

Various leftist thinkers, such as Phillips and Rozworski, have recently been pointing out that Amazon has so much size, computational power and control over its logistics that it is effectively a planned economy. This is true up to a point, but Jeff Bezos also popularized the “two pizza team” guidelines for agile team size. Internally Amazon has strong central control, platform planning, and for skilled software workers, devolved control of details to empowered teams that perform a high degree of horizontal co-ordination. And they build a lot of things. More Marx-inflected analysis of why that succeeds, while high modernist and Soviet central planning failed, would be welcome.

References

Cockburn – Agile Software Development

Phillips and Rozworski – The People’s Republic of Walmart