The Platform Socialist Calculation Problem

The Socialist Calculation Problem refers to the difficulties planned economies have in co-ordinating production and consumption in even moderately complex societies. The Platform Socialist Calculation Problem occurs when computational resources are cheap, but inputs to the calculation are of low quality because of poorly structured social co-ordination.

One of the clearest and earliest presentations of the original Socialist Calculation Problem is by Hayek. With striking foresight, and concurrently with Shannon’s invention of information theory, Hayek pointed out that the economy is an immensely complicated computational problem. In a market economy, this computation is massively distributed, and mediated through the price mechanism, with many different people adjusting prices based on local information, and thereby propagating information on supply and demand throughout the entire system. 

Every attempt at socialist planning we’ve seen in practice has basically fallen over, due at least in part to this problem. The calculation is more or less a  linear programming problem with a truly enormous square matrix, every row and column of which is a different product in the economy. Even though computational power has increased by many orders of magnitude since the 1940s, there’s good arguments that it doesn’t match the staggering scale that such a combinatorial explosion requires. Cosma Shalizi’s informal, but very well-informed, left-wing re-presentation of the problem in terms of algorithmic complexity is a good summary.

Soviet central planning was the largest scale demonstration of this failure. For a fictional, but historically well-grounded, exploration of it, you really can’t go past Francis Spufford’s novel Red Plenty. It is beautiful, and computational, and sad. It weaves through the multiple perspectives of bureaucrats, engineers, farmers and intellectuals as they try to make the experiment of central planning work, and how it fails, on both the primitive computational hardware available, and the social gaming of the system that happens so that people can survive.

And yet.

A number of recent books and articles have made the argument that however scary the Big O notation for calculating a modern economy is in theory, in practice, the problem is tractable to a good enough approximation. The theoretical argument by people like Cottrell and Cockshott is that most of the goods across an entire economy are not needed as inputs to the entire economy, so most of the entries in the enormous matrices are zeroes. This simplifies the calculation. Building on that, the authors of The People’s Republic of Walmart argue in cheerful, accessible prose that giant computational planned economies have grown up in front of our eyes in the last twenty years. The most obvious of these are Walmart in the US, and globally, Amazon. The close links between WeChat, delivery systems, and the Chinese state is also notable. The basic idea of Platform Socialism is that a government could take over these economies and run them not according to wealth but according to need.

(You might also be asking: Why does it seem now like I’m reading a god-awful, capitalism-fellating airport business book? Suck it up. Socialism is about logistics, comrade.)

Phillips and Rozworski, The People’s Republic of Walmart

I think platform socialists may be onto something. We have so much more compute power now, and so much more comprehensive, cheaply accessible inventory, that I think there is a good argument that the computational aspects of the Socialist Calculation Problem are solvable in many practical settings, through good approximations. My view has shifted over the last twenty years. But so has the tech, and the logistical power of these companies; and I’ve also become aware how even in WWI the House of Morgan played a central planning role in the simplified war economy. 

But there’s a second aspect to the the problem of socialist planning, and that’s of needing an alternative input to prices to signal and resolve production and consumption needs and desires. This is the problem of feedback against material reality. Amazon and Walmart don’t suffer from this, because as big as they are, they still sit within a wider market economy, its prices, and the purchase decisions of hundreds of millions of consumers. 

The Soviet, and traditional planned economist, substitute for external prices, is that work units and/or living units go through a budget planning process on some annual cycle, then there are iterations that resolve conflicts, etc. I don’t see how this would be any less painful and detached from reality than the bureaucratic budgeting process in large corporations or government departments. I’ve read or listened to a number of expositions of such planning systems, and they are universally dire and unconvincing, usually in turgid Marxist prose that sounds like it was emptied out of the oil trap of an undermaintained Soviet tank, an expression of the authors’ secret wish for death over the prospect of living through departmental budget mud-wrestling splayed through every level of social interaction.

The answer given by PR Walmart is more interesting. They propose that something like product reviews at amazon.gov can be a more dynamic and useful input to the planning calculation, and the shadow prices therein. This would probably supplemented with lot of voting and discussion, often online.

The last two decades of platform experience is terribly reminiscent of just this kind of interaction. Instead of getting free or cheap stuff from governments in return for votes, we get free and cheap stuff from giant tech platforms in return for having ads served up to us. Generally, as a social form, it kind of sucks. In many ways it is worse than the pre-platform internet. As a result I am more pessimistic about socialist ratings being a solution than I would have been fifteen years ago. Make that economic ratings platform pervasive without some better structuring mechanisms and you won’t get farsighted climate solutions, you’ll get TikTok mobs stamping on a human Boaty McBoatface, forever.

This is an institutional design problem, in the same class of problems as parliamentary government itself: the Platform Socialist Calculation Problem. Feedback on material and social reality has to be timely, robust, and not vulnerable to spam and dumb mobs.

Socialism isn’t the only system with a calculation problem. Benjamin Bratton calls the failure of market economies to include inputs not easily represented as agent-centric numeric prices the Capitalist Calculation Problem. Climate change is the most obvious and urgent example. No agent represents the damage of excessive atmospheric carbon as a price, except, crudely and after much kicking, the state. Even then political will to set an explicit price reflective of the damage still isn’t there after more than thirty years of trying; what signals do exist are sent through carbon taxes and centrally planned net zero targets. So capitalism has a problem with feedback from social and material reality as well, though it has to be said that it kicks in at a much higher level of complexity.

The institutional nature of the problem suggests there may be constitutional solutions. I don’t really see any Western lefties working on it, except maybe two Marxist economists, and a few Ethereum DAO people (Distributed Autonomous Organizations). In China you have serious smaller scale experiments with social credit, but not with more material economic planning, that I’m aware of. The attempts to redirect monetary capital to social ends, like KlimaDAO, are more convincing (and Klima basically died). At any rate, across this selection, I have not been impressed by anything but the experimental gumption of the DAO soviets so far. One thing in their favour is that spinning up a small platform business/commune gives immediate feedback on and experience with these ideas; unlike the usual socialist fairy dust of “first we capture the state, then we get rid of capitalism, then we can invent the socialist economy”. Platform socialism that doesn’t learn how to help people to thrive from the ground up won’t be worth a damn.

We are learning to think of democracy not in terms of the history of an idea or the emergence of a social movement, but as the assembling of machines.

Mitchell, Carbon Democracy

References

Cottrell and Cockshott – Economic planning, computers and labor values

Hayek – Economics and Knowledge

Shalizi – In Soviet Union, Optimization Problem Solves You

Spufford – Red Plenty

Phillips and Rozworski – The People’s Republic of Walmart

Sinister Normative Compulsions

Michael Eby’s piece in New Left Review on Agile software development is interesting, and a little frustrating. I think it’s generally quite useful to get a left-wing, Marx-y class analysis on the structural of the software workplace. Eby argues Agile is a renegotiation within capitalism that by itself doesn’t challenge, and may indeed clarify, the executive power of the capitalist or manager, in the form of the Product Owner. This certainly seems true – it’s right there in the name.

I like the piece a bit more in concept than execution, though. There’s a few too many mistakes in the detail, getting the timeline of methods wrong, treating Scrum as separate to Agile when it’s the most popular method, talking about story points, and not lining up the jargon quite right in various ways. If I had to guess, I’d venture its a generalization from discussion with a few developers rather than a more careful reading of the key books and the c2 wiki. The full pedantic catalogue I will leave to the hacker news comments, while noting that process is more complicated to analyze than it looks, because software processes in practice rarely strictly follow a book. This is a point driven home by Alistair Cockburn’s description of developers always wanting to add stuff they don’t do to their description of their process – in Agile Software Development, as it happens.

Despite mixing up some of the details, Eby still has some sharp moments where he spots things you would never see in the usual software literature. Agile absolutely does have mechanisms of management discipline for workers, for instance:

It is clear that Agile dissolves many of the more visible features of hierarchical managerial control. But it does so only to recontain them in subtle and nuanced ways. For one, the self-organizing strategies of teams allow for certain workplace disciplinary mechanisms to take the form of normative compulsions rather than explicit instructions. 

The example that follows this quote isn’t great (again those details). I’d say instead that the most important disciplinary measure is frequent planning discussion and delivery – iterations and standups. It’s both a bug and a feature. Because they are the leverage points in this structure, iterations and standups are also where Dark Agile happens, if it happens.

But it’s still better. 

Agile work is more satisfying because a software worker has more control over the detail of what they produce. The owners and managers of the firm get more software delivery, through a change to internal communication and decision structures. To the degree society as a whole is helped by the software, it gets those helpful things sooner. It is too easy to forget how spectacularly wasteful waterfall development was. Enormous specification documents that were irrelevant by the time any code was cut. Months of kabuki theatre adversarial ping pong between development and testing teams. Years spent coding features and products never used. All of this was completely known and routine. Flows of waste and stupidity are hardly alien to software today, but at least in Agile a third of your projects aren’t thrown into the sea.

Agile techniques are more effective at delivering software, precisely by taking more technical decisions out of a bureaucratic org chart. Methodologies like Large Scale Scrum even emphasize that it is not a manager’s job to parcel out work, but to remove friction from the flow of money and production. This is surely fruit hanging low and ripe for a socialist critic to take.

Eby goes on to state:

[A] silent bargain between capital and wage-labour has occurred, with capital steadily shedding impediments to accumulation, and wage-earners forfeiting hard-won security in exchange for putative freedom.

This isn’t a terrible description of the new status quo, but again, the historical sequence seems off. The scythes of neoliberal deregulation had already been slicing into corporate and union bureaucracies alike all through the eighties and nineties. Downsizing, private equity buyouts and rightshoring were already well-rehearsed corporate practice before the Agile manifesto was signed in 2001. Agile has now shown itself so successful that it is propagated by management as corporate process. But you could also see it as skilled labour reacting to a disrupted expectation of long-term employment by taking control of low-level details of production, selling the change using increased labour productivity, and succeeding despite a lack of formal legal support. The lack of any explicit political theory (most of signatories of the manifesto are far from leftists) was probably helpful as well, in slipping under the radar, and avoiding the bureaucratic modes of 20th century unionism.

Various leftist thinkers, such as Phillips and Rozworski, have recently been pointing out that Amazon has so much size, computational power and control over its logistics that it is effectively a planned economy. This is true up to a point, but Jeff Bezos also popularized the “two pizza team” guidelines for agile team size. Internally Amazon has strong central control, platform planning, and for skilled software workers, devolved control of details to empowered teams that perform a high degree of horizontal co-ordination. And they build a lot of things. More Marx-inflected analysis of why that succeeds, while high modernist and Soviet central planning failed, would be welcome.

References

Cockburn – Agile Software Development

Phillips and Rozworski – The People’s Republic of Walmart