Sinister Normative Compulsions

Michael Eby’s piece in New Left Review on Agile software development is interesting, and a little frustrating. I think it’s generally quite useful to get a left-wing, Marx-y class analysis on the structural of the software workplace. Eby argues Agile is a renegotiation within capitalism that by itself doesn’t challenge, and may indeed clarify, the executive power of the capitalist or manager, in the form of the Product Owner. This certainly seems true – it’s right there in the name.

I like the piece a bit more in concept than execution, though. There’s a few too many mistakes in the detail, getting the timeline of methods wrong, treating Scrum as separate to Agile when it’s the most popular method, talking about story points, and not lining up the jargon quite right in various ways. If I had to guess, I’d venture its a generalization from discussion with a few developers rather than a more careful reading of the key books and the c2 wiki. The full pedantic catalogue I will leave to the hacker news comments, while noting that process is more complicated to analyze than it looks, because software processes in practice rarely strictly follow a book. This is a point driven home by Alistair Cockburn’s description of developers always wanting to add stuff they don’t do to their description of their process – in Agile Software Development, as it happens.

Despite mixing up some of the details, Eby still has some sharp moments where he spots things you would never see in the usual software literature. Agile absolutely does have mechanisms of management discipline for workers, for instance:

It is clear that Agile dissolves many of the more visible features of hierarchical managerial control. But it does so only to recontain them in subtle and nuanced ways. For one, the self-organizing strategies of teams allow for certain workplace disciplinary mechanisms to take the form of normative compulsions rather than explicit instructions. 

The example that follows this quote isn’t great (again those details). I’d say instead that the most important disciplinary measure is frequent planning discussion and delivery – iterations and standups. It’s both a bug and a feature. Because they are the leverage points in this structure, iterations and standups are also where Dark Agile happens, if it happens.

But it’s still better. 

Agile work is more satisfying because a software worker has more control over the detail of what they produce. The owners and managers of the firm get more software delivery, through a change to internal communication and decision structures. To the degree society as a whole is helped by the software, it gets those helpful things sooner. It is too easy to forget how spectacularly wasteful waterfall development was. Enormous specification documents that were irrelevant by the time any code was cut. Months of kabuki theatre adversarial ping pong between development and testing teams. Years spent coding features and products never used. All of this was completely known and routine. Flows of waste and stupidity are hardly alien to software today, but at least in Agile a third of your projects aren’t thrown into the sea.

Agile techniques are more effective at delivering software, precisely by taking more technical decisions out of a bureaucratic org chart. Methodologies like Large Scale Scrum even emphasize that it is not a manager’s job to parcel out work, but to remove friction from the flow of money and production. This is surely fruit hanging low and ripe for a socialist critic to take.

Eby goes on to state:

[A] silent bargain between capital and wage-labour has occurred, with capital steadily shedding impediments to accumulation, and wage-earners forfeiting hard-won security in exchange for putative freedom.

This isn’t a terrible description of the new status quo, but again, the historical sequence seems off. The scythes of neoliberal deregulation had already been slicing into corporate and union bureaucracies alike all through the eighties and nineties. Downsizing, private equity buyouts and rightshoring were already well-rehearsed corporate practice before the Agile manifesto was signed in 2001. Agile has now shown itself so successful that it is propagated by management as corporate process. But you could also see it as skilled labour reacting to a disrupted expectation of long-term employment by taking control of low-level details of production, selling the change using increased labour productivity, and succeeding despite a lack of formal legal support. The lack of any explicit political theory (most of signatories of the manifesto are far from leftists) was probably helpful as well, in slipping under the radar, and avoiding the bureaucratic modes of 20th century unionism.

Various leftist thinkers, such as Phillips and Rozworski, have recently been pointing out that Amazon has so much size, computational power and control over its logistics that it is effectively a planned economy. This is true up to a point, but Jeff Bezos also popularized the “two pizza team” guidelines for agile team size. Internally Amazon has strong central control, platform planning, and for skilled software workers, devolved control of details to empowered teams that perform a high degree of horizontal co-ordination. And they build a lot of things. More Marx-inflected analysis of why that succeeds, while high modernist and Soviet central planning failed, would be welcome.

References

Cockburn – Agile Software Development

Phillips and Rozworski – The People’s Republic of Walmart

Dodgy Steve’s Software Manifestagogo

We have totally nailed better ways of developing software by drinking beer and talking about it. This incontrovergitibly shows you should choose:

Beer over processes and tools
Beer over comprehensive documentation
Beer over contract negotiation
Beer over following a plan.

That is, while there is value in the items on the right, we value the items on the left more.

We also really feel like a kebab.

Industrializing The Noosphere

Control Environment

We are not practicing Continuous Delivery. I know this because Jez Humble has given a very clear test. If you are not checking your code into trunk at least once every day, says Jez, you are not doing Continuous Integration, and that is a prerequisite for Continuous Delivery. I don’t do this; no one on my team does it; no one is likely to do it any time soon, except by an accident of fortuitous timing. Nevertheless, his book is one of the most useful books on software development I have read. We have used it as a playbook for improvement, with the result being highly effective software delivery. Our experience is one small example of an interesting historical process, which I’d like to sketch in somewhat theoretical terms. Software is a psychologically intimate technology. Much as described by Gilbert Simondon’s work on technical objects, building software has evolved from a distractingly abstract high modernist endeavour to something more dynamic, concrete and useful.

The term software was co-opted, in a computing context, around 1953, and had time to grow only into an awkward adolescence before being declared, fifteen years later, as in crisis. Barely had we become aware of software’s existence before we found it to be a frightening, unmanageable thing, upsetting our expected future of faster rockets and neatly ordered suburbs. Many have noted that informational artifacts have storage and manufacturing costs approaching zero. I remember David Carrington, in one of my first university classes on computing, noting that as a consequence of this, software maintenance is fundamentally a misnomer. What we speak of as maintenance in physical artifacts, the replacement of entropy-afflicted parts with equivalents within the same design, is a nonsense in software. The closest analogue might be system administrative activities like bouncing processes and archiving logfiles. What we call (or once called) maintenance is revising our understanding of the problem space.

Software has an elastic fragility to it, pliable, yet prone to the discontinuities and unsympathetic precision of logic. Lacking an intuition of digital inertia, we want to change programs about as often as we change our minds, and get frustrated when we cannot. In their book Continuous Delivery, Humble and Farley say we can change programs like that, or at least be changing live software very many times a day, such that software development is not the bottleneck in product development.

With this approach, we see a rotation and miniaturisation of mid-twentieth century models of software development. The waterfall is turned on its side.

Continue reading