Sinister Normative Compulsions

Michael Eby’s piece in New Left Review on Agile software development is interesting, and a little frustrating. I think it’s generally quite useful to get a left-wing, Marx-y class analysis on the structural of the software workplace. Eby argues Agile is a renegotiation within capitalism that by itself doesn’t challenge, and may indeed clarify, the executive power of the capitalist or manager, in the form of the Product Owner. This certainly seems true – it’s right there in the name.

I like the piece a bit more in concept than execution, though. There’s a few too many mistakes in the detail, getting the timeline of methods wrong, treating Scrum as separate to Agile when it’s the most popular method, talking about story points, and not lining up the jargon quite right in various ways. If I had to guess, I’d venture its a generalization from discussion with a few developers rather than a more careful reading of the key books and the c2 wiki. The full pedantic catalogue I will leave to the hacker news comments, while noting that process is more complicated to analyze than it looks, because software processes in practice rarely strictly follow a book. This is a point driven home by Alistair Cockburn’s description of developers always wanting to add stuff they don’t do to their description of their process – in Agile Software Development, as it happens.

Despite mixing up some of the details, Eby still has some sharp moments where he spots things you would never see in the usual software literature. Agile absolutely does have mechanisms of management discipline for workers, for instance:

It is clear that Agile dissolves many of the more visible features of hierarchical managerial control. But it does so only to recontain them in subtle and nuanced ways. For one, the self-organizing strategies of teams allow for certain workplace disciplinary mechanisms to take the form of normative compulsions rather than explicit instructions. 

The example that follows this quote isn’t great (again those details). I’d say instead that the most important disciplinary measure is frequent planning discussion and delivery – iterations and standups. It’s both a bug and a feature. Because they are the leverage points in this structure, iterations and standups are also where Dark Agile happens, if it happens.

But it’s still better. 

Agile work is more satisfying because a software worker has more control over the detail of what they produce. The owners and managers of the firm get more software delivery, through a change to internal communication and decision structures. To the degree society as a whole is helped by the software, it gets those helpful things sooner. It is too easy to forget how spectacularly wasteful waterfall development was. Enormous specification documents that were irrelevant by the time any code was cut. Months of kabuki theatre adversarial ping pong between development and testing teams. Years spent coding features and products never used. All of this was completely known and routine. Flows of waste and stupidity are hardly alien to software today, but at least in Agile a third of your projects aren’t thrown into the sea.

Agile techniques are more effective at delivering software, precisely by taking more technical decisions out of a bureaucratic org chart. Methodologies like Large Scale Scrum even emphasize that it is not a manager’s job to parcel out work, but to remove friction from the flow of money and production. This is surely fruit hanging low and ripe for a socialist critic to take.

Eby goes on to state:

[A] silent bargain between capital and wage-labour has occurred, with capital steadily shedding impediments to accumulation, and wage-earners forfeiting hard-won security in exchange for putative freedom.

This isn’t a terrible description of the new status quo, but again, the historical sequence seems off. The scythes of neoliberal deregulation had already been slicing into corporate and union bureaucracies alike all through the eighties and nineties. Downsizing, private equity buyouts and rightshoring were already well-rehearsed corporate practice before the Agile manifesto was signed in 2001. Agile has now shown itself so successful that it is propagated by management as corporate process. But you could also see it as skilled labour reacting to a disrupted expectation of long-term employment by taking control of low-level details of production, selling the change using increased labour productivity, and succeeding despite a lack of formal legal support. The lack of any explicit political theory (most of signatories of the manifesto are far from leftists) was probably helpful as well, in slipping under the radar, and avoiding the bureaucratic modes of 20th century unionism.

Various leftist thinkers, such as Phillips and Rozworski, have recently been pointing out that Amazon has so much size, computational power and control over its logistics that it is effectively a planned economy. This is true up to a point, but Jeff Bezos also popularized the “two pizza team” guidelines for agile team size. Internally Amazon has strong central control, platform planning, and for skilled software workers, devolved control of details to empowered teams that perform a high degree of horizontal co-ordination. And they build a lot of things. More Marx-inflected analysis of why that succeeds, while high modernist and Soviet central planning failed, would be welcome.

References

Cockburn – Agile Software Development

Phillips and Rozworski – The People’s Republic of Walmart

Metis and General Intellect

“The general intellect” can be interpreted as tacit craft knowledge embedded in individual cunning and social relations. This definition sets general intellect in contrast and opposition to formal information systems. Framing it this way may not be completely true to historic usage, but has some revealing consequences. It applies to either abstract information machines like traditional bureaucracies, or concretized ones, like specific computer programs. Craft and process arguments in software development are also a lens on this transformation of skills in cognitive capitalism, information valorization and their relationship with bureaucracy and flows of stupidity. To expand the general intellect is to accelerate.

The term the general intellect (sometimes general social knowledge) originates from Marx and was used by operaismo thinkers to grapple with the emergence of cognitive capitalism. Virno states:

Whereas money, the “universal equivalent” itself, incarnates in its independent existence the commensurability of products, jobs, and subjects, the general intellect instead stabilizes the analytic premises of every type of practice. Models of social knowledge do not equate the various activities of labor, but rather present themselves as the “immediate forces of production.”

They are not units of measure, but they constitute the immeasurability presupposed by heterogeneous operative possibilities.

They are not “species” existing outside of the “individuals” who belong to them, but axiomatic rules whose validity does not depend on what they represent. Measuring and representing nothing, these technico-scientific codes and paradigms manifest themselves as constructive principles.

From here I suggest general intellect as a form of tacit and social knowledge, of metis, defined in contrast to formal knowledge, numeric or systematic knowledge. The general intellect is cognition but it is not data. It is highly contextualized, by experience, locality and specific social relations. James C. Scott offers the knowledge embodied by a soccer team or a ship’s pilot as good examples of metis. He also uses it in contrast to systematic forms of knowledge used in high modernist, often Fordist projects of top-down political control, state or corporate.

At the economy scale, the general intellect also seems to have an intersection with the idea of “institutions” in economic development. Institutions established ongoing government policies but also more abstract things like property rights and the rule of law; they are not primarily buildings, but persistent social relations, not commoditized or readily transferable between nations. Acemoglu and colleagues found significant correlation between historic settler mortality and modern economic success, putting forward the type of colonial institutions as the causative link.

Though this is oppositional with computation as a form of knowledge, the two are complementary in production. Operators draw on the general intellect to make machines work and produce things. This is true of concrete machines, like a coffee maker, or abstract machines in the Simondon / Deleuze / Guattari sense. And machines, especially large abstract machines, make use of operator black boxes to be effective. Traditional bureaucracy before the advent of modern computers and networks is an example of an abstract information machine which uses human operators as black box components – for instance to persist information to longer term storage, by writing on paper.

If we look at the skilled cognition involved in designing complex machinery, such as computer programming, we find the general intellect in the sense of individual and organizational craft knowledge. The rise of agile software development techniques, emphasising teamwork and craft skills over Fordist or high modernist planning, is one example of this over the last fifteen years. Yet the act of programming depends very much on an individual mental model, as pointed out by Naur. Programming is not typing; the main productive activity in programming is building a coherent mental model, the actual executable code produced is a side-effect. “Programming in this sense must be the programmers’ building up knowledge of a certain kind, knowledge taken to be basically the programmers’ immediate possession” (Naur). The spread of algorithms and software throughout society would then suggest a shattering of the general intellect into millions of shards of specific intellects. The general intellect – the entirety of system relations – could decay even as systemic shards expand in sophistication.

None of this is deny a certain translatability from metis to formalized knowledge. It can all be boiled down to bits in the end. Translation for functional use is costly and lossy, though. The mechanics of deep learning parallel this metic transformation from formal data structures into occluded knowledge. To understand how a deep learning system internally navigates a problem space requires a separate systematic analysis alien to the learning mechanism of the algorithm. Deep learning is a localized preview of machinic metis.

A countertrend to the fragmentation of general intellect might be the success of open source, but the point of open source is precisely to make the executable details of machines more readily available through social processes. It is a common platform rather than a general intellect, where evolution of the platform happens through patches (explicit formal communication) rather than primarily through evolved social understanding, though those dynamics still exist. It is striking that open source communities are organized primarily around a specific machine or platform rather than user products. This is true from the GNU C compiler through to the Apache web server and git source control. They echo Simondon’s critique of objects made in capitalism not evolving but merely accumulating features. Simondon’s comments on technical culture also parallel general social knowledge:

Now that he is a technical being no longer, man is forced to find for himself a position in the technical ensemble that is something other than the position of individual.

The trend for computer programming to promote skilling up for designers but sometimes exporting deskilling elsewhere was noted by Mackenzie in 1984; it’s because capitalism is a valorizing process rather than a deskilling process per se. Likewise there are deskilling trends in the software industry around outsourcing highly specific work to remote or offshore teams, so long as it promotes valorization (increases shareholder value).

The frustrations of working in or with a bureaucracy are often those of being a black box cog in a larger abstract machine, either through alienation from the meaning of the work, or because the work actually causes an undesirable effect which conflicts with personal goals, or even the stated goal of the organization itself. That is a form of stupidity but relates to all bureaucracy. Deleuze and Guattari say in capitalism:

The apparatus of antiproduction is no longer a transcendent instance that opposes production, limits it, or checks it; on the contrary, it insinuates itself everywhere in the productive machine and becomes firmly wedded to it in order to regulate its productivity and realize surplus value which explains, for example, the difference between the despotic bureaucracy and the capitalist bureaucracy.

eg in a despotic state the army may come and confiscate food and labour from a subsistence farmer when fighting a war, but in capitalism this military anti-production is in the form of a military-industrial complex, production interleaved with anti-production. Yet those critiques could apply to Soviet socialism too; only capitalism manages to create demand and ensure lack in the midst of overabundance.

Deleuze takes stupidity as an inability to dissociate from presuppositions, sense rather than common sense. Contemplating flows of stupidity, I am reminded of the slogan of engineer Jesse Robbins for making useful things in corporate bureaucracies: “Don’t fight stupid, make more awesome”. This could also serve as an accelerationist slogan, and can be critiqued the same way. Are you pushing forward as an elite ignoring politically hard problems, or are you building a concrete alternative solution that will attract change? Are you wasting time trying to change a system accelerated beyond human comprehension, or are you using accelerated human components to build a new system?

References

Acemoglu et al – The Colonial Origins of Comparative Development
Beck et al – Agile Software Development Manifesto
Deleuze and Guattari – The Anti-Oedipus
Garton – Excavating the Origins of Accelerationism
Land – A Quick and Dirty Introduction to Accelerationism
Mackenzie – Marx and the Machine
Naur – Programming As Theory Building
Scott – Seeing Like A State
Simondon – On The Mode of Existence of Technical Objects
Virno – Virtuosity and Revolution: The Political Theory of Exodus, in Radical Thought In Italy
Williams / Srnicek – #accelerate Manifesto for an Accelerationist Politics

Early Thoughts On Large Scale Scrum

Some people lose their sense of perspective. I’ve lost my sense of scale. – Will Self, Scale

We learn, in computing, that choice of data structure determines the algorithms one can run, their complexity, and the corresponding performance characteristics of an application. To talk of an algorithm without a data structure is to risk meaninglessness, for the two exist in a system, like the hub and spoke of a bicycle wheel. Likewise, Scrum and Scrum-like techniques are software processes, but they are just as fundamentally about changing team structure as about the various meetings and practices then enabled. Waterfall processes are pipelines of handoff between pinned threads organized by a task allocator-manager. Scrum is a common thread pool pulling work from a priority queue, then creating changes to a shared blackboard of state, which a product manager thread reacts to by adding or editing tasks on the queue. It’s model-view-controller: the code is the model, the production system is the view, and the product owner is the controller.

When organizations find success with agile, or Scrum, or something they call that which improves their delivery, they might reasonably want to expand on that success, simply because many organizations are larger than a team of seven people. Hence the demand for agile processes at scale. The demand, or criticism, of agile techniques ignoring large teams has been there for pretty much the whole history of the movement, but fifteen years in, we now have some road tested processes that try to address it directly. One of them is Large Scale Scrum (LeSS), and I had the good fortune this year to be introduced to it by one of its creators, Bas Vodde. That three day course and a bit of reading is the main source for these notes. There’s a lot of good material on the LeSS site.

Vodde and Larman say that Large Scale Scrum is Scrum, with some justification. There are various ways to combine small teams in delivering solutions for a client; you can federate, you can have cross cutting program managers and architects of some kind, create product governance committees, and so on. LeSS chooses to scale beyond one team working on a product by keeping the single product work queue pattern, and increasing the thread pool. The worker thread that pulls issues is now a team instead of a developer. That’s it. There is still only one product owner. Everyone in the organization works in the same sprint.

As in Scrum, everything else about the process is a support activity to facilitate humans maintaining that queue data structure. For scrum masters this is usually a combination of presiding over Scrum rituals and malediction of alternative work channels born of earlier software traditions – destruction of Old Ways, like long specification documents or individual users pushing their own priorities on particular developers. Other functions such as managers and sales exist not to direct development tasks but to secure resources from the larger organizational and commercial milieu.

This focus on the product at the heart of the software organization gives a rather existential question great importance: what exactly is the product? Why, exactly, is your organization here? What is the meaning of work life? Vodde and Larman don’t put the questions quite that way, but they do make product definition a key part of LeSS adoption, and strongly favour more expansive definitions based on actual user experience. This may seem obvious on the surface, but is the sort of obvious that is easily lost in the complex interactions of a large established organization. If your business provides sports news to its users, your real product is probably not a statistics vending web service with three “internal clients”. Or at least if you define your team’s product that way, you won’t be as effective at helping your users do something; it will at best be a local optimization. Part of the training was trying to define this product boundary for real systems we worked on, and real organizations. Often the best product definition cuts across senior managerial hierarchies, and this may limit what is the practical product definition can be in the near term. Crossing an organizational boundary usually requires coordination overhead that splits the thread pool of development teams, or introduces multiple product owners, both of which muddle the pattern. LeSS then uses the definition of done to ratchet the team closer to a product delivery mindset in every sprint. You are done only when the user experiences a change in the product.

LeSS mandates feature teams, which are fashionable at the moment, for mostly good reasons. This is again to support the work queue data structure and reinforce a whole-product view across the development teams. The argument is that programmers can learn, and the cost of context switches is worth it for product knowledge, reduced handoff queuing and integration risk, and flexibility of always having developers working on the most valuable features. Bas in particular emphasized learning as at the heart of software development. “When you switch components, velocity will go down, and that’s a good thing!” he said at multiple points during the three days. “It means you’re learning.”

Large Scale Scrum owes much of its success to Bas Vodde’s worldview of constructive organizational nihilism. “You have to give up the idea that things happen for a reason,” he says. “Or because someone decides them.” When I described it this way on morning three of the course Bas laughed, and tilted his head; then someone else asked what nihilism was. Study of nineteenth century philosophy isn’t a focus of a typical software engineering education, though Bas as it happens has squeezed some into his schedule around coding, becoming fluent in Chinese, raising a young family and keeping a day job of ripping internal corporate bureaucracies to pieces in the cause of building better software. He hadn’t seen Scrum and nihilism as linked.

Nihilism is the position that things have no inherent meaning, or, for Bas Vodde, that organizations have no inherent meaning. I call it constructive as I think he likes building useful things, and that’s what the most powerful parts of LeSS are about too. LeSS draws on Lean Engineering in its move away from an idea of best practice to one of continuous learning. In Taylorist scientific management, specialist managers discover a best practice and then encode it as a procedure to be followed by the organization. Nietzsche famously said that God was dead, that we had killed him, and set up Truth in his place, as another lie. Taylorism has the same tendency: it sets up procedures, and a hierarchical organizational tree, as an organizational singular Truth which is received by workers and mediated by a priesthood of managers. LeSS is pretty explicit about wanting to tear that down.

Hello? Is it Truth You're Looking For?

Bas himself, responding to an earlier version of this post, says his position is a little more moderate: that organizations usually have a purpose, but the things they actually do don’t achieve it, or often have any purpose at all, if you look for them.

The LeSS training explicitly frames itself against Ford and Taylor – sections contrasting Taylor with the Toyota way were part of the first day – but the argument is strictly pragmatic. Taylor’s techniques suit a world where large numbers of lightly schooled agricultural workers are coming to work in a factory, in industrial capitalism. It’s not well suited for knowledge workers building software under cognitive capitalism. You don’t need to discard God or Truth in general, both of which engineers can be fond of, to recognize that they aren’t a good model for software development, or other processes of design invention. An idea of a negotiated social truth, the artifacts and understanding of which mutates over time and depend on context, is rather more recognizable. That’s why the agile manifesto says it favours what it does; it is all about rejecting attempts to fixate on a flakily specified organizational high modernist Truth in favour of grappling with messy user reality. (Since the training I found Paulo Virno and the operaismo thinkers already made this link between nihilism and just-in-time production; eg quote and interview.)

Vodde and Larman tear down the idea of a single managerial Truth, then, but they also erect a replacement: the product.

Scrum is a strict process, but productive even in partial use. Ron Jeffries describes this himself when he describes many organizations as not really doing Scrum, and this is echoed by plenty of agile coaches, though they by their role tend to see this as a gap rather than positive variation. Let’s call this use of only some tools from the Scrum toolbox following a Scrum-like process. Scrum isn’t designed to work like this: it’s not a rational unified monstrosity where you are theoretically supposed to start by editing 90% of it out. You may have also added extra guff. Many teams running Scrum-likes may be doing it based on misconceptions, they may be Scrum-but, and they may well benefit from coaching. My guess, from the teams I’ve seen, is that most get some benefit from a short defined iteration (ie a sprint, though an over in Test match cricket might have been a better metaphor). They also benefit from a retrospective and a short daily standup meeting, even if the rest is a bit of a muddle. Crucially, Scrum-like teams also accommodate experiments and localization while keeping a common jargon.

Large Scale Scrum seems to have less room for this variation, because LeSS is Scrum, but scaled up, so each ritual and artifact bears more process weight. It has to do this because much variation will come from organizational inertia and dysfunction that will destroy the queue-worker thread structure at the heart of Scrum.

Scrum, with its focus on a single product owner and a single team, could conceal its essentially political nature pretty well. It was very directly about work that individuals in a small team do, so it could be introduced organically, at a team level. LeSS is about rebuilding your organization around a product, so it’s too big to hide. It is explicitly revolutionary and constitutional in its impact on office politics. For implementations smaller than nine teams, it even prescribes transitioning in a single day, starting a new organizational epoch with a dramatic software process Year Zero. This new Dictatorship of the Product has a politics which are collaborative and decentralizing, but shouldn’t be mistaken for something democratic. The rule of the product turns on the fulcrum of the product owner’s vision. Feature team programmer übermensch escape the shackles of component specialization, spreading their influence across technologies and languages. And of course the waterfall hierarchies of Taylorism were even less democratic and collaborative. LeSS might instead, when working well, have the character of an austere republicanism. There is a separation of powers, there is an executive product owner, and the duty to the Product takes the place of duty to the State (or The People).

Scrum, and Agile before it, build in the idea that the business and technology organizations are separate, and that one serves the other. In many industries that’s simply a recognition of organizational reality. The Scrum community aren’t deaf to it, and they tend to translate it into advice on hunting down, catching and breaking in a product owner. It’s pretty decent advice on the whole, if that’s what you want. In something like a startup, that may not make sense. Perhaps the lead developer has the best idea of the product, so role separation is meaningless. If, like Giles Bowkett, you want Scrum to die in a fire, I’d guess you’d want Large Scale Scrum to die in a fire too; maybe a larger one.

The most passionate argument in the three days of the course – and there were a few good ones – wasn’t about turning org charts upside down, or sidelining managers, or because I called Bas a nihilist, or what a product really is. It was about Jira. Bas recommends using a simple tool for the feature backlog, preferably a spreadsheet, and almost certainly not Jira. The argument, which has something to it, is that Jira is lightweight enough to be used, but heavyweight enough to destroy team ownership and collaborative planning. I have seen this too: it gets worse the more you type in meetings and the more workflow you add.

Say what you will about the tenets of National Socialism, Dude, at least it has traceability.

Say what you will about the tenets of National Socialism, Dude, at least it has traceability.

The Jira users were ready to circle the wagons and have it prised from their cold dead hands. I had forgotten how important the immediate tactility of sticky notes on boards is to a certain type of agilist. “Our ‘process’ is mainly github”, Bowkett says. The toolset for remote collaboration is much richer now, and the assumed developer toolset is a much stronger foundation. (Are you old enough to have had a serious argument about whether to use source control in your career? How about unit tests? With a senior developer?) Both Vodde and Bowkett are right to say tools shape processes. We default to their defaults. It made me realize we had been conflating the feature backlog and the sprint tasks for years, because that’s what’s easy in Jira. We also use Jira to coordinate across multiple cities, sometimes within teams – but I already knew that wasn’t Scrum.

One of the strengths of Scrum is its clarity of definition: you can talk clearly about whether or not you are following it, it doesn’t pretend to fit existing teams without change, and the same is true of LeSS. Vodde and Larman’s insight that making better software in large organizations requires refactoring the organization is spot on. LeSS is an organizational pattern for the product development interface. It’s not the only such pattern. MVC isn’t the only user interface pattern either, but its variants dominate the design space for a reason. If you don’t have the problems of product definition, flexibility and responsiveness in large organizations LeSS was designed to address, and you ship useful features at high quality every week already, don’t use it. For myself, and I suspect many software development organizations, those problems are of daily relevance. Software politics is the art of the possible. A software process coup d’état isn’t possible over here tonight; I’ll probably mangle and cajole a while instead.

Dodgy Steve’s Software Manifestagogo

We have totally nailed better ways of developing software by drinking beer and talking about it. This incontrovergitibly shows you should choose:

Beer over processes and tools
Beer over comprehensive documentation
Beer over contract negotiation
Beer over following a plan.

That is, while there is value in the items on the right, we value the items on the left more.

We also really feel like a kebab.

Industrializing The Noosphere

Control Environment

We are not practicing Continuous Delivery. I know this because Jez Humble has given a very clear test. If you are not checking your code into trunk at least once every day, says Jez, you are not doing Continuous Integration, and that is a prerequisite for Continuous Delivery. I don’t do this; no one on my team does it; no one is likely to do it any time soon, except by an accident of fortuitous timing. Nevertheless, his book is one of the most useful books on software development I have read. We have used it as a playbook for improvement, with the result being highly effective software delivery. Our experience is one small example of an interesting historical process, which I’d like to sketch in somewhat theoretical terms. Software is a psychologically intimate technology. Much as described by Gilbert Simondon’s work on technical objects, building software has evolved from a distractingly abstract high modernist endeavour to something more dynamic, concrete and useful.

The term software was co-opted, in a computing context, around 1953, and had time to grow only into an awkward adolescence before being declared, fifteen years later, as in crisis. Barely had we become aware of software’s existence before we found it to be a frightening, unmanageable thing, upsetting our expected future of faster rockets and neatly ordered suburbs. Many have noted that informational artifacts have storage and manufacturing costs approaching zero. I remember David Carrington, in one of my first university classes on computing, noting that as a consequence of this, software maintenance is fundamentally a misnomer. What we speak of as maintenance in physical artifacts, the replacement of entropy-afflicted parts with equivalents within the same design, is a nonsense in software. The closest analogue might be system administrative activities like bouncing processes and archiving logfiles. What we call (or once called) maintenance is revising our understanding of the problem space.

Software has an elastic fragility to it, pliable, yet prone to the discontinuities and unsympathetic precision of logic. Lacking an intuition of digital inertia, we want to change programs about as often as we change our minds, and get frustrated when we cannot. In their book Continuous Delivery, Humble and Farley say we can change programs like that, or at least be changing live software very many times a day, such that software development is not the bottleneck in product development.

With this approach, we see a rotation and miniaturisation of mid-twentieth century models of software development. The waterfall is turned on its side.

Continue reading