metis

Metis and General Intellect

/ Adam / 1 Comment

“The general intellect” can be interpreted as tacit craft knowledge embedded in individual cunning and social relations. This definition sets general intellect in contrast and opposition to formal information systems. Framing it this way may not be completely true to historic usage, but has some revealing consequences. It applies to either abstract information machines like traditional bureaucracies, or concretized ones, like specific computer programs. Craft and process arguments in software development are also a lens on this transformation of skills in cognitive capitalism, information valorization and their relationship with bureaucracy and flows of stupidity. To expand the general intellect is to accelerate.

The term the general intellect (sometimes general social knowledge) originates from Marx and was used by operaismo thinkers to grapple with the emergence of cognitive capitalism. Virno states:

Whereas money, the “universal equivalent” itself, incarnates in its independent existence the commensurability of products, jobs, and subjects, the general intellect instead stabilizes the analytic premises of every type of practice. Models of social knowledge do not equate the various activities of labor, but rather present themselves as the “immediate forces of production.”

They are not units of measure, but they constitute the immeasurability presupposed by heterogeneous operative possibilities.

They are not “species” existing outside of the “individuals” who belong to them, but axiomatic rules whose validity does not depend on what they represent. Measuring and representing nothing, these technico-scientific codes and paradigms manifest themselves as constructive principles.

From here I suggest general intellect as a form of tacit and social knowledge, of metis, defined in contrast to formal knowledge, numeric or systematic knowledge. The general intellect is cognition but it is not data. It is highly contextualized, by experience, locality and specific social relations. James C. Scott offers the knowledge embodied by a soccer team or a ship’s pilot as good examples of metis. He also uses it in contrast to systematic forms of knowledge used in high modernist, often Fordist projects of top-down political control, state or corporate.

At the economy scale, the general intellect also seems to have an intersection with the idea of “institutions” in economic development. Institutions established ongoing government policies but also more abstract things like property rights and the rule of law; they are not primarily buildings, but persistent social relations, not commoditized or readily transferable between nations. Acemoglu and colleagues found significant correlation between historic settler mortality and modern economic success, putting forward the type of colonial institutions as the causative link.

Though this is oppositional with computation as a form of knowledge, the two are complementary in production. Operators draw on the general intellect to make machines work and produce things. This is true of concrete machines, like a coffee maker, or abstract machines in the Simondon / Deleuze / Guattari sense. And machines, especially large abstract machines, make use of operator black boxes to be effective. Traditional bureaucracy before the advent of modern computers and networks is an example of an abstract information machine which uses human operators as black box components – for instance to persist information to longer term storage, by writing on paper.

If we look at the skilled cognition involved in designing complex machinery, such as computer programming, we find the general intellect in the sense of individual and organizational craft knowledge. The rise of agile software development techniques, emphasising teamwork and craft skills over Fordist or high modernist planning, is one example of this over the last fifteen years. Yet the act of programming depends very much on an individual mental model, as pointed out by Naur. Programming is not typing; the main productive activity in programming is building a coherent mental model, the actual executable code produced is a side-effect. “Programming in this sense must be the programmers’ building up knowledge of a certain kind, knowledge taken to be basically the programmers’ immediate possession” (Naur). The spread of algorithms and software throughout society would then suggest a shattering of the general intellect into millions of shards of specific intellects. The general intellect – the entirety of system relations – could decay even as systemic shards expand in sophistication.

None of this is deny a certain translatability from metis to formalized knowledge. It can all be boiled down to bits in the end. Translation for functional use is costly and lossy, though. The mechanics of deep learning parallel this metic transformation from formal data structures into occluded knowledge. To understand how a deep learning system internally navigates a problem space requires a separate systematic analysis alien to the learning mechanism of the algorithm. Deep learning is a localized preview of machinic metis.

A countertrend to the fragmentation of general intellect might be the success of open source, but the point of open source is precisely to make the executable details of machines more readily available through social processes. It is a common platform rather than a general intellect, where evolution of the platform happens through patches (explicit formal communication) rather than primarily through evolved social understanding, though those dynamics still exist. It is striking that open source communities are organized primarily around a specific machine or platform rather than user products. This is true from the GNU C compiler through to the Apache web server and git source control. They echo Simondon’s critique of objects made in capitalism not evolving but merely accumulating features. Simondon’s comments on technical culture also parallel general social knowledge:

Now that he is a technical being no longer, man is forced to find for himself a position in the technical ensemble that is something other than the position of individual.

The trend for computer programming to promote skilling up for designers but sometimes exporting deskilling elsewhere was noted by Mackenzie in 1984; it’s because capitalism is a valorizing process rather than a deskilling process per se. Likewise there are deskilling trends in the software industry around outsourcing highly specific work to remote or offshore teams, so long as it promotes valorization (increases shareholder value).

The frustrations of working in or with a bureaucracy are often those of being a black box cog in a larger abstract machine, either through alienation from the meaning of the work, or because the work actually causes an undesirable effect which conflicts with personal goals, or even the stated goal of the organization itself. That is a form of stupidity but relates to all bureaucracy. Deleuze and Guattari say in capitalism:

The apparatus of antiproduction is no longer a transcendent instance that opposes production, limits it, or checks it; on the contrary, it insinuates itself everywhere in the productive machine and becomes firmly wedded to it in order to regulate its productivity and realize surplus value which explains, for example, the difference between the despotic bureaucracy and the capitalist bureaucracy.

eg in a despotic state the army may come and confiscate food and labour from a subsistence farmer when fighting a war, but in capitalism this military anti-production is in the form of a military-industrial complex, production interleaved with anti-production. Yet those critiques could apply to Soviet socialism too; only capitalism manages to create demand and ensure lack in the midst of overabundance.

Deleuze takes stupidity as an inability to dissociate from presuppositions, sense rather than common sense. Contemplating flows of stupidity, I am reminded of the slogan of engineer Jesse Robbins for making useful things in corporate bureaucracies: “Don’t fight stupid, make more awesome”. This could also serve as an accelerationist slogan, and can be critiqued the same way. Are you pushing forward as an elite ignoring politically hard problems, or are you building a concrete alternative solution that will attract change? Are you wasting time trying to change a system accelerated beyond human comprehension, or are you using accelerated human components to build a new system?

References

Acemoglu et al – The Colonial Origins of Comparative Development
Beck et al – Agile Software Development Manifesto
Deleuze and Guattari – The Anti-Oedipus
Garton – Excavating the Origins of Accelerationism
Land – A Quick and Dirty Introduction to Accelerationism
Mackenzie – Marx and the Machine
Naur – Programming As Theory Building
Scott – Seeing Like A State
Simondon – On The Mode of Existence of Technical Objects
Virno – Virtuosity and Revolution: The Political Theory of Exodus, in Radical Thought In Italy
Williams / Srnicek – #accelerate Manifesto for an Accelerationist Politics

VII.1 Reuse

/ Adam / Leave a comment

子日,述而不作,信而好古,窃比于我老彭. – 论语 七:一

The Master said, ‘I transmit but do not innovate; I am truthful in what I say and devoted to antiquity. I venture to compare myself to your Old P’eng.’ – Analects VII.1 (Lau)

Before contemplating the process implications of this radically static statement, let’s note that from the perspective of designed code itself, it is always true. Code transforms and transmits information. This is the garbage-in garbage-out principle. Designed code (not genetic or evolved code) does not innovate.

Backups of the user directory for the Analects’ source control repository are, alas, lost to antiquity, and though many sophisticated data recovery techniques have been tried, with some success, none have yielded the identity of Old P’eng. Our ignorance of him highlights our relationship with Confucius and with any classical tradition. To us, Confucius founded a philosophical school, but in his own words he merely continued a tradition that we can see indirectly, if at all.

Scholarly consensus is that Confucius is deliberately overstating his lack of innovation for reasons of rhetoric or modesty (see eg DC Lau, AC Graham, or just wikipedia on this verse). Nevertheless the verse is considered pivotal in understanding Confucius’ traditionalism and conservatism in a time of extraordinary violence and social change.

Existing solutions are useful in at least two ways. 

Firstly they may capture unintuitive theoretical results in accessible ways. Many algorithm design and data structure results are now in this category, such as sorting algorithms and efficient concurrent maps (eg the Java 6 lock free implementation of java.util.ConcurrentHashMap). The formal scientific characterization of such solutions in terms of, say, algorithmic complexity and performance benchmarks  make computer theoretic literacy crucial. Programmers will be unlikely to understand the derivation by reading the code, so they must be able to read the documentation. 

Secondly they may capture highly specific details of the environment and robust solutions to managing it. This will include successful workarounds for under-specified elements of protocols, or flat-out incorrect but popular implementations. Any user of say Ruby on Rails or Tomcat takes advantage of this kind of reuse. Consider too the domain specific details and tolerances of a fly-by-wire control system for a particular make and model of plane.

These two kinds of reuse may be contrived to lie on a spectrum, but I’ve chosen to distinguish them here for their correspondence to two different categories of knowledge – logos and metis. In classical Greek epistemology logos is theoretic universal knowledge and metis is hard won cunning, “feel”, or craft knowledge (as an aspect of techne, craft knowledge and theory). James C. Scott describes the Greek hero Odysseus, surgeons and maritime pilots as all relying on metis (Seeing Like A State). Scott also makes the connection between traditional knowledge – which is particular and tied to a society and geography – and common law conservatism in the tradition of Edmund Burke and Michael Oakeshott.

Confucius is claimed as a kind of Burkean conservative, for instance, by James Kalb. Both Confucius and Burke grew up in societies with small literate elites and large impoverished peasantries. They both share senses of the worth of settled convention, the importance of teaching and the literary canon, a paternalistic affection for heredity power, and a sympathy for the welfare of everyday people.  Neither are they reactionaries, but welcome improvement at a humane pace (IX.3).

Seeing Burke and Confucius as similar is not mainstream and deserves a dedicated analysis of its own. (My searches revealed more extant work linking both of them individually to Wittgenstein than to each other, but pointers are always welcome.) In a comprehensive entry for Burke in the Stanford Encyclopedia of Philosophy, Ian Harris argues that despite being more often claimed by the right wing, he does not have a clear modern partisan successor. Nevertheless, distinguished scholars like DC Lau or AC Graham stay well clear of Western political comparisons, while happily comparing classical Chinese figures with Western philosophers. 

Unusually, a software library, and all the hard won craft knowledge that comes with it, can be imported into another with extraordinary ease when compared to other forms of craft knowledge. A pilot is of little advantage outside his home port, and Ruby on Rails is of little use for 3D rendering, but in software we can copy the pilot and use him on innumerable ships entering that port. We can also ultimately read the source code to Ruby on Rails and determine how it tolerates the idiosyncrasies of particular browsers and servers. This is because all code is built on a formal information substrate – the computational medium. (This is Harrison Ainsworth’s term and his note on reuse provided a number of the connections in this post.)

Not all craft knowledge of a codebase is encapsulated in the codebase. There are particularities of the install, workaround scripts, configuration, scheduled jobs and so on, but these are ultimately digital artifacts easily included within a slightly broader view of what a codebase is (this latter is a premise of DevOps and for anyone serious about a controlled environment). More problematically, there are conventions of use, design choices, oral traditions of “check here when you change there”, and so on. At the limit, all codebases are incomplete. They depend on co-texts, results and knowledge of the domain that need not be encoded. An air traffic control system does not need a textbook description of Bernoulli’s Principle.

Burke and Scott argue that in an established society important, non-obvious, traditional knowledge is captured in social conventions and established practice, and the practice cannot be simplified without a loss of valuable situational knowledge. Scott additionally points out that such an environment is very difficult for an outsider to navigate and there are strong motivations for central political power to apply simplifications to it.

Yet highly particular, ‘local’ code that requires hands on experience and knowledge of accompanying conventions most frequently has another name in software development: bad. Or: spaghetti. Or: legacy. The sentiment is well captured in Qi’s koan on fear, even if it does riff off an opposing classical Chinese tradition. (In Confucian terms we might note the building is not harmonious.)

In No Silver Bullet, Brooks distinguishes accidental and inherent complexity, with the latter being an attribute of the underlying problem rather than any specific software or hardware implementation. Complexity due to poor or improvable design is always accidental; that due to the problem domain is by definition inherent.

An aesthetic sense of good or poor design becomes crucial when pursuing aggressive reuse (VII.14). Without it you will simply perpetuate junk.

Having argued the link between conservatism and software reuse, it is worth being a little more precise about flavours of conservatism. William F. Buckley famously described it as that which “stands athwart history, yelling Stop, at a time when no one is inclined to do so, or to have much patience with those who so urge it.” Despite its partisan origins, this is a good start, as it illustrates certain threads of environmentalism and the idea of heritage listing fall easily under the same banner. ((It is also useful to think of contemporary US Democrats defending Franklin D. Roosevelt’s New Deal, or opposition to changes to Britain’s NHS in this frame.))

In its purest form, this can be “return to a golden age” conservatism. There’s certainly an argument that Confucius would have been happy with a reversion to the society of the Eastern Zhou. We should again temper our interpretation by wondering how much is rhetoric covering adaptation of tradition to new times. In software, certainly, simply reactionary approaches are of little use. Brooks and the founders of eXtreme Programming have both noted that a more effective strategy is to embrace change. Oakeshott argues in On Being Conservative that settings of widespread and enthusiastic change are in particular need of an awareness of the value of what exists now. A traditionalist most often defends the present versus the future, not the past versus the present. This conservative disposition’s usefulness to software is more apparent if taken as an analytic tool rather than an inherent aspect of personality. After all, the greenfield doesn’t exist (see X.18), and any project that pretends to be a greenfield is an interesting lie.

Conservative thought in this vein usually emphasizes working within a tradition and a community – in software we would say platform. This also suggests interesting contours for the breadth of possible reuse; and there are other verses, such as XVI.11, where that might be explored. What is immediately apparent is the narrowness and fragility of an entirely in-house platform due to the smallness of its developer community; and the need for a shared jargon (XIII.3) and perhaps a canon (XVI.13).

Given the corpus of extant code in the form of libraries, to adore antiquity is to know your platform, including its innards, not just thoughtless rote quoting via copy and paste. At this moment in software, to reuse and extend is a greater service than extraneous self-involvement masquerading as innovation.

If you can easily find some code and copy it, you get the result at zero cost. That is an efficiency that cannot be beaten: no amount of programming tool and technique improvements can ever do that. So we want to maximise reuse. – Fred Brooks, No Silver Bullet