Post

The Nobility of Software Development

/ Adam / Leave a comment

Confucians represent one of the earliest groups of knowledge workers in human history. They were trained in specialized knowledge for specialized tasks in a complex society, and they had a sense that their specialized knowledge made them an elite group in society. — Marc Hamann

Software development is both meritocratic and social. In the short and thoughtful note above, Marc Hamann argues this makes the thoughts of Confucius particularly relevant to the endeavour.

Marc well describes the sense of recognition I first felt when reading The Analects, the curiosity and respect for people that underlie this jumble of quotes from Confucius and his disciples. He also describes the disconnect between the brittle hierarchies of Confucianism in its most ossified form and the conservative anti-manifesto at its heart.

He also draws a contrast between the trope of the scruffy, arrogant hacker and the gentlemen 君子 in The Analects. Could we instead imagine a kind of nobility to software development? It would imply a sophisticated engagement and sense of duty to society that is the very opposite of the cliched antisocial developer who cares for nothing but code.

Nobility implies privilege and institutionally supported power as well, of course. But software development already has that. Software is pervasive within most social institutions – civil government, banks, the military, manufacturing, energy production, publishing, movies. It’s embedded in the way we communicate with one another and record our personal histories. And the job of software development is embedded along with it. Citizen programmers have jobs in all these places, vote, own houses, and are thoroughly woven into the social fabric.

The scruffy hacker avant-garde exists, and maybe its a breeder for technical innovation, sometimes. But it’s also a mask for hiding behind, and avoiding responsibility, as argued by historians of technology like Leo Marx.

It’s hard to escape that trope without alternatives. Steve Jobs and Bill Gates provide two: the first not scruffy, rigorous in his focus on usable design, the second for his philanthropic ambition. Their fabulous wealth limits our ability to emulate them. Perhaps someone like Confucius, with his respectfulness and humanism, his thoughts on working as part of a flawed system, can offer another model.

Panopticonarchy

/ Adam / Leave a comment

I learnt about another two panopticons the other day. One is a watch that quantifies everything you do, physically. Treating oneself as big data is a very twenty first century thing to do. It’s the equivalent of a nineteenth century scientist dosing herself with her new experimental vaccine, but now risking nasty civic information death instead of horrible biological death. It also recommends itself as what Jamais Cascio calls a participatory panopticon: it is generated cooperatively by individuals, though once the information is collected it will presumably want to be liberated, and expensive. The prospects for medical advancement and self-empowerment seem rather marvelous; the prospects for medical insurance, whether state or private, seem less so. It all makes my use of runkeeper to track exercise look as wheezing and amateurish as the runs themselves.

The second is the British Library’s endearingly batty plan to store everything on the UK web. An enlightened quirk of British copyright law already gives them the right to do this, and it’s a beautiful, public spirited, idea. This can perhaps be grouped together with various initiatives worldwide to open access to government data, driven from within and without the state itself.

Let’s add to these the leviathans in the room – the security state panopticons. The US and UK ones we know of courtesy of Mr Snowden, the Chinese system has been in the open for some time, and presumably everyone in a uniform globally is at it by now, wrongly or wrongly.

Jeremy Bentham conceived of The Panopticon as a prison where everything a prisoner did could be observed, in order to reform him. That isn’t what we’re building today. We have a system of fledgling panopticons, built by competing interests, used for interacting institutional ends: a Panopticonarchy.

I’d say this system will be hard to completely avoid, and to do so successfully will be the equivalent of today’s statelessness. You will have to be like Hamlet and go live in a nutshell, and most people won’t want to. The choice for most individuals, already, is which panopticons to support and which to pollute, constrain and resist.

Twitter Abuse Solution Sketches

/ Adam / Leave a comment

Rilke used to say that no poet would mind going to gaol, since he would at least have time to explore the treasure house of his memory. In many respects Rilke was a prick. – Clive James

So a particularly nasty bout of threatening, possibly illegal, abuse against Caroline Criado-Perez triggered a petition asking for a report abuse button. Brooke Magnanti counters with examples of how this, and a twitter boycott, may be unproductive; its insightful in itself, and as former Belle Du Jour she does have an interesting angle on pseudonymity and publishing.
So this is society’s pathology, mediated by technology, and because Twitter is pretty neat, mediated in real time and connecting strangers at massive scale. It’s Larry Niven’s Flash Crowd of course, taken to its fastest immaterial instantiation. There are slow hard things to change about human society to make it less awful. The petition is right, though, in that technology got us into this specific version of the problem and there are surely smarter technical things to limit it, but it’s worth noting that right now no-one actually knows what they are. So here’s a few design assumptions and speculations, in the hope it sparks ideas in others.

Parameters / Assumptions
Manual abuse reporting is a deliberate usability choice. It makes you think about the accusation of abuse, and will place a premium on a coherent case. Abuse reporting is judicial and needs due process. It’s probably also rational laziness by Twitter: at small scales this is the cheapest solution to implement.
Adding structure is adding due process, but it’s also institutionalising abuse. At uni, I broke my right arm in a soccer game. I had a lecturer in rationality at the time who noted that soccer had incorporated a whole system of breaking the rules into the game itself, with yellow / red cards. That then motivates the entire diving substructure (pretending to be injured or fouled to get advantage). As in soccer, so in Twitter: all systems will be gamed, especially judicial ones. This effect manifests right down to the amount of structure you put on the report abuse form. Each element narrows the likely scope of human judgement; an abuse form also describes the sort of thing that might be considered abuse.
Human review is needed – with tools that scale. I don’t know any Twitter employees, so this is speculation, but it sounds like it is just reading emails and kicking individuals at this point.
The criminal justice system is needed, and shouldn’t be outsourced to a corporation. This part will be slow. Write to your government, but also keep in mind a certain slowness is a side effect of due process.

Sketch
Use data visualization to analyse abuse events rapidly and at scale. Using new data views to augment human judgement is a digital humanities problem. Require one example tweet in form submission. The abuse support person needs to be able to rapidly see the extent and intensity of the abuse. To facilitate this, when they open an abuse ticket, they should be able to see the offending tweet, the conversation it happened in, and all of the user and reporters twitter network. This consists of followers, people followed, people replied to, people mentioned, people mentioning, people using the same hashtag. They can view much of this as a literal graph. All this can be pre-calculated and shown as soon as the ticket is open without any automated intervention in the tweets themselves. Show ngrams of word frequencies in reported tweets. In the recent example, they aren’t subtle. Allow filtering by time window.
Rank tickets in an automated way and relate to other abuse tickets. The time of the abuse team is limited, but the worst events are flash mobs. Make it easy to see when network-related abuse events are occurring by showing and linking abuse reports in the graph visualization above. Identify cliques implicated in abuse events, in the social and graph-theoretic senses. Probably once an abuse mechanism is established, there will be events where both sides are reporting abuse: make it easy to see that. And yes, show when identified users are involved – but don’t ditch pseudonymity as an account option.
Allow action on a subgraph, slowdowns and freezes. Up until now we have just described readonly tools. Through the same graphical view, identify subgraphs to be acted on. Allow operators to enforce slowdowns in tweeting – the tweet is still sent, but after a number of minutes or hours. The advantage of being able to set say one minute is it will be less obvious investigation is going on. A freeze is a halt on posting until further notice. The operator can choose to freeze or slowdown any dimension of the graph – eg a hashtag, or all people who posted on that tag, or all people in a clique replying to certain users with a certain word. This is similar to a stock exchange trading halt. This has to be a manual action because its based on human judgement and linguistic interpretation. Finally allow account deletion, but not as a mass action.
Capture and export all this data for use by a law enforcement agency you are willing to collaborate with.
Open the API and share at least some of the toolset source so people can get perspective on the shape of an attack when it happens. And of course, don’t do this at once – start with simple read only monitoring and iterate rapidly. Remember that the system will be gamed. Keep the poets out of gaol.

Duckrabbit Bugfeature

/ Adam / Leave a comment
Programmer folk art, origin unknown

Programmer folk art, origin unknown

516. If somebody showed me the figure and asked me ‘What is that?’, I could answer him only that way. –I couldn’t answer: ‘I take that to be a . . .’ or ‘Probably that is a . . .’ Any more than I take letters to be this or that when I’m reading a book.
— Wittgenstein, Philosophical Investigations

Duckrabbit. Source: Wikimedia

Development, Epidemiology, China; Course Reviews

/ Adam / Leave a comment

I’ve taken a couple of these online courses now, and seen my wife take a few more, and its definitely been interesting and worthwhile.

PH207x: Health in Numbers: Quantitative Methods in Clinical & Public Health Research from edX Harvard was the most technical, the most rigorously graded, and generally the most recognizably similar to the undergraduate courses of my science degree. The organizers arranged for all enrolled students to have time-limited copies of Stata, the commercial stats processing package, and work in Stata made up a lot of the assessment. The student body had a lot of expertise to share, either from a medical or mathematical background. The lecturers and TAs were also well engaged, with responses to student questions and discussion being incorporated back into the material.

14.73x The Challenges of Global Poverty was an interesting lecture series by two genuine academic superstars of development economics, Abhijit Banerjee and Esther Duflo at MIT. The course is more or less an extension of their book Poor Economics, which advocates systematic use of randomised controlled trials in economics. It shares lessons from the research for the worlds very poor across issues like food, jobs, risk and access to even simple financial services. It’s a great book, and I wouldn’t have read it so thoroughly, let alone worked through the extra lecture material, without the structure of a course to fit it around. I’m glad I did work through it, as it included things like an excellent walkthrough of the famous Acemoglu et al paper The Comparative Origins of Colonial Development.
There was an electronic copy of Poor Economics available as part of the course, the reader was a little clunky and didn’t work on mobile devices, but given it was all free complaining seemed churlish. A heartening number of the key papers they mentioned are also freely downloadable. Assessment in this course was multi-choice usual based on a fairly strict reading of the course material. This could be frustrating for certain questions; the reputation of two-handed economists has a truth to it, and it could be seen here as hedging in the questions with qualifiers like might, could or should, as well as in the answers with multi-choice. This works in a short answer or essay question, where the student has a chance to explain their reasoning, but not multi-choice.
This is a translation of an existing undergraduate course, but I got the feeling the assessment was dumbed down: if I were paying for an education at MIT I would expect more too. There was engagement from staff but it didn’t seem to be a huge focus. The huge, global and very diverse student body seemed to overwhelm the usability of the edX discussion boards, which had moments of interest but were mostly dominated by typical Internet discussion dross.
Marie Hicks, far from a thoughtless cheerleader of the medium, has suggested MOOCs can be a “new way that we get our research out into the wild, taken seriously, and used as part of larger intellectual, social, and economic debates”. Banerjee and Duflo have seized on that very successfully. Rehabilitating the lecture as a piece of public performance and education is a heartening feature of the early twentieth century, and a course like this gives rather meatier content than the aperitifs at TED.

STSCHINA-001 Science, Technology and Society in China I sits slap across the middle of a swathe of my interests. Naubahar Sharif does a rapid fire tour of philosophy of science and engineering and history of science in China before discussing innovation systems in more detail. A lot of the MOOC discussion is resolutely US-centric, but this is run out of the Hong Kong University of Science and Technology. It’s also a short course format – it was only three weeks long, with two followup courses forming a trilogy about the normal semester length. I found this length easier to fit around other ongoing commitments. There are excellent reasons for regional studies to be pursued outside the focus region, but I do wonder how, eg, the chance to learn about China from teachers in China will change the field.
This course also had the challenge of examining humanities material in an online and massive student body. They chose peer graded short essays, two of which I slapped up on this blog. I thought this was a good compromise given the premises of MOOCs, and good on them for trying a different format. I didn’t get much engaging discussion out of the forums for this course, but the assignment marking process did let me see other angles on the topic. A lot of people seemed to complain about it though – it might have been a double shock to someone from a non English speaking background and used to multi choice or short problem questions.

There’s lots of things to speculate about around online courses, their open spirit, the dot-com venture capital hype and greed, the institutions about to truck crash, the young and not so young postdocs already hammered by the structure of the academic labour market, and I’d like to ruminate on all that too, in another post. From a purely personal and selfish perspective, though, these were courses I couldn’t do locally, on fascinating and important ideas, that pull material together in a way a teacher can and a book usually doesn’t. +1 would experience birth of new pedagogical genre again.