Democracy, Short-Termism, and Outside Context Problems

An Outside Context Problem was the sort of thing most civilisations encountered just once, and which they tended to encounter rather in the same way a sentence encountered a full stop.

Iain M Banks, Excession

We’re used, by now, to the stories.

Story 1: Democracy is built in a way that means that it’s too short-termist in its thinking, incapable of taking slow-burn projects or problems seriously, happy to accept unknowns and uncertainties as long as this administration doesn’t have to deal with them. Tethered solely to the populist demands of election and re-election seeking, our representatives lack incentives to engage with the problems that are starting now but could become hugely impactful a little further down the road.

Story 2: Blinded by self-interest, or just our inability to imagine influences from beyond our usual sphere of experience, we fail to foresee the huge exogenous issue until it’s too late. It’s the scenario that we see occasionally in the movies – A big spaceship turning up in Independence Day, a rogue planet looming in the sky in When Worlds Collide, the Spanish boats bringing Catholicism, plague and cannon to the new world in Apocalypto.

Apocalypto (2006)

What connects these stories? Both of them entail the claims about the dangers of our limited epistemic perspective. Story 1’s short-termism is marked by our failure to comprehend the future implications of emerging problems (or our failure to care if the likely implications of such problems will be some future generation’s headache). In the end, it boils down to our response to risks, our comfort with unknowns and unintended consequences when their impact will be felt by people other than ourselves.

Story 2 removes the possibility of present-day intervention from the equation. Our ignorance is total, because the universe is serving up a curveball. While in story 1 we are ignorant of the ramifications of current trends (and lack incentives to invest our time and resources in evaluating their risks and responding sensibly to them), in story 2 we have had no opportunity to predict or prepare for the problems at hand. Such scenarios loom large in our cultural consciousness, as a trip to a cinema or book shop can demonstrate. They are Rumsfeld’s unknown unknowns, Taleb’s black swans, Banks’ outside context problems.

Democracies are generally thought of as being especially vulnerable to problems like these. They may actually magnify human short-termism. There’s something natural about short-term thinking – human beings are lucky if they even get close to living for 100 years, and it may be relatively unreasonable to expect them to give a stuff about the possible state of affairs in hundreds of years’ time. But democracies don’t operate by human generations – they operate by political generations, terms of office. They shorten the scope of possible human foresight and long-termism even further than our biology does, so that issues spanning political generations, and not only actual generations, become someone else’s problem.

With random events, or events that cannot be anticipated, the emphasis shifts to the survivability of democracy. David Runciman views the robustness of democracy as a kind of puzzle, and a potentially dangerous one: by successfully surviving challenges, it has established a “confidence trap” which causes us to think of it as safer and more stable than it really is. When the big, unforeseeable challenges of the future come knocking, he argues, democracy will be in real trouble – we will be clinging to a form of social order that we imagine to be the most robust imaginable, but which cannot possibly protect us from a real shock to its system.

Discussions of democracy’s failings in these regards are often quite problematic. See, for example, Jørgen Randers’ recent article on short-termism and climate change for Democratic Audit UK. Norway’s popular rejection, in 2006, of a plan to cut greenhouse gas emissions by two thirds by raising income taxes from 36% to 37% seems to Randers to be an extraordinary error:

In my mind, the cost was ridiculously low … given that this plan would eliminate the most serious threat to the rich world in this century. In spite of this, a vast majority of Norwegians was against this sacrifice. To be frank, most voters preferred to use the money for other causes – like yet another weekend trip to London (or Sweden) for shopping.

But there are some significant errors in reasoning here. For one thing, Norwegian voters may have been sceptical that such a plan could work as well as advertised, making the increased taxes less appealing. More importantly, Norway’s domestic action on green house gas emissions would not, as Randers states, “eliminate the most serious threat to the rich world in this century”, but rather represent a tiny part of all of the greenhouse gases emitted by human activity on the global scale. Voters are not always rational, but there is nothing particularly selfish or malign about Norway’s failure to accept this trade-off. Randers is eventually moved to appeal to “common sense”, “income security” so that we are less averse to the risks of significant structural change, and, finally:

[T]o install enlightened dictatorship for a time limited period in critical policy areas. Like the Romans did when the city was challenged. And which is the solution currently pursued by the Chinese Communist Party, with obvious success in the poverty/energy/climate area. But I agree that the obvious solution of strong government appears unrealistic in the democratic West.

The fact that this “obvious solution” is unlikely to be implemented speaks to the possibility that desperate times call for desperate measures: like Runciman, he is arguing that there are certain situations where democracy is not up to the job, and something (or someone) needs to step in.

With this in mind, I would like to make three further points of my own here:

  1. The line between short-termism and outside context problems is more blurred than we usually imagine
  2. Short termism is often justified by the positing of beneficial unforeseen consequences, or outside context benefits
  3. There’s no form of social order that is likely to outperform democracy in dealing with either long-term problems or outside context problems

1. Short-termism revolves around knowledge problems, just as outside context problems does. If we knew with real certainty that a given current trend is going to turn into an existential threat, it is likely that even a short-term political establishment would find means to address or mitigate it. Such certainty is hard to come by, because the future is, by definition, unknowable.

2. These two types of problems intersect in another important way. One of the reasons we might doubt that a current process will turn into an existential threat is our faith in human and technological innovation and progress. In this way, outside context benefits – the good things we cannot yet imagine – are often imagined to dwarf the threats of current trends.

3. Why do we imagine that democracy is worse than other types of social order for dealing with big problems? This assumption – that democracy goes hand-in-hand with complacency and softness – has been on the agenda since classical civilisation. But anyone who argues, like Randers, that some kind of dictatorship will outperform a democracy in confronting a big problem like climate change is making extraordinary assumptions about the way that knowledge is distributed in society. The possibility of an epistocracy – rule by the knowers – revolves around the idea that there really is some subset of a population that knows better than the rest. While it may be plausible that such a subset exists, it is deeply implausible that we are capable of recognising it when we see it. Willingness to act in the long-term interests of a species does not necessarily imply that we understand what those interests really are. Democracies already take steps to ensure longer-term institutional memory and agency – this is why the UK civil service stays much the same year by year, while different administrations come and go.

And when an outside context problem comes knocking, there are good reasons to expect a democracy to be more robust than other kinds of social orders – as shall be discussed in future posts. But a big, unexpected problem will have little respect for the long-termism of a benevolent dictator who is necessarily bound by the same epistemic limitations as the rest of us.

The usual example given to illustrate an Outside Context Problem was imagining you were a tribe on a largish, fertile island; you’d tamed the land, invented the wheel or writing or whatever, the neighbours were cooperative or enslaved but at any rate peaceful and you were busy raising temples to yourself with all the excess productive capacity you had, you were in a position of near-absolute power and control which your hallowed ancestors could hardly have dreamed of and the whole situation was just running along nicely like a canoe on wet grass… when suddenly this bristling lump of iron appears sailless and trailing steam in the bay and these guys carrying long funny-looking sticks come ashore and announce you’ve just been discovered, you’re all subjects of the Emperor now, he’s keen on presents called tax and these bright-eyed holy men would like a word with your priests.

Iain M Banks, Excession

Hayek and the Left

This is the draft of a piece whose definitive, edited version has been published on the LSE British Politics and Policy blog as Friedrich Hayek and the Left: A Response to Simon Griffiths.

Friedrich Hayek’s ideas, once inalienably associated with the particular brand of new-right trickle-down economics espoused by Thatcher and Reagan, have for some time been building up a degree of broader appeal. After all, Hayek’s analysis and reasoning are highly persuasive, even if one’s ideological or normative conclusions tend in other directions. Simon Griffiths’ recent post on the LSE’s Politics and Policy Blog summarises some of the elements of Hayekian thought that might be put to use by the political left. But to what extent can Hayek’s analysis be divorced from his conclusions? Can the left really learn from Hayek’s thought without being persuaded by it?

Giving a flock: does the left get something out of the idea of spontaneous order?

Griffiths writes that three key Hayekian ideas might be engaged-with by the left: knowledge distribution, spontaneous orders and the significance of individual liberty. While all of these overlap in significant ways, the first two – Hayek’s epistemology and theory of catallaxy – are deeply integrated with each other, and are probably best dealt with together.

(Some of) Hayek in a Nutshell

Hayek articulated an epistemology that was marked by thoroughgoing scepticism and fallibilism. Most people, by his account, lack the opportunities and the the incentives to attain anything approximating expertise in a given area, let alone the holistic expertise that might be required for good governance, effective political reform or centralised control of an economy. Rather, knowledge is highly localised, contextual, and widely distributed throughout a given population.

Of the various systematic ways of aggregating the widely-distributed ‘bits’ of information in a society, Hayek viewed markets as being the most effective, due to their capacity for non-coercively producing a unified price signal – a ‘marvellous’ package of information based on the cumulative effect of small-scale individual decisions of buying and selling. Because this process needs no particular centralised orchestration, no coercive or authoritarian influence for its implementation, and approximates resource-efficiency without any human design at the macro scale, it could best be understood as an emergent, spontaneous order or catallaxy.

From here, Hayek develops a whole worldview, where three types of order (designed, evolutionary and catallactic) are weighed against each other. There’s a lot to write about here, and many misconceptions persist in the interpretation of Hayek’s work (lots of people conflate catallaxy with evolution, for example). There are also questions to be raised about the realism, the pragmatism and the normative desirability of spontaneous orders in real politics (some of which will be raised in future posts on this blog). But, for now,are there leftist conclusions that can be drawn from Hayekian analysis?

A Response to Griffiths

Griffiths writes:

Understood socially, knowledge can be shared by people taking action to overcome the limits of their individual perspectives. … Hayek’s argument over the dispersed nature of knowledge can be used to support a form of radical social movement politics, not simply a scepticism about socialist planning.

It’s certainly right that a critique of the epistemic limitations of central bureaucracy does not, by itself, add up to a one-sided argument for the priority of markets. However, the “radical” grass-roots politics described here would necessarily be engaging with highly specific, contextual problems, and would necessarily lack the knowledge required to safely take action at larger scales, or to wholly understand the range of possible consequences even at a local level. If Hayek is right, this agency limitation effect would effectively mean that the likelihood of messing up – that is, disturbing or distorting the spontaneous order – would increase in direct correlation with the scale and significance of the political movement (or problem) in question. For Hayek, the conservatism of his response wasn’t optional, but a rationally risk-averse response to the fact of limited human knowledge.

Griffiths moves his focus to the kinds of state action that might be most beneficial given the reality of spontaneous orders:

[T]hose at the centre could set the framework and help point institutions in particular directions. For Gamble, the idea of spontaneous order results in the state taking a more experimental approach, using trial and error to establish new types of organisation that disperse power as widely as possible in order to make the most efficient use of the knowledge that these orders contain. Cultivating spontaneous orders can be as important in the growth of a good society as building new ones.

Is the conclusion here a realistic one? At first blush, the idea of “cultivating spontaneous orders” seems to be a contradiction in terms. It is unlikely, within Hayek’s epistemological world, that such an effort at cultivation will do much more than disrupt existing spontaneous orders. Certainly we can dispute Hayek’s epistemology – but if we are willing to do this, then why particularly value spontaneous orders in any case?

The experimentalism described by Andrew Gamble may also prove to be problematic. Hayek seems to have been in broad agreement with Karl Popper on the point that societies are not functionally laboratories, and so the potentially dire social ramifications of an experimental process would make ‘piecemeal social engineering’ the only responsible course of action.

Liberty and the Left

However, the third Hayekian insight brought up by Griffiths – his emphasis on freedom – may be more fruitful for the left. After all, freedom is an essentially contested concept, and Hayek’s insistence on the importance of individual liberty need not be wholly incompatible with the various underpinning functions of the state, and the careful design of policies that are designed to maximise human flourishing in the difficult trade-off between positive and negative liberty. The rest of Hayek’s work serves as a warning about unintended consequences that should be heeded by left and right alike, necessarily placing limits on the reach of such policies. But Hayek’s is not the only legitimate notion of freedom, just as spontaneous orders are not necessarily the only producers of desirable outcomes in the social world.

Where this debate leaves the UK’s parties of the left is anybody’s guess. Beyond peripheral and academic interest, the name Friedrich Hayek is likely to remain toxic for the Labour Party – and, indeed, for the modernised Conservative Party – for years to come.

– Simon Kaye

Political Ignorance and the British Public

This is the draft of a piece whose definitive, considerably better final version has been published on the LSE British Politics and Policy blog as On the Complex Relationship Between Political Ignorance and Democracy.

Public ignorance is a central consideration in my academic work, which encourages the idea that politics, at least in part, is best understood as throwing up considerable ‘knowledge problems’; problems that are pervasive at every level of participation and administration in contemporary democratic states. The justification of democracy must, if it is to have meaning, take the political ignorance of democratic citizens seriously.


The media occasionally pick up on and discuss the depth and significance of the most obvious kind of political ignorance – that displayed by would-be voters. Naturally enough, these concerns can make for attention-grabbing headlines as our minds are concentrated by the rapid approach of a general election. The Independent‘s buzzfeed-imitating ‘i100’ site recently pointed out some recently-accumulated evidence:

46% of respondents were unaware that a general election is taking place this year

57% of respondents were unaware of what a general election is

59% of respondents could not name the current British Prime Minister

How trustworthy is this evidence? Well – not very trustworthy at all, methodologically. As i100 noted, the 1900-odd respondents were an unweighted sample, quizzed via email by a voucher codes website. We’re learning a lot about the political ignorance of the savings-savvy with a bit of time on their hands, perhaps, but should be wary of extrapolating such information into a generalised indictment of the state of public political knowledge.

But are these statistics outliers? Do they fail to correspond with existing (more sound) empirics on similar questions? Would an expert on public opinion be surprised by these findings?

Absolutely not. While most research around public ignorances arises from surveys conducted in the USA, there’s a growing body of work concerned specifically with the UK context as well, and the majority of such evidence confirms the realism of the statistics set out above.

For example, Denver & Hands have found that 77% of British 18-year-olds cannot reliably differentiate between the policy platforms of mainstream political parties. When the survey is limited to those who are students or former students in politics, this figure only improves to 50%.

More recently, a survey for the Royal Statistical Society and KCL provided a widely-reported embarrassment of riches in this area, including:

  • the general belief that under-age pregnancies are  25x more common than they actually are
  • the over-estimation of the severity of benefit fraud by 24x
  • 29% of those surveyed thought that Jobseekers’ Allowance is more expensive to the taxpayer than pensions, when the latter costs about 15x more
  • The generalised belief that crime and violent crime rates were rising, when in fact they have been falling for decades
  • The systematic over-estimation of immigration rates

These findings are particularly interesting in that they seem to indicate a kind of pessimistic bias in British political ignorance. Is it possible that many of these areas imagine things to be worse than they actually are because of the media and cultural landscape in which we find ourselves? In a more balanced informational environment, would we cease to be wrong, or simply cease to be wrong in a systematically pessimistic way?

For now, we can at least say that there’s no need to turn to i100’s dodgy sources if we want to get a sense of the ‘knowledge problems’ at work in British democracy – but we needn’t dismiss those findings as unrealistic, either.

The next, fascinating challenge is to offer meaningful and relevant political theory in response to such evidence. Cynicism – or, at least, simplistic cynicism – need not be the only response available to us. Within the British context, the extent of public ignorance may even have the surprising effect of helping us to make peace with the various systemic and electoral features of our political establishment that make the public far less influential than they might imagine or prefer themselves to be. After all, what’s the value of a more responsive electoral system – of more democracy – if the wider citizenry is incapable of voting well?

My own research strikes a more positive note than this, and tries to offer a more encouraging justification of democracy. More on that in future posts.

Just as most empirical work on public ignorance comes from the USA, so too does most of the theoretical consideration of its implications. To those interested in reading a critical perspective on democracy due to the extent of political ignorance, I’d say that this (plentifully controversial) book by Ilya Somin is a really great starting point.