Sunday, 19 February 2012

What is it about evidence?

A recent review of what sounds like a frankly terrible book by yet another climate change denier contained a very good summary of its approach to evidence. Data is most certainly out. 'Ideology-based assertion, simple common sense and the ever trusty anecdote' are in. We are in the world of the Top Gear school of argument.

What is firmly locked out by the Top Gear school is anything approximating to the scientific method. Not just data but any rigorous attempt to understand, to test hypotheses robustly and to drop them when they don't pass muster. 

It's obviously important not to be too po faced about the use of evidence when it comes to decision making in public office (and certainly about applying something akin to formal scientific method). There is always judgement. Everyone comes with baggage. But one might still think that a properly informed understanding would be a good basis for making decisions even if a, perhaps considerable, degree of ambiguity or uncertainty remains.

The approach to 'evidence' is, however, indicative of some wider shifts in the approach to argument and policy making in a democracy and also of the confidence of the decision makers.

The arrival of New Labour in Government in 1997 was accompanied by some trumpeting of 'evidence based policy' as the new mantra. This was largely a defensive stance, part and parcel of a concern to differentiate the new Government from previous administrations which had been dangerously ideological and 'Old'. Bright things, young and old, duly scampered around collecting reams of data and other material as the basis for great new thoughts, chins were stroked and doorstop reports were produced.

At some point all of this rather morphed into what some wags described as 'policy based evidence'; a more selective approach to picking the facts and the evidence that suit and giving rather less emphasis (or indeed just plain ignoring) the bits that were inconvenient. Again that didn't stop the cottage industries whirring away producing the material and doorstop reports.

More recently, politicians seem inclined to go well beyond 'policy based evidence'. When presented with statistical evidence or rigorous, detailed analysis that seriously questions the basis for, or likely consequences of, a policy there is a tendency to dismiss this fairly directly with something along the lines of 'well, you might say that'. As a response this probably ranks some way below 'I hear what you say' .

The interesting thing is why people hold such very different views about the role and force of evidence and indeed what sorts of evidence they see as valid.

Evidence produced through something approximating to the scientific method have come under increasing attack, notably of course in respect of climate change but now more generally.

Two things are notable:

- belief and conviction (some might even call it ideology) are back big time (and are not big on the inconvenient uncertainty which might emerge from looking at the evidence)

- other forms of evidence - personal experience, intuition and particularly anecdote - are increasingly the chosen means for making choices and determining policy.

It's hardly surprising. Notable examples are powerful. They trump statistics and hard evidence every time. They also tend to trump typical examples.

But what is even more worrying is that we are being encouraged rapidly along a spectrum to a point where a significant proportion of the political decision makers and the population as a whole feel that more scientific evidence is problematic in itself. That it is almost inherently dubious. The product of an elite. At odds with a more authentic lived experience and good solid 'common sense'. And peculiarly messy old lived experience actually provides a simpler, truer explanation than any amount of analysis ever can.

The old adage that hard cases make bad laws (see the Dangerous Dogs Act as the example par excellence) seems now to have been reversed. Exceptional cases now seem to make good policy.  And in a populist sense, they often do.

There is, however, also something deeply cultural about this. In the UK that frequently comes down to class one way or another (and class does cross party lines).

Those more insecure about their position may well feel a need for substantial external support which has traditionally tended to come from a more scientific approach to evidence. In short, there is a felt need to demonstrate that a position is correct. 

Those inherently confident about their position are much more likely to take the correctness of their view as a given and be much more concerned to persuade or browbeat others into agreeing. This is where the sort of evidence that we see being deployed nowadays tends to come in; largely anecdotal with a smattering of personal experience and a pinch of intuition. As a witches brew it can be remarkably powerful, not least since it is often about reinforcing prejudices.

So, we still have evidence based policy of a sort but sadly, its the kind of evidence that wouldn't generally stand up to much scrutiny in court.

What though is also interesting is why some experts still seem to hold such power. The most instructive example here is the Institute for Fiscal Studies. At times it seems as if the country (alright, not quite the whole country) holds it's breath waiting for the IFS to pronounce. Yet it's a bunch of experts. But we do on the whole believe what they say. Why? Partly because its complicated, most of us do not profess to understand or to have the time to go through the detail. Partly because it's less susceptible to anecdote. Partly one hopes because they do set out the parameters and the uncertainty and provide a more rounded view that builds confidence. And of course partly, perhaps predominantly for many people, because we no longer trust our politicians on these issues and look for some validation.

This is not an argument in support of technocrats. Robust argument is fundamental to a functioning democracy and that no-one is going to hand their opponents an open goal if they can avoid it.

But it is to say that a political culture in which reasoned evidence often seems to be decried as the product of an intellectual elite is in danger of becoming seriously degraded.

Just look across the Atlantic to see where some of these tendencies can ultimately lead.


  1. I have been thinking about this a lot just lately. I think you have expressed my unease very well. My knowledge is mainly in the field of education, but the anxiety seems present everywhere. Simple minded views of science are lining up against postmodern rejection of technocracy to create what amounts to a storm of rhetorical noise and sloganising. Something important has been dropped somewhere and I don't really understand it. Do you have any readings in mind?

  2. Thank you. I don't have any specific readings in mind but I do think we are seeing the development of a phenomenon which has been gaining increasing traction in the US for the last few decades (hence my final line). A good general account of that can be found in something like 'What's The Matter With Kansas' by Thomas Frank which ties together a number of strands which come together in the Tea Party.

    The development of these ideas in relation to climate change denial would also be worth examining since this is where the strongest and most aggressive antagonism towards scientific evidence is found.

    In the UK what we now see in relation to welfare reform, housing policy, and health policy are good examples of the degree to which the current Government is highly resistant to considering the very well grounded evidence about the likely consequences of their policies. To my mind it is entirely deliberate. They simply don't want to see it since it doesn't fit the theory.

    In relation to education it is interesting that the Swedish free schools and the charter schools in the US are used as evidence. But the extensive literature about the actual effects of both systems is ignored.

    As I suggested this is not entirely new; Governments of all complexions have been guilty. However, the sense that evidence may be more than just inconvenient and actually part of a different way of looking at the world does seem to be me to be new.