Tuesday, 28 February 2012

More evidence

Further to a recent post about evidence, a very interesting piece in The Guardian today -  http://tinyurl.com/6ovqnz2 - about significant new interest by Labour in upping the role of evidence in relation to education. Just how far this would have put a serious dent in recent political initiatives which have been highly selective in their use of evidence in terms of school organisation, the approach to teaching and the curriculum is of course moot.

Much would depend on how far such an Office for Educational Improvement would have teeth to stop or seriously embarrass Ministers; whether reports would be public as a matter of course and prior to decisions being reached; how independent it would be in terms of initiating research as opposed to testing proposals; how well funded it would be; what the terms of reference would be in relation to the tests to be applied particularly in terms of gauging effectiveness and how its relationship with Parliamentary scrutiny would be structured. 

At the level of teachers and schools there would also be a significant dynamic about the approach from such an Office in terms of initiating and promoting good practice or in hobbling innovation. Some fleet footwork would be required to ensure that the Office is seen as advancing discussion and change, not hobbling it.

But leaving all such questions to one side, it is surely significant that there is interest in greater transparency about evidence. Amidst all the sound and nonsense about various forms of transparency in recent years, putting an emphasis on how it could apply to the basis for policies rather than simply on how much is being spent on penny packets of services can only be a good thing.

Sunday, 19 February 2012

Incidentally ..

The recent ruling in relation to the legality of saying prayers as part of a formal council meeting has generated a great deal of heat and rather less light. The judge simply found that the council did not have the necessary powers either directly or indirectly (in a refrain familiar to those working in or with councils it was not possible to find support for the prayers as 'incidental to the incidental').  There is a suitably droll passage in the judgement which says that:

'it is not for a Court to rule upon the likelihood of divine, and presumptively beneficial, guidance being available or the effectiveness of Christian public prayer in obtaining it'.


Now the Government has stepped in to apparently provide a 'power to pray' through the general power of competence in the Localism Act.

If nothing else this will mean that many more people are aware of the new power than would otherwise have been the case.

Two further observations are, however, worth making:

- firstly, it is not absolutely certain that the power of general competence will necessarily provide this apparently much sought after deliverance. The new power is subject to public law constraints and simply asserting that an individual has the ability to decide to pray may not provide complete justification for a council deciding to do so. Indeed there should be a strong case in terms of maintaining the secular approach of neutrality in the public sphere to say that operating in a civic capacity is indeed quite distinct from operating in an individual one;

- secondly, it is as a minimum rather amusing that the Government trumpets the new power as allowing for the saying of prayers at the start of formal council business whilst explicitly denying on the face of the Act the ability for councils to organise their own decision making procedures in ways other than those that are prescribed by legislation or secure the approval of the Secretary of State. 

One doesn't entirely anticipate that changing any time soon or indeed there being anything like the amount of sound and fury about it as there has been about saying prayers in a formal meeting rather than outside it.

What is it about evidence?

A recent review of what sounds like a frankly terrible book by yet another climate change denier contained a very good summary of its approach to evidence. Data is most certainly out. 'Ideology-based assertion, simple common sense and the ever trusty anecdote' are in. We are in the world of the Top Gear school of argument.

What is firmly locked out by the Top Gear school is anything approximating to the scientific method. Not just data but any rigorous attempt to understand, to test hypotheses robustly and to drop them when they don't pass muster. 

It's obviously important not to be too po faced about the use of evidence when it comes to decision making in public office (and certainly about applying something akin to formal scientific method). There is always judgement. Everyone comes with baggage. But one might still think that a properly informed understanding would be a good basis for making decisions even if a, perhaps considerable, degree of ambiguity or uncertainty remains.

The approach to 'evidence' is, however, indicative of some wider shifts in the approach to argument and policy making in a democracy and also of the confidence of the decision makers.

The arrival of New Labour in Government in 1997 was accompanied by some trumpeting of 'evidence based policy' as the new mantra. This was largely a defensive stance, part and parcel of a concern to differentiate the new Government from previous administrations which had been dangerously ideological and 'Old'. Bright things, young and old, duly scampered around collecting reams of data and other material as the basis for great new thoughts, chins were stroked and doorstop reports were produced.

At some point all of this rather morphed into what some wags described as 'policy based evidence'; a more selective approach to picking the facts and the evidence that suit and giving rather less emphasis (or indeed just plain ignoring) the bits that were inconvenient. Again that didn't stop the cottage industries whirring away producing the material and doorstop reports.

More recently, politicians seem inclined to go well beyond 'policy based evidence'. When presented with statistical evidence or rigorous, detailed analysis that seriously questions the basis for, or likely consequences of, a policy there is a tendency to dismiss this fairly directly with something along the lines of 'well, you might say that'. As a response this probably ranks some way below 'I hear what you say' .

The interesting thing is why people hold such very different views about the role and force of evidence and indeed what sorts of evidence they see as valid.

Evidence produced through something approximating to the scientific method have come under increasing attack, notably of course in respect of climate change but now more generally.

Two things are notable:

- belief and conviction (some might even call it ideology) are back big time (and are not big on the inconvenient uncertainty which might emerge from looking at the evidence)

- other forms of evidence - personal experience, intuition and particularly anecdote - are increasingly the chosen means for making choices and determining policy.

It's hardly surprising. Notable examples are powerful. They trump statistics and hard evidence every time. They also tend to trump typical examples.

But what is even more worrying is that we are being encouraged rapidly along a spectrum to a point where a significant proportion of the political decision makers and the population as a whole feel that more scientific evidence is problematic in itself. That it is almost inherently dubious. The product of an elite. At odds with a more authentic lived experience and good solid 'common sense'. And peculiarly messy old lived experience actually provides a simpler, truer explanation than any amount of analysis ever can.

The old adage that hard cases make bad laws (see the Dangerous Dogs Act as the example par excellence) seems now to have been reversed. Exceptional cases now seem to make good policy.  And in a populist sense, they often do.

There is, however, also something deeply cultural about this. In the UK that frequently comes down to class one way or another (and class does cross party lines).

Those more insecure about their position may well feel a need for substantial external support which has traditionally tended to come from a more scientific approach to evidence. In short, there is a felt need to demonstrate that a position is correct. 

Those inherently confident about their position are much more likely to take the correctness of their view as a given and be much more concerned to persuade or browbeat others into agreeing. This is where the sort of evidence that we see being deployed nowadays tends to come in; largely anecdotal with a smattering of personal experience and a pinch of intuition. As a witches brew it can be remarkably powerful, not least since it is often about reinforcing prejudices.

So, we still have evidence based policy of a sort but sadly, its the kind of evidence that wouldn't generally stand up to much scrutiny in court.

What though is also interesting is why some experts still seem to hold such power. The most instructive example here is the Institute for Fiscal Studies. At times it seems as if the country (alright, not quite the whole country) holds it's breath waiting for the IFS to pronounce. Yet it's a bunch of experts. But we do on the whole believe what they say. Why? Partly because its complicated, most of us do not profess to understand or to have the time to go through the detail. Partly because it's less susceptible to anecdote. Partly one hopes because they do set out the parameters and the uncertainty and provide a more rounded view that builds confidence. And of course partly, perhaps predominantly for many people, because we no longer trust our politicians on these issues and look for some validation.

This is not an argument in support of technocrats. Robust argument is fundamental to a functioning democracy and that no-one is going to hand their opponents an open goal if they can avoid it.

But it is to say that a political culture in which reasoned evidence often seems to be decried as the product of an intellectual elite is in danger of becoming seriously degraded.

Just look across the Atlantic to see where some of these tendencies can ultimately lead.