In praise of Stephen McIntyre

Stephen McIntyre is a statistician with a career background in mining exploration. Huge amounts of money are at stake in the evaluation of mining prospects and the legal constraints are quite rightly extremely tight. There is no room for wishful thinking or sloppy analytical procedures in this business. The prospectus for a mining development has to be diligently written because it will be investigated equally as diligently by potential investors.

Since 2003 McIntyre has held the climate science community to scientific account, in the way that the mining business is held to account. McIntyre's blog is accurately titled: Climate Audit.

It's not easy for newcomers to Climate Audit to orient themselves. McIntyre writes clearly and precisely, without hyperbole or editorialising, but after more than a decade on the front line much of the terminology and argumentation is in a shorthand form that presumes a certain amount of knowledge on the part of the reader. It's worth the effort. Posting is relatively infrequent, which allows time for the knowledgeable commenters on the blog to do their work.

Frying tonight

Nowadays we are all meant to be in a panic about the average atmospheric temperature of the globe (no, I don't know what that is either) the question of whether it is rising or falling, or has risen or fallen over long periods of time needs to be posed. Is recent warming or cooling unusual?

It is a difficult question to answer, very difficult. Thermometers were invented in the middle of the 18th century. Half a century of refinement and standardisation followed, then another half century went by before they came into widespread use for measuring atmospheric temperature and reliable records started to be kept. If we want to know the air temperatures of a place before say, 1800, we are in trouble. We shall leave the problem of world coverage of thermometer measurements for another day, but it is fair to point out that even the modern thermometer coverage of the globe leaves a lot to be desired.

So we look around for some physical phenomenon that over a long time-span can be taken to be a 'proxy' for temperature. In our everyday household mercury or alcohol thermometer we are actually measuring the expansion or contraction of a column of liquid as the temperature rises or falls, not 'temperature' itself. But because there is a linear relationship between the expansion of the liquid and its temperature we can take the size change of the liquid as a proxy, calibrate some scale of units and then say that our thermometer is measuring temperature.

The question therefore is: can we find something in nature, from a time before man devised the thermometer, that behaves like the liquid in our modern thermometers, that changes in some consistent way as temperature changes?

The simple answer for the person in a hurry is no.

A number of possible proxies have been investigated – sediments, ice sheets and the composition of gases trapped in them, corals and shells – but none of them can be said to function as the simple atmospheric temperature proxy we are looking for. They all vary in complex ways in response to a range of influences, not just temperature. Some deliver not uninteresting information and seem to be reliable in what they say, but their use as proxies of atmospheric temperature has to be treated with extreme caution. A popular historical temperature proxy is tree growth, as measured by tree-ring width (or some other factor of the ring). We can count rings to establish a date and analyse the individual rings to establish… something.

If you go down to the woods today…

At the merest suggestion of using tree-rings as a proxy for temperature we just need to apply our Post-it™ note sanity test and the idea would be dismissed within minutes. If we wanted to list the reasons not to use tree-rings we would need an awful lot of Post-it™ notes. Here are a few hints for the curious, not intended to be comprehensive.

Growth does not take place through the whole calendar year, a portion of that time the tree is dormant, so our proxy measurement can only really relate to the growing season, i.e. spring and summer. During that growing season there will be major weather variations. Air temperature is only one of a huge range of influences, some of which are interdependent, for example the availability of water. When plenty of water is available and temperatures are high then growth will potentially be good, whereas high temperatures and water shortage create drought conditions that result in low or no growth. A cold and wet season is also not going to result in high growth rates. Temperatures and water availability and sunlight all vary in complex interactions, as does the concentration of carbon dioxide in the atmosphere, which itself has an appreciable effect on tree growth.

Trees are not just forest collectives but individuals, their growth affected by many individual factors. Neighbouring trees die, are blown over, struck by lightning, have their bark and cambium gnawed by forest animals and their leaves, trunk and branches attacked by insects. The neighbouring survivors gain more light, more water and nutrients and for a few seasons grow better until the competition catches up. The outer trees of a forest may get more light, but may be at the limits of the growing area and also be more vulnerable to wind and animal attack. As the light conditions change trees can rotate to optimise their growth. A few years of grazing by sheep or other animals will have a dramatic effect on undergrowth competition and soil fertilisation.

At this point the rational person would stop, take a single Post-it™ note and write on it: 'Will not work'.

…you're sure of a big surprise

In contrast, the climate scientist will start to ponder. If only we took a large number of samples from different trees over a large area and crunched the numbers from long periods in various inventive ways using statistical magic to filter out all possible individual, local and regional factors – we might be left with a temperature record.

The weakness of this argument is that removing 'noise' from a signal is only meaningful when there is a signal there in the first place; creating a signal out of noise is not the same thing. There is no continuous, independent temperature signal in this mass of data: in times of plentiful water and nutrients and when everything else is beneficial then increasing temperatures (within limits) would lead to better growth; in times of drought or other deficits increasing temperature may have just the opposite effect. All of the growth factors are varying interdependently in complex ways.

The problems of tree-ring analysis start right at the beginning of a research project when the samples are taken. Why take cores from certain trees and not others? Are the fieldworkers just sampling the easy-to-reach trees? The biggest trees? The healthy trees? Is the sample randomised, in which case is the sample sufficiently large?

Once the data is in, the research team is tempted to look at it and weed out data that appears to be inconsistent or, worst of all, doesn't match the required narrative. If the sample was supposedly random, then its statistical integrity has been immediately violated.

Once through that filter, the data from the remaining samples is then run through a suite of badly understood and often quite ad hoc statistical processors that spit out some results. There will be weighting of the various records on the basis of something or other – often wishful thinking – then the team looks once more at the results and removes those that don't seem to be 'right' – that is, that don't fit the desired narrative. Yet one more violation of the integrity of the sample.

The various sets of the remaining results are blended together in magical ways and emerge from the Moulinex as a series of temperature values plotted over a period of time.

The resulting data is published as a temperature reconstruction. Over the course of time it will be merged into the results of other studies, some of them from tree-rings, some from sediments etc. and become relied upon and respected as a 'multi-proxy' study. Over the years, Climate Audit has gained much bitter amusement from such constructs.

Not all tree-ring research is conducted this badly, but then, only one of these misdeeds is sufficient to invalidate the results. A notorious example of skewed sampling and processing comes from the Yamal peninsula in Russia. It was Climate Audit that managed through tenacity and attention to detail to carry out the autopsy on this extremely flawed reconstruction. A good, readable account of the Yamal story (as well as some other climatology horrors) is here.

Of course, we can say with some confidence that, aside from all these local factors, trees (like people) flourish during periods of benign weather, such as the 'Medieval Warm Period' and struggle during periods of inclement weather such as the 'Little Ice Age'. It's a pity that we can't just leave it at that. Unfortunately the y-axis of the graph is not labelled 'happy trees - sad trees' or 'nice weather - nasty weather', it is labelled '°C' or '°K'.

The divergence problem

If you need more convincing about the shaky base on which dendroclimatology rests it should be sufficient to note the existence of what the tree-ring crowd call the 'divergence problem' – making it sound like some itsy-bitsy triviality which is just about to be fixed. It isn't, nor is it. It refers to the fact that when a period of tree-ring temperature data overlaps with a period of thermometer temperature data, the tree-ring temperatures frequently diverge – and not just slightly, either.

In other words, one of these plots is wrong – and it's not the thermometer plot, even allowing for all the defects it might have. A sane observer would therefore regard tree-ring temperature proxies as discredited, but there are still dendroclimatologists desperately trying to think up reasons why, after a thousand years or so, trees suddenly stop responding to temperature (or whatever it is to which they are really responding).

Until someone comes up with an explanation, something has to be done to cover up this embarrassing artefact. Climate Audit has been relentless in unearthing and analysing these ruses. There are, for example, climate 'scientists' who slice the embarrassing end off the tree-ring data (Hide-The-Decline™), then cover up the amputation extremely artfully under a mess of intersecting lines. It was illustrated well by the Mail on Sunday in a piece by David Rose in 2009:

As you can see, knowledge of the obfuscation technique reached up into one of the largest outlets of the mainstream media nearly seven years ago. Consequences: none.

Those sensitive souls among you who are shocked at such dissembling should note that, according to a review by McIntyre in Climate Audit in 2014, the same technique had been used 22 times over a period of ten years before its five minutes of fame in the Mail on Sunday in 2009. On six additional occasions, one as late as 2013, the inconvenient tree-ring temperature reconstruction that didn't fit the desired narrative was simply expunged.

Jesuitically-minded readers can puzzle at their own leisure about the severity of the sin of commission (truncating the plot) versus the sin of omission (leaving the reconstruction out altogether). And when they have done that they can plunge deeper into the ethical swamp to consider the other attempts to manipulate data during processing using various 'fudge factors' that simply redraw the 'decline' to make it acceptable. And after that they have to consider the thousands of ostensibly bright people – the journal readers, reviewers and editors – who never noticed such practices over the course of ten years, or if they did, never said anything.

The author of the truncated reconstruction as well as the Yamal research has since risen to the heights of emeritus professor. His Wikepedia entry surprisingly makes no mention of his greatest claim to fame.

Mannian manipulation

The attempt to remove the divergence problem, that blemish on the desired narrative of modern global warming, has attracted some of the most formidable intellects of climate science, chief among them Michael Mann, the author of the 'Hockey Stick' temperature reconstruction. Mann, according to his carefully crafted Wikipedia entry, 'has pioneered techniques to find patterns in past climate change, and to isolate climate signals from "noisy data"'.

Mann's isolation of climate signals in his 'Hockey Stick' reconstruction flattened the warm periods in the last 1,000 years into insignificance to create the almost level 'shaft' of the stick and delivered a remarkable uptick of temperatures in the last century to create its 'blade'.

Thanks above all to the efforts of Steve McIntyre and Climate Audit Mann's 'Hockey Stick' has lost all credibility, even among many climate scientists.

If we leave aside the much criticised processing techniques of the first nine-tenths of the reconstruction, we see that Mann came up with a sticking-plaster solution for the final century that is known as Mike's-Nature-Trick™. This spliced the instrumental record onto the conclusion of the paleoclimate record using statistical techniques. Climate Audit proved without a shadow of a doubt that this had been done. Bad enough – but what makes it much worse is the way that Mann and his myrmidons repeatedly flatly denied that this had been done and piously asserted that such things would never be done by anyone.

The revulsion at this trickery was so great that Michael Mann was made 'Distinguished Professor of Meteorology' at Pennsylvania State University in 2013.

There have been several such tricks and ruses. If you have a stomach for this stuff, you will find an overview on Climate Audit here. The divergence problem has not been solved, just obfuscated or hidden, and that with effort and cunning. Fortunately, we had Climate Audit to tell us about such improprieties.

Fighting the good fight

Since 2003 McIntyre and his helpers have demolished a large amount of the palaeoclimatological work to which they have had access. Many of these studies have consisted in collections of results from various sources, not only tree-rings. Stephen McIntyre would probably not approve of wild language such as 'demolished', but when you stand back at some distance from his work that is what you see.

The phrase 'to which they have had access' is an important one. There is a desire in the philosophy of science that scientific research should be capable of replication and should at some point actually be replicated. Where the experimental set-up consists of commonly available kit – photospectrometers, lasers etc. such replication is relatively easy to perform and is a valuable exercise.

Where collecting the data involves extensive and expensive fieldwork in distant inaccessible forests full replication is out of the question. The only replication that can be done is examining and reprocessing the data from the original study. Again and again, McIntyre's access to data and the algorithms that were used in such studies has been blocked by 'scientists' – sometimes quite openly, sometimes through subterfuge, sometimes with some politeness, sometimes with great rudeness or simply by ignoring requests.

In some cases, particularly the infamous and influential 'Hockey Stick' curve produced by Michael Mann and associates, McIntyre and his colleagues had to effectively reverse engineer the results in order to try to get at the procedures that were used. Again and again, determined and precise work has revealed how defective the examined work was.

The data behind palaeoclimate work is not always made public, at least not to anyone who might challenge the result or might be tempted to try and replicate the processing. It would be fair to say that it is very often either wilfully hidden, or, if a journal demands that it be archived, as is the normal practice nowadays, it will often be archived in a deliberately sloppy or obstructive way: in an outdated version, parts missing, missing metadata (the descriptive complement to the data), comments removed. At least we have to assume it is deliberate, because if this is the normal way the scientists organize their work then they have serious problems.

The rule seems to be: if you want to belong to the climate science club you don't question the work of the other members.

Without access to the original data and the processing information the verification principle of scientific research is immediately sabotaged, because unless some research team has the money and time to go to the back of beyond and do all the sampling all over again, which is completely unrealistic, no one else can check the data or the processing that has gone into it.

The response of those researchers affected? Typically, silence. Where the embarrassment has been too great even for climate scientists, who seem to have little sense of professional or personal shame, and corrections have had to be made as a result of McIntyre's work on Climate Audit, his work is never credited. Many professional climate scientists never mention Climate Audit, only a very few engage occasionally with its work. If they ignore it, they think it will go away. Since 2003 it hasn't.

A peek behind the curtain

For Climate Audit, the two releases of emails known as Climategate I and II were very important events, particularly the first of these. The black box that was climate research was opened to the light and the core group of climate scientists were seen to be conspiring in quite outrageous ways to rig the journal publication system and to stifle any opposition to what had become the notorious 'consensus'. The following email, released in Climategate III, gives an extraordinary insight into the attitudes of the climate science community to any possible dissent, and particularly to Steve McIntyre:

At 11:14 19/10/2003 -0400, Michael E. Mann wrote:

FYI–thought you guys should have this (below). This guy “McIntyre” appears to be yet another shill for industry–he appears to be the one who forwarded the the scurrilous “climateskeptic” criticisms of the recent Bradley et al Science paper.

Here is an email I sent him a few weeks ago in response to an inquiry. It appears, by the way, that he has been trying to break into our machine (“multiproxy”) [a publicly accessible FTP site]. Obviously, this character is looking for any little thing he can get ahold of.

…

p.s. I’m setting up my email server so that it automatically rejects emails from the “usual suspects”. You might want to do the same. As they increasingly get automatic reject messages from the scientists, they’ll start to get the picture…

But that is how it is in the looking-glass world that is climate science: the 'outrage' was directed at McIntyre for highlighting the problems and causing climate scientists extra work in having to respond, not at the climate scientists who caused them in the first place. There was also much hand-wringing at the lèse-majesté of those who read the 'stolen/hacked', 'private' emails that this malignant group of researchers had sent to each other. But, in the end, no climate scientists were harmed in the making of 'Climategate', none criticised or disciplined, no one lost his or her job, was demoted or even mildly ticked off. Careers continued on an orderly trajectory and promotions came along punctually.

Damning with no praise

The behaviour of some climate scientists towards Stephen McIntyre and the valuable work done at Climate Audit has been reprehensible: mendacious, dissembling, vicious – we could string out a long series of critical adjectives here. I am not giving chapter and verse: just go to Climate Audit and take your pick – you have over a dozen years' worth to choose from.

Even when we leave out the extremes there is a strong undercurrent of professional hostility. As an example let us take McIntyre's page on the English language Wikipedia. The entry for McIntyre is frostily 'neutral', which means it mentions a minimum of his achievements in climate science, and even then merely as the gadfly – the losing half of a pointless and distracting controversy. Wikipedia can't be bothered to fish out a photo of him, nor find out his date of birth. 'Mealy-mouthed', not neutral, is the expression that comes to mind.

It gives extensive and trivial details from his professional career in the mining industry, leaving us with the impression that his work in climate science is just some kind of crazed hobby-horse. Read carelessly it may even suggest to those conspiracists who need to find demons that he is somehow in the pay of unspecified 'mining interests' and so parti-pris in the CAGW debate. The article even manages to spell his first name in different ways. It gives the final word on him to a left-wing journal with a tiny circulation which notes McIntyre's influence but asserts that it 'might not be positive'.

As usual where Wikipedia is concerned, we on this blog like to compare its language versions: the page in French is effectively a translation of the English page; the German page is brief and dismissive – as we might expect from a country of climate conformity and deferent respect for Bildung (qualifications) – and gives the last word on several points to the arch-enemy, Michael Mann, of all people; the Dutch version is brief but quite fair; the Spanish departs from the others in many respects and is fairly even-handed (except that it has added six years to his age).

Some devil's advocacy – but not much

Why is this? Why do well-educated, supposedly rational scientists and the people who cluster round them exhibit such bovine solidarity spiced at times with vicious intemperance?

At this point the Feynman fans will recall his remarks to the effect that the experimental scientist should be the harshest critic of his or her own work: every possibility of error should be considered and openly presented. Well, that's an easy thing for a renowned theoretical physicist at the peak of his profession and with a Nobel Prize on his CV to say. For a post-doctoral student or an associate professor in an empirical science not so much, particularly a field such as climate science that is awash with money and survives on imprecision. The hits of the approved consensual type have to be delivered frequently, the press releases have to be written, the funds have to flow.

Climate scientists obviously aren't born twisted, they just become so as they grow up in this twisted field. The tenor of those at the top flows down through the entire structure. Polite enquiries – McIntyre is always polite – or questions are ignored or magisterially swatted away as coming from one of those 'industry-funded climate disinformation websites' (Michael Mann). And this is why the work that Stephen McIntyre has done at Climate Audit has been so valuable and should be singled out for special praise, for it was done by the most independently minded man in climate science in the teeth of the most stubborn resistance.

Fifteen years of dogged, unrewarded and officially unrecognised work has exposed the swamp that is climate science. It is high time it was drained.

It shouldn't matter which side of the CAGW debate you are on. The machinations of some climate scientists are completely unacceptable and should be condemned by their own colleagues as well as by the great and the good. In any healthy institution (such as the mining business) such machinations would result in sackings and even prison. Until that broad condemnation comes, those who stand silent on the sidelines are all tarred with the same brush.

[This piece was revised on 03.09.2013 to correct some errors of fact that were pointed out by Stephen McIntyre. The author would like to express his gratitude to Mr McIntyre for these corrections.]