L'Ombre de l'Olivier

The Shadow of the Olive Tree

being the maunderings of an Englishman on the Côte d'Azur

21 November 2009 Blog Home : November 2009 : Permalink

CRU leak thoughts

Using my CRU search tool, I've been reading a selection of the emails by coming up with a variety of search terms and seeing what shows up - a search on "your eyes only" is interesting for example - and I'm coming to some conclusions.

Firstly the scientists themselves come across very differently and indeed in these internal emails frequently seem to disagree with each other in ways that they don't in public. Something else that comes across quite clearly is how tolerant (or rather how intolerant) some names are regarding criticism or contrasting views. Micheal Mann and Ben Santer in particular come across as deeply unpleasant individuals and ones that tend to go off the handle when facing even mild criticism from their peers let alone someone like Steve McIntyre. For example here's a couple of extracts from an early email about the MM03 paper - the first is written by Mann, the second by someone else:

Who knows what trickery has been pulled or selective use of data made. Its clear that "Energy and Environment" is being run by the baddies [...] The important thing is to deny that this has any intellectual credibility whatsoever and, if contacted by any media, to dismiss this for the stunt that it is..

[...]

Anyway, there's going to be a lot of noise on this one, and knowing Mann's very thin skin I am afraid he will react strongly, unless he has learned (as I hope he has) from the past...."

Others - e.g. Kaufmann - seem much more reasonable:

Regarding the "upside down man", as Nick's plot shows, when flipped, the Korttajarvi series has little impact on the overall reconstructions. Also, the series was not included in the calibration. Nonetheless, it's unfortunate that I flipped the Korttajarvi data. We used the density data as the temperature proxy, as recommended to me by Antii Ojala (co-author of the original work). It's weakly inversely related to organic matter content. I should have used the inverse of density as the temperature proxy. I probably got confused by the fact that the 20th century shows very high density values and I inadvertently equated that directly with temperature.

This is new territory for me, but not acknowledging an error might come back to bite us.

[...]

(5) McIntyre wrote to me to request the annual data series that we used to calculate the 10-year mean values (10-year means were up on the NOAA site the same AM as the paper was published). The only "non-published" data are the annual series from the ice cores (Agassiz, Dye-3, NGRIP, and Renland). We stated this in the footnote, but it does stretch our assertion that all of the data are available publicly. Bo: How do you want to proceed?

Should I forward the annual data to McIntyre?

Please let me -- better yet, the entire group -- know whether you think we should post a revision on RealScience, and whether we should include a reply to other criticism (1 through 5 above). I'm also thinking that I should write to Ojala and Tiljander directly to apologize for inadvertently reversing their data.

Not that reasonable means willing to accept that the "Anthropic Global Warming" hypothesis is anything other than fact or that they consider that anyone who shows even the slightest doubt concerning it is an idiot. As William Briggs observes, there is no conspiracy here, just a bunch of fervant believers. Even when looking at the unravelling of Briffa/Yamal the questions asked do not admit any doubt as to AGW, just whether the right data/techniques for showing it are being used, and how to spin a recovery:

It is distressing to read that American Stinker item. But Keith does seem to have got himself into a mess. As I pointed out in emails, Yamal is insignificant. And you say that (contrary to what M&M say) Yamal is *not* used in MBH, etc. So these facts alone are enough to shoot down M&M is a few sentences (which surely is the only way to go -- complex and wordy responses will be counter productive).

But, more generally, (even if it *is* irrelevant) how does Keith explain the McIntyre plot that compares Yamal-12 with Yamal-all? And how does he explain the apparent "selection" of the less well-replicated chronology rather that the later (better replicated) chronology? Of course, I don't know how often Yamal-12 has really been used in recent, post-1995, work. I suspect from what you say it is much less often that M&M say -- but where did they get their information? I presume they went thru papers to see if Yamal was cited, a pretty foolproof method if you ask me. Perhaps these things can be explained clearly and concisely -- but I am not sure Keith is able to do this as he is too close to the issue and probably quite pissed of.

And the issue of with-holding data is still a hot potato, one that affects both you and Keith (and Mann). Yes, there are reasons -- but many *good* scientists appear to be unsympathetic to these. The trouble here is that with-holding data looks like hiding something, and hiding means (in some eyes) that it is bogus science that is being hidden.

This email leads us on the other big issue. Secrecy.

These guys (see quoted para above) realize that with-holding data is scientificially a no no: many *good* scientists appear to be unsympathetic to [climate scientist's "reasons" for with-holding data] but they still do it, even though it "looks like hiding something, and hiding means (in some eyes) that it is bogus science that is being hidden."

Yet they remain secretive. Indeed they seem to be proud that they hide their data/methods and prefer to delete things rather than respond to requests for data:

Yes, we've learned out lesson about FTP. We're going to be very careful in the future what gets put there. Scott really screwed up big time when he established that directory so that Tim could access the data. Yeah, there is a freedom of information act in the U.S., and the contrarians are going to try to use it for all its worth. But there are also intellectual property rights issues, so it isn't clear how these sorts of things will play out ultimately in the U.S.

[...]

Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don't leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I'll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? - our does ! The UK works on precedents, so the first request will test it. We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it - thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who'll say we must adhere to it !

I think this email thread is probably one of the most damaging. Yes the "trick" thread is bad, but while I think the trick is pretty despicable and makes Mann a liar it isn't in itself evil so long as people can figure out what has happened.

Unfortunately what we see here is that they deliberately make it as hard as possible for people to reverse engineer what they are doing. If this were to do with attempts to model the sex life of nematodes (to pick an example at semi-random) then this probably wouldn't matter, but it isn't. Governments and industrial leaders are making multi-billion dollar (euro, pound etc.) decisions about taxes, investments and so on based on this research. If it is wrong then said politicians and business leaders are likely to make the wrong decisions and that could result in needless poverty, in global economic collapse and all sorts of other things (including I guess the extinction of humanity if it turns out AGW is rather more serious than it appears at present). I note that these emails (and the related code) does indeed show that Steve McIntyre was absolutely correct when he says that the various financial regulators (SEC etc.) would bar any stock tout who tried to raise money using a similar level of documentation of claims. It would be ironic that the economies of most of the nations of the world are being shaped by such poor quality science if it weren't so dangerous.