Climatic Research Unit data is valid – don’t let the sceptics tell you otherwise

The data used by the University of East Anglia’s Climatic Research unit is perfectly valid. Do not listen to a word the deniers have to say.

The anti-scientific side of the blogosphere, and increasingly the mainstream media, is alight with what David Cameron’s old University friend James Delingpole hysterically asserts via his Daily Telegraph column could be the ‘final nail in the coffin of anthropogenic climate change’.

What Delingpole, together with the blowhards and headbangers on the US right are calling ‘climategate’, revolves around emails from the University of East Anglia’s Climate Research Unit (CRU) which were hacked and posted on the internet on Friday.

These climate deniers seem to think the CRU, like the Royal Society, NASA and the US National Academy of Sciences, are agents of a clandestine global movement against the truth. By Saturday morning this story had already resulted in over 600 blog posts and around 200 mainstream press mentions.

For the best reaction to this hyped up story making its way around the right wing echo chamber – see Realclimate’s reaction. (This is a website run by some of the world’s pre-eminent climate scientists.) Their team of peer-reviewed climate experts conclude:

“The timing of this particular episode is probably not coincidental. But if cherry-picked out-of-context phrases from stolen personal emails is the only response to the weight of the scientific evidence for the human influence on climate change, then there probably isn’t much to it.”

Professor Bob Watson – a chief science adviser to the government and former IPCC author – told the Today Programme this morning:

“These scientists at the University of East Anglia are both honourable and world class, their data is not being manipulated in any bad way whatsoever and it is totally consistent with two independent data sets in the United States, one at NASA, and one at NOAA, the National Oceanographic and Atmospheric Administration and so I think while some of the wording in those emails is inappropriate and should have been more careful, these scientists are not manipulating or hiding anything. There’s absolutely no doubt the world’s climate is changing, and this data set, along with other data sets, proves that beyond doubt.”

This is also an amusing must read reaction that provides some perspective.

Today Professor Phil Jones, the director of the CRU, whose emails are at the centre of the story, said he wanted to put the record straight, saying he saw the hack “caused a great deal of ill-informed comment”, and Kevin Trenberth, another leading climate change scientist whose private emails were also among those stolen, said the leaks may have been aimed at undermining next month’s global climate summit in Denmark.

He said:

“It is right before the Copenhagen debate, I’m sure that is not a coincidence.”

UPDATE 12:00 25/11/09

Left Foot Forward’s Rupert Read has a follow-up on the story here.

31 Responses to “Climatic Research Unit data is valid – don’t let the sceptics tell you otherwise”

  1. Shamik Das

    RT @leftfootfwd: Climatic Research Unit data IS valid – don’t let the sceptics tell you otherwise:- http://is.gd/51R5J

  2. Claire Spencer

    RT @leftfootfwd: Climatic Research Unit data IS valid – don’t let the sceptics tell you otherwise: http://is.gd/51R5J

  3. grace the collie

    hahahahahahahahahahahahahahahah!!!!!!!!!!!!!!!!!!!!!!

  4. Forlornehope

    Papers presented for publication are subject to peer review. Experts are not peer-reviewed. For a rather more significant revelation of internal documents, a recent disclosure in a Federal law suit in the USA showed that the major contrarian campaign knew very early on that their arguments did not stand up. Here is the link:

    http://www.nytimes.com/2009/04/24/science/earth/24deny.html?_r=1&hpw

    A lot more significant than the kind of background chatter in the CRU emails.

  5. Alex

    'Climategate' filleted. http://bit.ly/6iaxWt

  6. Joss Garman

    Forlornehope – fair point on the ‘peer reviewed’ thing. I hope it was obvious what I meant, i.e. that these are REAL climate scientists with papers on climate science that have been peer reviewed as opposed to blogs on pseudo science written by non-climate scientists.

  7. willstraw

    Fantastically constructive comment from Gracie the Collie. Well done on engaging with this debate in such a mature manner.

  8. Anon E Mouse

    Joss Garman – so you think it’s OK not to give raw data to people who request it and then smear and lie about people who don’t share their “opinion” on the reasons for global warming (which as you know isn’t happening – oh sorry I should have said climate change).

    By forcing poor countries to accept this nonsense on global warming is just cruel but then I suppose you people don’t need to grow a lot of crops in Kensington.

  9. Shamik Das

    And another supremely constructive contribution from Anon there, once again accusing others of smearing people while doing the same thing himself. And the idea, the very idea that the Right-wingers and the deniers are claiming to speak on behalf of poor countries is too contemptible for words.

  10. Richard Blogger

    There are several things to note about this “leak”.

    The first thing is that they are all text files. That is not surprising because internet mail is exchanged as text. However, depending on the mail server, email may be stored in an RDMS database, or as individual files, or some proprietary flat file system (or maybe a combination of all three). Email clients’ message stores will be one of these mechanisms too. However, the general standard for emails stored as individual text files is that they have the extension eml, yet these have the extension txt. The extension indicates to me that they were created on a Windows machine (where the erroneous idea that extensions have to be three characters still persists). It is also the extension used by the standard text editor on Windows (Notepad).

    As mail is passed between mail servers (at least two, the SMTP server of the sender and the SMTP server of the receiver) headers are added. Further headers are added to give the message a unique ID and also to give the ID of the message replied to (which allows threading by mail clients). Also MIME headers are added so that attachments (as additional bodies) can be added to the email. Usually an email message will have no less than 20 lines of headers.

    Yet we find that the CRU emails only have To, From, Date and Subject headers (no Message-ID, no Received, no Return-Path, no References, no In-Reply-TO and no MIME information). Since all email messages will have far more than these four headers it is incontrovertible that these emails have been edited – if only to remove the extra headers, but who knows what else was changed. As I mentioned above, the editing probably was carried out using Notepad.

    Without the headers that were removed it is difficult to see any threading in the emails, but a cursory inspection shows that they appear to be a jumble of emails, 1073 of them. How many emails do you think that the CRU mail server has archived – and remember the “leaked” emails are from 7 Mar 1996 to 12 Nov 2009? My guess is hundreds of thousands, if not millions. Yet somehow only 1073 of them were leaked. Even if the emails came from a user’s mailbox, do you think that a user would have saved just 1073 emails in a period of 13 1/2 years? I have tens of thousands of emails on my hard disk.

    I know from the time I worked in a university that much of my inbox was in the form “going for a drink on friday?”, yet I couldn’t find any messages like that.

    Now imagine you are a hacker and you have broken the security of some outfacing server and you see the storage folders of the email server, or perhaps a user’s mailbox. What would you do? Most likely you would copy everything. Then what would you do? Well since you have “dynamite” you would want to post this stuff as quickly as possible (partly to save your ass, so that you can remove the evidence from your machine, partly because you are so excited about the “dynamite”). So you would post everything as you found it to the internet.

    That is not what has happened. Someone has carefully edited the emails to remove the extra headers (that admittedly get in the way of reading the emails when using Notepad, but most hackers would have an email client anyway), they have also removed attachments (why?) and they have sifted through them and given just a few emails, ones that tell a particular story. All of this takes time. Someone has been very methodical about this.

    This is not the result opf some random hacker breaking into a web server, it is a concerted attack from someone who knows what to look for in the emails. The rather cack-handed editing of emails makes me think that it was not done by the hacker, but instead by whoever it is sifting the emails.

    Anyway, one email caught my attention (1256353124.txt). First, bear in mind that the bane of the climate scientists is someone called Steve McIntyre who tried to “prove” that the hockey stick graph “was not true” (the US National Research Council convened to investigate McIntyre’s claims basically said that McIntyre was wrong, being publicly denounced like that is a bitter blow to any climate change denier). The following CRU email is about a preliminary paper posted on the public website, it reads:

    I’m not thinking straight. It makes far more sense to have password-protection rather than IP-address protection. So, to access those pages

    Username: steve
    Password: tosser

    Have a good weekend!

    Mike

    I am assuming that the steve mentioned here is one Steve McIntyre!

  11. Why Lord Lawson is wrong | Left Foot Forward

    […] this earlier post on Left Foot […]

  12. Anon E Mouse

    Shamik – Who did I smear?

    Accusing people of things they haven’t done when you are a moderator on this blog gives you every advantage and is not fair.

    And on the subject of poor countries your party has tried to bomb Iraq and Afganistan back to the stone age – and as a Labour voter don’t call me Right Wing – your New Labour party backed George Bush.

  13. Scott B

    The people claiming that this is the end of the AGW debate are foolish but so are those saying that CRU data is completely valid. We don’t know that and the information in this leak casts things in more doubt. At the very least, these e-mails show concerted efforts to get around the FOIA processes to keep their data from being reviewed by outsiders and suggestions that the peer review process was being gamed. This lack of transparency is going to cast doubt in peoples’ minds and I can’t understand why most on the left don’t have an issue with it. How do we know that HADCRUT3 database is accurate? We haven’t seen any of the code or procedures in place for the adjustments. It can’t be extremely off (like the right would like people to think) since we have GISS to compare to, but would it shock me if the way they pick sites to include and how they fill in missing was not exactly scientific? Not at all after seeing this.

    Specific comments to the original author’s piece.

    “These climate deniers seem to think the CRU, like the Royal Society, NASA and the US National Academy of Sciences, are agents of a clandestine global movement against the truth. ”

    Way to smear whole groups of people. Keep using the right’s tactics…that’ll win people over.

    “For the best reaction to this hyped up story making its way around the right wing echo chamber – see Realclimate’s reaction. (This is a website run by some of the world’s pre-eminent climate scientists.) Their team of peer-reviewed climate experts conclude: ”

    You fail to mention that RC is run by some of the same people captured in these e-mails. They are probably not the best place to go to get an unbiased reaction. Not that I’m claiming sites like WUWT are either but saying RC has the best reaction is showing your bias.

    “Today Professor Phil Jones, the director of the CRU, whose emails are at the centre of the story, said he wanted to put the record straight, saying he saw the hack “caused a great deal of ill-informed comment”, and Kevin Trenberth, another leading climate change scientist whose private emails were also among those stolen, said the leaks may have been aimed at undermining next month’s global climate summit in Denmark”

    There’s a very easy way this could have all been avoided. Release the data. We don’t need all of these e-mails, but for every peer reviewed paper, everything that would be needed to allow others to reproduce the paper’s results should be public. This would allow all people, no matter what their biases are, to make informed conclusions. If their papers and conclusions are correct, the data will show it and they truly won’t have to worry about anyone except the Fox News crowd. Until this happens, I’ll remain skeptical of the exact conclusions of climate science. Especially around the true uncertainty in our understanding about past climate and future projections.

  14. grace the collie

    Sorry Will I was still laughing at http://di2.nu/foia/HARRY_READ_ME-30.html and I came across your blog you lost me when you said “is alight with what David Cameron’s old University friend James Delingpole hysterically asserts” it’s always tribal with you lot. This will be spun in to “fake but accurate”

  15. David Jones

    grace the collie,

    Can you see the irony in accusing others of being ‘always tribal’ and then immediately referring to them as ‘you lot’?

  16. Anon E Mouse

    The data is NOT valid.

    Told you they were fiddling the data.

    Forget the emails. Anyone who writes computer programs in a high level language, such as C, puts in comments to remind themselves of what the code is dong and to allow future programmers to make changes to the code.

    There are two files normally generated, a “Source Code” file (where the comments are) and the compiled file or “Object” (.obj) file or .asm file.

    (I’m a 1980’s Z80 man so this may be dated but you get the point)

    The file, after compiling, has the comments and other stuff removed and it is basically (no pun there) the “computer code” for the machine to run on.

    So the climate change programmer has left his comments here:

    http://wattsupwiththat.com/2009/11/22/cru-emails-may-be-open-to-interpretation-but-commented-code-by-the-programmer-tells-the-real-story/

    Personally since they don’t include the effects of the sun I can’t take the data seriously but I’m surprised that this terrific blog, being progressive as it claims, hasn’t picked up on this.

  17. grace the collie

    Don’t know much about “irony” but when I see a blog entry entitled “Climatic Research Unit data is valid – don’t let the sceptics tell you otherwise” alarm bells go off. It’s already decided that this is the line and there is no need to question, don’t need to think and don’t even think at looking at the source code. The data is not valid.

  18. Anon E Mouse

    Rupert Read – You have, deliberately or otherwise, missed the point here.

    Let me spell it out for you. The software that is used by the Climate Change scientists to make their predictions and claims, is programmed incorrectly and cannot give valid data. Fact.

    Not only that Rupert but smoke screens about emails are being used to cover this up. What concerns me is that you are not concerned about that software yourself. Why not?

    Any claims you make regarding data about temperature rises cannot be substantiated by this software and are therefore an act of faith and not scientific proof.

    The majority of people do not believe you are right Rupert and there is no proof you are. Is that way comments on your blog are blocked?

  19. Tim Worstall

    Talk about missing the point. The unknown part of climate change is what is climate sensitivity.

    We know what a doubling in the atmosphere will do directly: 0.7oC rise (yes, really). What we don’t know is how the sum of secondary effects (positive and negative feedbacks) works out. If climate sensitivity is, say, 2oC, then we’ve not much of a problem. If it’s 6 oC then we’re in shit street.

    What CRU is supposed to be working out is what is climate sensitivity. It’s the most important result we want. And if they’ve been hiding raw data, indulging in a lot of hand waving and in general pushing a line rather than doing their job, research, then we’ve all got really rather a large problem.

  20. Anon E Mouse

    Tim – My point is not the temperature, increase or otherwise – my issue is with the reasons for it. Leave aside the bogus data being pedaled by the CRU there is NO EVIDENCE, bogus or otherwise that (man made) CO2 causes global warming.

  21. Evidence biased

    I have to say; when George Monbiot says it looks bad, it’s got to be pretty damning. He’s calling for Phil Jones’ resignation and a re-evaluation of the evidence referred to in the emails. As for his assertion that the timing of this hack/leak with Copenhagen isn’t a coincidence, I’m sure he’s just being cynical. It obviously is, just like CRU’s discovery last week that temperatures will rise by 6oC by 2100.

    Can I also suggest that, for an evidence-based blog, a link to the leaked/stolen material might be more appropriate than a link to a Guardian story?

    http://www.eastangliaemails.com/index.php

  22. Anon E Mouse

    Rupert – The real scandal is the fact the software is bogus so the evidence is incorrect and even though you know that to be true you persist to pedal this unproven myth.

    George Monbiot in the Guardian seems more skeptical now than you are and since you hold public office as a councilor for the Green Party I find it very concerning that you seem not to be interested in this matter that affects our lives. You continually repeat the same (now) unproven stuff.

    Monbiot seems angry about this I wonder why it doesn’t bother you. Your site says you: “specialise(s) increasingly in political and environmental philosophy”.

    What philosophy includes denial of data when presented with it? Worrying.

  23. Rupert Read

    Check this out:
    http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack-context/ An update, to help understand in techie detail why the CRU ‘scandal’ ain’t much of one

  24. Rupert Read

    In answer to brave Anon E Mouse: Monbiot called on Phil Jones to resign to take resposibility for ethical lapses revealed by the emails, NOT because of any fundamental weakness in the climate science. Ditto for Monbiot’s call for re-analysis of some data: precisely so that the climate sceptics can be openly rebutted, NOT because there is any reason to doubt the actual data or its standard interpretation.
    Monbiot is not ‘sceptical’ at all – that’s just another of your crude distortions. He is simply more extremely vexed than I am with the ethical lapses revealed in the emails. My own view is that it is premature to call for Jones’s resignation: let the independent review report first.

  25. Anon E Mouse

    Rupert Read – Thanks for calling me brave. Not sure why a person who shares the majority opinion in this country should be called ‘brave’ but I accept your compliment to me in the spirit it was sent.

    My point with Moonbat was to illustrate the fact that (amazingly) he is less dogmatic than you seem to be and unlike your postings both here and on your blog (where comments are not allowed – surprise surprise Rupert) he disagrees with your Stalin like tendencies to prevent opinion and free speech that differs from your own.

    The fact you don’t believe that claims made using bogus software and that you believe the suppression and manipulation of data is a fair way to behave not a serious matter makes me wonder just how serious you are about this subject.

    Worrying.

  26. Billy Blofeld

    Joss – you had me at “anti-scientific”

    I suggest you read this web site in depth

    You’ll find the web-site is riddled with scientific analysis of this issue.

    Do you think you do your cause any good by labelling people “anti-scientific”, “blowhards” or “head bangers”?

    Personally I’m all for looking after the planet, but I hate the way extremists are following these two routes:

    – Telling us that there is scientific consensus (and that if we are sceptical about anything in any way, that we are “headbangers” etc)

    – Using the Green movement to unleash political ideals on the world (you know, the sort of Lisbon Treat gone global stuff)

    I object to being labelled anti-scientific.

    From Billy Blofeld BSc Chemical Engineering (Hons).

  27. Evidence Based Blogging « Billy Blofeld

    […] There is something to be said for claiming to be an evidence based blogger. You can use “evidence” to prove whatever the hell you like. […]

Leave a Reply