Reading the "ClimateGate" Emails

On November 19, 2009, a 61 megabyte file called "FOI2009.zip" started to circulate on the internet. This contained a directory of over 1000 emails sent to and from people at the Climate Research Unit, University of East Anglia, UK, plus supporting documents and software. Check a BitTorrent site for "FOI2009.zip" if you want a copy, or you can find a searchable index of the email at http://www.climate-gate.org/.

CRU is one of the major climate research centers, and the people sending the messages are the biggest names in climate science. The revealed messages have been fairly embarrassing to the people involved. Phil Jones, the director of the CRU, has temporarily stepped down while it's being investigated.

Right-wing global warming "deniers" are having a field day with it. Mainstream press and the lefty blogs are taking a "move along, nothing to see here" attitude. Many commenters don't seem to have read the emails themselves. They are just repeating the same fragments over and over in the usual echo chamber.

I've been interested in climate issues for awhile now, and have written a couple of long pieces on where I think things stand. If you had to describe my position in a single sentence, it would be "the current data, models and theories of climate aren't solid enough to say one way or the other if humans are warming the planet." You can find my previous items here:

- Global Warming: Our Story So far
- NASA and the Warmest Year

What's in the Mail

The earliest email is March 7, 1996, and the latest, November 12, 2009. Most of them are to, from or CC'd to Phil Jones, Director of CRU, but many different authors are represented, and some of the emails do not even CC Mr. Jones. Also, despite the juicy comments that have been reprinted, the vast majority of the mail is day to day details on preparing research papers or chapters for the IPCC documents. It's dry as dust. See Email 1139932579.

An interesting question is where this all came from. It's usually described as the result of hacking the Climate Research Unit (CRU). I doubt this is right. These are a very carefully selected set of emails, with none of the personal messages or administrative junk you'd normally find in an archive. It also just doesn't have the "nyah, nyah" shock value and sloppiness of most hacker efforts. I think this is more likely to be an inside job.

Despite the name, and some speculation along those lines, I don't think this set of messages was selected as part of an official Freedom Of Information Act reply. No administrator, lawyer or CRU member would include emails that insult other members of the community, especially when there's no actual climate science mentioned. On top of that, the emails themselves show that CRU had no intention of answering the FOI requests.

Since there's no single person listed as sender, recipient or CC on all of the messages, it's hard to see who could have collected all of these. Phil Jones himself comes closest, so perhaps this is extracted from his personal archive. It's hard to believe he'd have kept some of these though. It will be interesting to see what develops.

Climatology Is Hard

If you want to go straight to the accusations against the CRU and my take on them, go here.

But first, I think we need some background, including a discussion of the data that climate science relies on. In my opinion, the "smoking gun" in the messages are not the badmouthing of critics, the attempts to manipulate peer review, or the refusal of the FOI requests. Instead, I was surprised by their own doubts about the quality of climate data.

If we see warming today, and want to blame it on human activity, it would help if we could answer two questions.

1. Is this warming unique, or has it happened before? If it has, then it seems unlikely to be just due to humans. Also, if we've had warm temperatures before, and no "runaway" warming or "tipping point", then it removes some of the urgency of the issue.

2. If we are warming, when did it start? If it starts in 1950, industrialization could be a cause. If it starts in 1850 or earlier, natural factors are more likely.

To answer either of those questions, we need historical climate data. Knowing current temperatures and following the recent trend isn't enough.

Data is Scarce

The first problem is that the historical data is scarce, and the farther back in time you go, the worse the problem gets. The map below compares the warming of the 1930s with the warming of the 1990s. The gray parts of the map are places where there is no temperature data in the 1930s, and so no comparison can be made. You can see that much of the world is a bit warmer now than in the 1930s, but some of it is colder. The temperature in the U.S. is about the same now as then.

Data is missing in South America, Africa, the Middle East, Central Asia, the Arctic and Antarctic. The Year 1900 would look even worse, almost completely blank outside North America and Western Europe.

I haven't plotted the sea data from CRU because it's heavily interpolated to show the entire world. This is a problem, since there's no way they have data for the entire world. Instead, they have temperatures collected by working sailors traveling shipping lanes (no global warming research vessels or automated buoys in 1930!). The modern shipping lanes look like this:

Anything far off these lanes won't be in the data. Again, there would be little at the poles, and not many points from the middle of the Pacific Ocean. The Southern Hemisphere would be poorly covered.

What's the point of these maps? Any graph that shows global warming this century has to have used this data. All of the climate research centers have access to the same land weather stations. Their climate graphs and models only differ in how they process the data.

The climate researchers can't create data where there is none. They can't create a true average temperature for 1930 with a map this skimpy. A big part of their trends could very well be determined by how they compute these global temperatures. If the true data for 1930 showed a hot spot over Africa or Antarctica, it would completely revise the graphs of 20th century temperatures. All of those graphs should be displayed with wide error bars, with the size of the error increasing as you go back in history.

In addition, any claim that this decade is the "warmest in a century", or "warmest in 1000 years" is stretching things. We don't have enough data to compare even to 1930, when we know it was comparably warm in the U.S..

There are several types of data used in climate research: satellite data, land temperature stations, sea surface temperatures, and proxies. Let's look at each, and see what the CRU emails have to say about them.

Satellite Data

Satellite data is widely regarded as the best data there is, because it covers the entire planet, and seems independent of data gathering issues or "urban heat island" effects. I thought this was the gold standard, but there's an email that makes me doubt it:

Email 1079108576: (fourth para, last msg)

Even though much of the differences [between satellite and ground temps] may now be apparently explained, it's still a terribly messy job. The satellite system wasn't designed to measure tropospheric temperatures, the calibration and orbital decay and retrieval algorithm and all the other technical issues are ugly, and nobody knows how much the lower stratospheric cooling ought to have infected the upper troposphere, among other points one might make.

Good or not, the real problem with satellite data is that it only goes back to the 1980s. This means we don't have any satellite coverage of prior warm periods like the 1930s to use as comparison. Instead, we compare standardized ground temperatures with standardized satellite temperatures. Since the data was collected with completely different types of instrument, the comparison will only be right if your standardization techniques are right.

Land Temperature Stations

These are the ground stations at airports and cities, used for weather reports and forecasting before satellites. Some go back to 1880 in the U.S. The rest of the world is very different. The farther you go back, the fewer there are. For earlier periods in the 20th century, the coverage is minimal outside North American and Western Europe.

There are issues with this data as well. Stations have been moved, the time of day readings were taken sometimes changed, and there are breaks in the data. If a city grows up around the station, the "Urban Heat Island" effect may bias the readings upwards over time. A website, http://www.surfacestations.org/ claims that two thirds of the sites they have checked do not meet the standard for accurate readings. How much to correct for this is an open question. How much to correct for Urban Heat Island effects is an open question as well.

Email 1231279297: quoted msg

At 17:01 on 01/06/2009, Phil Jones wrote to Chris Folland,

City population size and urban effects are not related that well. I think a lot depends on where the city is in relation to the sea, large rivers and water bodies as well. I did try and get population figures for London from various times during the 20th century. I found these, but the area of London they referred to kept changing. Getting the areas proved more difficult, as I thought population density would be better. Those I could find showed that the area was increasing, so I sort of gave up on it.

Sea Surface Temperatures

Temperature has been measured by ships at sea for decades. Here's one of the emails on Sea Surface Temperature (SST) data. This is a writeup for publication (not sure by whom), forwarded from Gavin Schmidt to Phil Jones:

Email 1212276269:

... Like almost all historical climate data, ship-board sea surface temperatures (SST) were not collected with long term climate trends in mind. Thus practices varied enormously among ships and fleets and over time. In the 19th Century, simple wooden buckets would be thrown over the side to collect the water (a non-trivial exercise when a ship is moving, as many novice ocean-going researchers will painfully recall). Later on, special canvas buckets were used, and after WWII, insulated 'buckets' became more standard - though these aren't really buckets in the colloquial sense of the word as the photo shows (pay attention to this because it comes up later).

The thermodynamic properties of each of these buckets are different and so when blending data sources together to get an estimate of the true anomaly, corrections for these biases are needed. For instance, the canvas buckets give a temperature up to 1C cooler in some circumstances (that depend on season and location) than the modern insulated buckets. Insulated buckets have a slight cool bias compared to temperature measurements that are taken at the inlet for water in the engine room which is the most used method at present. Automated buoys which became more common in recent decades tend to be cooler than the engine intake measures as well. The recent IPCC report had a thorough description of these issues (section 3.B.3) fully acknowledging that these corrections were a work in progress.

Maritime records back to 1900 or before (handwritten, probably) would have been digitized. Then each record would have been coded for the type of bucket used. Then each measured temperature would have been "standardized" to adjust the temperature record based on type of bucket, to make them all uniform.

This all matters because if you adjust a bucket temperature by 1 degree, you are adjusting it by more than the warming signal you are looking for. If you get that wrong (systematically wrong, because of your software or assumptions) you can bias the data upwards. There's even an email about this:

Email 1224176459:

On Oct 16, 2008, at 6:52 AM, Phil Jones wrote:

... In 1990, for example, almost all SST values came from ships. By 2000 there were about 20% from Buoys and Drifters, but by 2008 this percentage is about 85%. We're also doing comparisons of the drifters with the ships where both are plentiful, as it is likely that drifters measure a tenth of one degree C cooler than ships, and the 1961-90 period is ship-based average.

New version of the dataset coming in summer 2009.

All the skeptics look at the land data to explain differences between datasets and say urbanization is responsible for some or all of the warming. The real problem is the marine data at the moment.

Proxies, including Tree Rings

Prior to 1880, you have few temperature records from thermometers. So they try to reconstruct climate from "proxies" -- natural events, like the growth of trees (shown in the width of the rings), that are related to temperature.

The ideal case would be something like a California Redwood tree, sitting in a forest undisturbed by people or nearby cities, for 2000 years. If the growth of the tree were affected strongly by temperature, and that location had a strong global warming signal, you'd be set. Unfortunately, that's not the case. Here are some issues with tree rings:

• High elevation trees respond to temperature more than low elevation ones (which respond mainly to water supply.)

• Different types of tree have different temperature response characteristics.

• One email from a critic states that the response just "isn't linear."

• Once temperature rises enough, perhaps other factors predominate?

• You have to correct for aging of the tree. Young trees grow faster (thicker rings) under the same conditions.

• Something called the "strip bark" problem. Not sure if this is a type of tree, or regrowth of a damaged area of a tree, which changes the ring pattern.

• You would like to compare nearby land temperature records during the modern period with tree ring growth during modern period. Apparently the kinds of trees they are looking at (slow-growing in remote areas) don't have any nearby temperature stations.

• During the modern period (after 1850) there has been an increasing growth trend, noted as a "divergence" in the literature and unaccounted for. CO2 fertilization (extra growth because there's more CO2 in the atmosphere) is assumed, and the derived temps are standardized on that basis.

At 11:57 11/16/2006, Keith Briffa wrote:

The main one is an ambiguity in the nature and consistency of their sensitivity to temperature variations. It was widely believed some 2-3 decades ago, that high-elevation trees were predominantly responding to temperature and low elevation ones to available water supply (not always related in a simple way to measured precipitation)
...
This last adjustment obviously will have a large influence on the quantification of the link between these Western US trees and N.Hemisphere temperatures. At this point , it is fair to say that this adjustment was arbitrary and the link between Bristlecone pine growth and CO2 is, at the very least, arguable.
...
The bottom line though is that these trees likely represent a mixed temperature and moisture-supply response that might vary on longer timescales.
...
The point of this message is to show that that this issue is complex, and I still believe the "Western US" series and its interpretation in terms of Hemispheric mean temperature is perhaps a "Pandora's box" that we might open at our peril!

At 10:34 10/02/2009, Donald Keiller wrote to Kieth Briffa:

Dear Professor Briffa, my apologies for contacting you directly, particularly since I hear that you are unwell. However the recent release of tree ring data by CRU has prompted much discussion and indeed disquiet about the methodology and conclusions of a number of key papers by you and co-workers.

As an environmental plant physiologist, I have followed the long debate starting with Mann et al (1998) and through to Kaufman et al (2009). As time has progressed I have found myself more concerned with the whole scientific basis of dendroclimatology. In particular;

1) The appropriateness of the statistical analyses employed
2) The reliance on the same small datasets in these multiple studies
3) The concept of "teleconnection" by which certain trees respond to the "Global Temperature Field", rather than local climate
4) The assumption that tree ring width and density are related to temperature in a linear manner.

Whilst I would not describe myself as an expert statistician, I do use inferential statistics routinely for both research and teaching and find difficulty in understanding the statistical rationale in these papers. As a plant physiologist I can say without hesitation that points 3 and 4 do not agree with the accepted science.

The Unfortunate Timing of Climate Records

Here's the "hockey stick" graph from the IPCC report, based on tree-ring data from Michael Mann:

Although this graph is in dispute, it still illustrates a fundamental problem. Proxy records like tree rings are used up to around 1850-1880. There's debate on which to use, and whether global temperatures show the "Medieval Warm Period" and the "Little Ice Age". But no one thinks man-made global warming is happening during this period.

Right around 1900 we see a sharp warming in this graph and others. Unfortunately, that's right when the "instrumental record" (land and sea temperatures measured with thermometers) starts. Wouldn't it be nice to have a proxy like tree rings that exactly matched the modern record from thermometers? But apparently, we don't have that. The kinds of long-lived trees that have been cored for records don't grow near populated areas where there are long temperature records. Cores are coming from the high Rockies or Siberia, and are very difficult to calibrate to modern records.

Finally, we see a second sharp warming in the 1980s. But this is when the instrumental record is replaced with the satellite record! Again, it would be really nice if satellite data went back far enough to see the cold period from 1950-1980, but it doesn't. And in fact, the instrumental records starts to thin out again after 1980, as weather forecasting switches to satellites from ground stations. There are fewer ground stations now than there were in 1975, and big parts of Africa have dropped off the land-based maps again.

What you would want ideally is a single record that covers the last 1000 years, showing all the historical events and extending into the modern period where it could be compared to thermometers and satellites. We don't have that. Instead we have a patchwork of records from different sources, that all have to be standardized with lots of fiddly little adjustments. This is just asking for statistical errors and biases to be added into the data, producing a trend where there is none. The Accusations Other blogs have covered the juicy bits of the emails, so I'm not going to. You can find lots of details out on the net, or look at the emails yourself. Let me give a summary of what CRU has been accused of, and my take from reading the email.

CRU is Evading Freedom Of Information Act Requests

It's a matter of record that Steve McIntyre and others made FOI requests for the data held by CRU. These requests were refused. The emails make it clear that CRU had no intention of answering the requests. They found excuses to refuse the requests and that was the end of it as far as they were concerned. There's also a single email from Phil Jones to Michael Mann asking him to delete emails concerning the preparation of the IPCC Fourth Assessment Report (AR4).

If they get into any legal trouble, it will be because of this issue. The more interesting question is why they refused. Possibilities include:

1. The data are seriously flawed and independent researchers would soon discover that. This is the favored explanation of the skeptics and right-wing blogs.

2. CRU doesn't have all the data, and can't reconstruct its own results. This idea has been floated since the emails came out. A large file describes many problems with their software, and the programs in the archive are pretty awful (more on that below.)

3. CRU doesn't have permission to release the data. Some of it is owned by weather services, and some of it comes from other researchers who have not given permission to release it. This is the explanation CRU has given.

There are two other reasons supported by the emails.

4. The CRU people regard Steve McIntyre and the other FOI requestors as useless pests who are just trying to score political points and waste their time with FOI requests.

Email 1228330629 (farther down in the reply chain):

At 01:17 03/12/2008, Ben Santer wrote to Tom Wigley:

One of the problems is that I'm caught in a real Catch-22 situation. At present, I'm damned and publicly vilified because I refused to provide McIntyre with the data he requested. But had I acceded to McIntyre's initial request for climate model data, I'm convinced (based on the past experiences of Mike Mann, Phil, and Gavin) that I would have spent years of my scientific career dealing with demands for further explanations, additional data, Fortran code, etc. (Phil has been complying with FOIA requests from McIntyre and his cronies for over two years). And if I ever denied a single request for further information, McIntyre would have rubbed his hands gleefully and written: "You see - he's guilty as charged!" on his website.

You and I have spent over a decade of our scientific careers on the MSU issue, Tom. During much of that time, we've had to do science in "reactive mode", responding to the latest outrageous claims and inept science by John Christy, David Douglass, or S. Fred Singer. For the remainder of my scientific career, I'd like to dictate my own research agenda. I don't want that agenda driven by the constant need to respond to Christy, Douglass, and Singer. And I certainly don't want to spend years of my life interacting with the likes of Steven McIntyre.

I hope LLNL management will provide me with their full support. If they do not, I'm fully prepared to seek employment elsewhere.

5. After academic teams gather the data (which takes a lot of effort), they want to spend a few years milking it for research papers. After all, it's the peer-reviewed papers in major scientific journals that make their reputation, not going out to the Arctic and taking cores from trees. Releasing the data before the papers have been written is like doing all the scut work for no reward.

At 16:50 02/10/2009, Malcolm Hughes wrote to Keith Briffa:

What's going on? 21st September I got an email from Tom M that contained the following para, among other more general discussion:

"Keith has been complained at by Climate Audit for cherry picking and not using your long Indigirka River data set. Not used because we did not have the data. Please, could we have the data? We will make proper acknowledgement/coauthorship if we use the data."

I replied pretty much straight away thus: "Hi Tom - please find the Esper article in question attached. The so-called Indigirka River data set is not yet available because it has not been published. I am currently working on that with Russian colleagues, and was indeed in Switzerland the week before last to work with one of them on specifically this. All being well, there will be an accepted manuscript before next summer, and at that point I will make the data freely available. Once we get to that point, I'll let you know, of course."
...
As you know, it is my intention to friendly, cooperative and open, but I'm determined to get some scientific value from all the years of work I've invested in the Yakutia work, and in cooperation with Russia in general. Releasing these data now would be too much.

It's not clear to me from reading the emails when the tree cores were done. Some of the messages make it sound like it was done in the 80s and 90s, which would be a long time to hold on to data. The email above makes it sound more recent.

Of course, this would all be acceptable if climate science were still just an academic exercise. It's not so defensible when trillions of dollars and a whole new energy infrastructure are on the line. You see that with a lot of the emails. They want to do science the traditional way, and aren't prepared to deal with investigations, FOI requests, making data and software public, and answering blogs. They don't seem to understand this strange urgency their critics have...

Also, CRU could have (and should have) just released what they had -- raw sea temperature records, the software used to adjust it, and their software for interpolating the data to cover the entire world. I think they didn't because it was not in very presentable form. To meet any kind of "publication" standard, lots of documentation would have to have been written, software cleaned up (and "tricks" documented), etc. It would have been a major piece of work.

Apparently, CRU is promising to do this now, with the results available in a few months. I think to answer their critics, they should dump all the rough-and-ready versions they have now out to the net. While they work on it, the wider community will be happy to sift through it all.

CRU is Corrupting the Peer Review Process

The CRU group has been accused of throwing their weight around to get papers by their critics rejected by scientific journals. Then they sneer at skeptics saying they have no published peer-reviewed papers.

There's some evidence in the emails that they tried, and some of the blogs make it sound like they succeeded to some extent. It's hard to decide exactly what they were thinking though. There are a range of explanations.

1. For a biologist, evolution is settled, basic science. If a research journal started running creationist papers, biologists would be appalled and try to get the editors fired, stop submitting papers, etc. You could regard the CRU people as reacting the same way. They think human-caused global warming (AGW) is settled, basic science and any contrary papers must be the work of cranks.

2. They could be willing to hear good arguments about their work, but don't think there are any, especially not from the "usual suspects", who they also think are second-raters doing bad science. So they could just be trying to get rid of bad papers. There are messages to this effect. They certainly are not complimentary to their critics. No "nice try, but wrong" sorts of messages. But no "wow, they almost caught us" messages either.

3. They could just be trying to silence critical research papers to defend their own work, not as scientific or political advocacy. A lot of the messages give the feeling they felt under siege, and were having to continually defend their integrity.

4. And it could be as the right-wing blogs make out, that they know they have a bad case and are just trying to shut up critics, to advance their own scientific/political agenda. There's a tiny bit of evidence of that. For example, this email:

   Email 1170724434:

   At 20:13 02/05/2007 Michael Mann wrote to Stefan Rahmstorg (and others):
   
   I find it terribly irresponsible for you to be sending messages like this to Singer and Monckton. You are speaking from ignorance here, and you must further know how your statements are going to be used. You could have sought some feedback from others who would have told you that you are speaking out of your depth on this. By instead simply blurting all of this nonsense out in an email to these sorts charlatans you've done some irreversible damage. shame on you for such irresponsible behavior!

   Fred Singer and Lord Monckton are both outspoken critics of global warming. The message Michael Mann is referring to (quoted in the link above) is very mild and doesn't deserve this kind of scolding. Other items on the net have described Michael Mann as "famously thin-skinned."

Reading the email, I think the biggest problem is that they have no "loyal opposition" internally, or even in the field -- people who would take warming seriously, but force them to validate their results more carefully. None of the CRU emails that I've read come across as "Devil's Advocate" sort of messages. It's mostly true believers there, and they are a bit defensive.

Trying to get reviewers fired, etc., is not anyone's idea of how science is done, but it's not clear to me that it has made any difference. Papers refused in prestige journals can still be submitted elsewhere. The problem is that no one seems to take any challenges to the basic idea of warming seriously at this point.

CRU is Unable to Reproduce its Own Results

In addition to the email, there is a "documents" directory, and it contains a file named "HARRY_READ_ME.txt". Speculation is that this is by Ian "Harry" Harris, a CRU member. It documents an attempt to get some of the CRU software and datasets in usable condition, after the original authors have left, or perhaps just after years of neglect. The comment "Argh!" can be read at many, many points.

You can go to open-source guru Eric S. Raymond's blog for scathing analysis of this code, ClimateGate issues, and global warming in general. Many sarcastic comments by programmers.

I'm not going into this in detail because this item is too long already. Here's what I take away from the discussion:

1. Academics who are not programmers take a course in Fortran (or learn it on their own), and never learn much programming beyond that. That means their code can be full of bugs, be very sloppy and slow, and be full of obsolete hacks learned years ago. Their programs are written for a specific purpose, like creating the graphs in a research paper, and then never updated. Chances are, they are never documented properly.

   "Harry" has been assigned to plow through a huge stack of this junk and figure out how it all works, what files these programs are supposed to run on, and generally tidy it all up. Very frustrating work!

2. Datasets are produced by multiple authors and can have very skimpy documentation. There's usually a line of column headings (all acronyms), then multiple lines of data. The data are a mix of printed numbers (ex: 3.1415) alphabetic codes (ex: CA, NY or FL) and "special values", like 99.999 for a missing value. The scale of numbers isn't documented and can vary from file to file (millimeters in one file, meters in another.) That doesn't mean that these datasets are all garbage, or that CRU is just making things up. It just means that the documentation is missing.

   "Harry" is clearly running various programs with different options, on different datasets, trying to find which combinations work and produce the graphs or tables used in their publications. When nothing works, he's forced to dive into the code to try and figure out what was intended. In some places, he's clearly gone from that to rewriting the code, trying to get it to generate output more or less like what was published.

3. CRU should have just published all of its data, all of its software, and documented everything to the last detail. That's the ideal. In practice, they are running this like an academic lab, where everyone keeps his own data, his own notes (or keeps it in his head), and if you need access to someone's data or figures, you ask him.

   Again, there's a mismatch between the way academics like to work, and the stakes of this issue. CRU has announced they are releasing everything, but it will take them months to get it all sorted out.

4. One big issue mentioned repeatedly is the comments in the software and email about "hiding the decline." From what I have read, this all applies to tree-ring data. The issue is that for the last 150 years, the tree rings haven't been tracking temperature well at all. This "divergence" in tree ring data is documented in the scientific literature, so it's not a CRU secret.

   However, the CRU people clearly thought this was an anomaly, not a fundamental problem with the data, and so they corrected for it. They either applied a correction to this data in their software, or they cut off the tree-ring series at arbitrary dates. That seems to be the source of the comments. Whether this is valid at all is another question. But it doesn't necessarily indicate deceit.

I'm not trying to give them a pass here. There's really no excuse for not documenting every step of the process. It's not good science, and it's certainly not good public policy to base a huge decision on science that can't even be documented properly, or reproduced by other teams.

CRU is Altering the Raw Data

A lot has been made of CRU "hiding the raw data" or "altering the raw data", or throwing it away after they've produced their "value-added" data. The fact is, there is no single source of pure "raw" data, and the raw data isn't usable anyway.

Datasets come in from many different sources and CRU "standardizes" them to put them all on the same temperature scale, so they can all be compared to one another, or fed into computer models that generate climate predictions. It's this "standardized" data that CRU keeps, and the original, unstandardized records that it has thrown away.

These same issues pervade all of the data. Satellite data is adjusted. Land and sea measurements are adjusted. Tree-rings and other "proxies" that go far back in time are very adjusted. They should have kept original sources (sea temperature logs, for example) and a description of what was done to each reading to standardize it. For tree rings, even the selection of which trees to core affects the "raw data". For ground temperatures, the siting of each thermometer (whether it's in a city, or next to an air-conditioning unit) is "raw data." It should all be online, with each number linked to software and a description and images. Dream on!

The original data is probably still out there somewhere (and it's not clear how much CRU has actually thrown away.) A team that wanted to replicate everything CRU has done would digitize it, code it for source, adjust it (with their own software) and produce a new temperature record. That would be a lot of work.

So did they really "alter" the data in the way that skeptics are claiming? I have two issues from reading the emails.

First, the use of tree-ring data seems very problematic. As I discussed in the Climatology section above, there are a list of questions about whether tree rings can be used to say anything about global climate. And as far as I can tell, trying to use this data is the source of all the "artificial" corrections in their software.

Second, there's this email that seems odd:

Email 1254108338:

At 23:25 on 09/27/2009, Tom Wigley wrote to Phil Jones:

Here are some speculations on correcting SSTs [Sea Surface Temperatures] to partly explain the 1940s warming blip.

If you look at the attached plot you will see that the land also shows the 1940s blip (as I'm sure you know).

So, if we could reduce the ocean blip by, say, 0.15 degC, then this would be significant for the global mean -- but we'd still have to explain the land blip.

I've chosen 0.15 here deliberately. This still leaves an ocean blip, and i think one needs to have some form of ocean blip to explain the land blip (via either some common forcing, or ocean forcing land, or vice versa, or all of these). When you look at other blips, the land blips are 1.5 to 2 times (roughly) the ocean blips -- higher sensitivity plus thermal inertia effects. My 0.15 adjustment leaves things consistent with this, so you can see where I am coming from.

Removing ENSO does not affect this.

It would be good to remove at least part of the 1940s blip, but we are still left with "why the blip".

Let me go further. If you look at NH [Northern Hemisphere] vs SH [Southern Hemisphere] and the aerosol effect (qualitatively or with MAGICC [their climate model for predicting future temperature]) then with a reduced ocean blip we get continuous warming in the SH, and a cooling in the NH -- just as one would expect with mainly NH aerosols.

The other interesting thing is (as Foukal et al. note -- from MAGICC) that the 1910-40 warming cannot be solar. The Sun can get at most 10% of this with Wang et al solar, less with Foukal solar. So this may well be NADW, as Sarah and I noted in 1987 (and also Schlesinger later). A reduced SST blip in the 1940s makes the 1910-40 warming larger than the SH (which it currently is not) -- but not really enough.

So ... why was the SH [Southern Hemisphere] so cold around 1910? Another SST problem? (SH/NH data also attached.)

It's hard to say what they are getting at here. Since he mentions the 1910-1040 warming (which is nearly as large as the 1980-2000 one), I doubt this is what he means by "1940s blip." Still, it does sound like they are trying to impose an artificial correction to the sea temperature data so that a smooth warming curve, without any inconvenient "blip" is shown on the result.

This is the opposite of starting with the data and looking for trends. This sounds like imposing a trend (or smoothing it, at least) on existing data. But without context, it's hard to be sure.

The Bottom Line

I have a lot of reservations about climate science and the quality of the data. The emails confirm some of those reservations. But does the ClimateGate archive really prove anything about the science?

The archive (probably leaked) is damaging to the people involved. It makes them look nasty, petty and defensive. It raises many suspicions about whether, as Global Warming True Believers, they have maybe found what they are looking for.

For example, when the tree ring data comes out "wrong", you could just keep processing it until it looks "right" -- that is, agrees with all the other data you have. Or you could take the portions of the record you think bolster your case, and ignore the modern parts that don't. If you Truly Believe, and have put a lot of work into gathering tree ring data, you are less likely to say "tree rings are worthless as records of climate" or "we just don't understand why these numbers diverge from other records."

It's not clear to me that CRU has really distorted the process much, or made a mess of climate research. I think the FOI and peer review issues are both overblown. They are very bad form, and perhaps legal trouble for some of the people involved, but they don't say much about the science of Global Warming.

Going forwards, I don't think this release will help the science much. Activists on both sides will dig in their heels and refuse to admit problems. No climate scientist will write an email saying what he honestly thinks of a paper or a person for awhile. In fact, I expect a lot of email has been deleted around the world this week!

Perhaps there will be more openness. I wouldn't be surprised if some scientific journals bend over backwards to publish some skeptical papers. But the bigger problem is that the science is "settled" in the minds of so many researchers. That is premature, in my opinion.

Let me finish with an analogy. You've probably seen these Discovery Channel documentaries on dinosaurs. In one, a guy with a silly hat and wild hair will get on and declare that T-Rex was a predator, no doubt about it! And he'll point to bones and teeth, and then say something like "anyone who says otherwise just isn't doing good science!"

Then the next guy with wild hair will come on and say that T-Rex was clearly a scavenger, and point to the absence of any fossil bite marks or trackways. He'll argue that T-Rex would have fallen over trying to chase something. And just look at the silly little arms! "Anyone who thinks T-Rex was a predator has watched too many movies."

This is all very amusing (I keep expecting it to turn into a Monty Python sketch), and matters to almost no one outside the perhaps 100 people in the world who have ever measured a dinosaur bone.

Now imagine that activists descended on these people, saying they were going to spend a few trillion dollars cloning a T-Rex, and absolutely had to know, right now if T-Rex was a predator or scavenger! Most of the scientists would freeze like deer in headlights. Some of them would mumble about uncertainties and lack of data, but the activists and politicians don't want to hear it.

Instead, the few True Believers would step forward and say "T-Rex has been proven by science to be a predator (or a scavenger.) I'll stake my scientific reputation on it!" Policy makers would pick the side that was politically popular. They would be influenced by the public, who think "of course, it's a predator -- I saw Jurassic Park!", and whatever industry concerns there were.

After a winner was picked by politicians, funding would dry up for the opposition. Without grants, they can't do their own fieldwork to get raw data. Without data, they can't write papers. Without publications, they can't get grad students. Without money or students, they can't get more data. The science would be declared "settled" and skeptical work would be shut down.

That, unfortunately, is what I think has happened to climate science. These groups are still acting like academics -- sniping at rivals, keeping minimal records and writing amateur software as if this were a purely academic issue. But the politics have taken over and the stakes are too high to continue business as usual.

Mike's Newspaper
Writings
Projects
Sea of Memes
Photos