Next up, there is a stream of submissions to Nature: a comment by M&M, call it MM04a, a reply (call it MBH04), and a revised comment by M&M (call it MM04b). If MBH revised their reply, I haven't seen it. In the end, the exchange was rejected. Some of the referee reports are reproduced at McIntyre's website: the ones selected to be presented seem mixed but cautiously positive, the final referee reports and the rejection letter are not published. So what do these papers say?
MM04a accepts that MM03 deleted certain data series, and then argues that they are of poor quality, and ought not to have been included in the first place. I have two issues with this train of thought: first off, any evaluation of the quality of raw data which occurs after you've already assessed its impact on your hypothesis risks introducing your own bias into the analysis, which is about the biggest no-no in science. I also have a difficult time accepting McIntyre and McKitrick as authorities on data quality issues in tree rings; these kinds of considerations are better taken up by the people who took the data in the first place. The usual pattern, if you believe that existing data are low quality, is to take better data yourself, and then show that your data is a better explanation of nature.
My second problem with this is a pure data analysis issue. If the data is poor proxy for climate, as MM04a argues, then this will turn up in the calibration. Noise will be a poor fit to the 20th century instrumental records, do a poor job reconstructing 19th century temperatures, and will have little impact on the backpredicted earlier temperatures. In other words, the poor quality of the data will be established by the data analysis itself, and contrariwise, its excellence will also be so established.
MM04a introduces for the first time a complaint with the analysis procedure, in particular the preprocessing before taking principal components on the North American treeline database. This seems to be the main argument that gets carried forward these days, so it's worth spending some time on.
Principal component analysis (PCA) is a way of trying to automatically extract some patterns from a dataset. The dataset is a set of vectors in a high dimensional space, but you won't go far wrong by just imagining them as a cloud of points. PCA selects a particular rotation of the cloud: that's it, that's all it does. The idea is to try and get the "most important" directions, for some value of "most important", lined up along particular axes. PCA defines importance as "explaining the variance", which is to say distance from the origin. Now, clearly there are a number of choices to make here. Do we center the cloud first? Are all directions of the cloud in the same units, and have similar values, or do we want to scale them first? How should we scale them? Our answers to these questions will determine which particular rotation of the data we produce.
Now, the purpose of applying PCA to the North American treering data is that there are many highly correlated series, so there aren't really 70-odd independent datapoints here. The idea is to extract the dominant features, correlate 'em to the 20th century data, and forget the rest. Now we need to make choices about what we see as "dominant features". MBH98 says, hey, we're correlating to 20th century temperatures here, let's normalize around the 20th century, do our PCA on the 20th century, then correlate and drop components once they stop contributing.Given this context, that sounds pretty reasonable.
Wait a minute, says MM04. You're effectively choosing which patterns to emphasize. And look: the hockey-stick shape appears in PC1, the first principal component. And watch, if I apply the same algorithm to random data, I can also see hockey sticks. Therefore, the entire hockey-stick picture is bunk.
The problem with this point of view is that the shapes of the various PC's have no significance, except insofar as they correlate to 20th century temperatures. What do I care about patterns in North American tree rings? Nothing: I'd never even heard of them before starting reading this stuff. The whole point is to use them as temperature proxies, so as to learn about past climate. Some rotations will bring out the pattern of correlation earlier than others, but the data remains the same, and the correlation will come out, sooner or later. In a sense, MM are correct that the procedure is biased to produce hockey sticks, but the first principal component is not the final purpose of the procedure. The true bias is provided by the fit to 20th century temperatures, which show unambiguous warming.
Myself, I probably wouldn't have done it this way. Since I don't care about the data patterns per se, only correlations with an output variable, I'd use some technique designed for that purpose. Partial least squares is one, with a few variants available. But if it matters, you're doing something wrong.
The response, MBH04, says there's nothing wrong with the data. The complaint about PCs they address, in addition to bloviating, by simply redoing the entire calculation using the entire database (as opposed to the first few PC). The result stays the same, but the cross-validation statistics are slightly worse: exactly what you'd expect if the leading PCs after their procedure captures the leading behavior of climate during the calibration period. It should be clear, from the proceeding discussion, that this is pretty determinative. There is no rotation, and no selection of PC's at all. If you want to avoid the conclusions of MBH98, you have to get new data, or remove the old data.
MM04b, the updated comment, mostly repeats the same points as MM04a, and does not seem to address the direct criticisms of MBH04. Two things are new. One is an off-hand comment that another data series, Gaspe cedars, is also fairly high leverage. This may be damning with faint praise, but it's the first time I've seen that M&M undertake independent examination and analysis of the data itself, as opposed to merely attempting to repeat the existing analysis. The other is the first appearance of a cross-validation calculation. The numbers concur with MBH04's conclusion: dropping the data results in worse cross-validation. Unfortunately, MM04b get the conclusion inverted (better validation means better data, not the reverse) but it's encouraging to see that they are attempting validation in the first place. Nevertheless, I'm not surprised the exchange was rejected. It's things like this which I pay the editors of Nature to keep me from wasting my time on.
Perhaps the biggest observation about MM04 is what is missing. Unlike MM03, MM04a and MM04b do not claim to produce an alternative reconstructed temperature with a 15th century warmer than the present, they just attack the MBH reconstruction.
Stay tuned for my analysis of the GRL 05 paper, McIntyre and McKitrick's first ever contribution to the peer-reviewed scientific literature, and one of sufficient importance to merit a front page article on the Wall Street Journal and attention from Senator Barton's committee on transportation.
Danielle and I have given our bit to Katrina relief, but I keep thinking that there's more that could be done.
For example, right now there are a lot of people who own underwater real estate and would like to have some or all of the value of it to buy food, clothing, etc. And there are quite a few banks who lent money on 80% of the value of land which is now underwater and buildings which are now demolished. And there are probably a fair number of people with a lot of capital who wouldn't mind picking up some New Orleans real estate cheap -- in the understanding that if New Orleans is not rebuilt, their investment is a dead loss (Or perhaps an attractive yacht berth?)
Obviously insurance is the first great complication. In fact, insurers already committed to the region represent the first wave of gamblers/speculators to invest in the New Orleans flood, before it was even flooded. As far as I know, conventional home insurance will cover replacement costs for the structure, but not remediation of the land. Flood insurance may in part cover those costs. Perhaps it would be worthwhile to hold on to your waterlogged title deed until all the numbers are in, and then decide whether to take the money or walk away from the property.
But nevertheless, there may be people who are willing to sell out, chuck it all, and go live in Colorado or somewhere equally altitudinous -- and transfer the risk to someone more willing to handle it. Sadly, if this was going on in the open, I expect it would be called profiteering and shut down. So it will happen in semi-secret, and the homeowners will get a worse deal -- both because of less competition and because the speculators must manage the additional risk of being thrown in jail.
I'm not saying that I want to go around buying underwater property from flooded-out homeowners. In fact, I would have serious moral qualms about doing it. But the government should not take it upon itself to shield all of the people from New Orleans from "profiteers" just because some of them may be distraught.