Flutterby™! : Analysis

MIT Technology review gives an overview of Steven McIntyre and Ross McKitrick's claims that the analysis techniques of the famous "hockey stick" graph of global warming were flawed.

A while back I wrote some Principal Component Analysis routines for protein expression analysis software that Phil was playing with. Say you've got several thousand samples taken over some time period. PCA lets you take the difference in the samples in that space of thousands of dimensions and find the "vectors of most energy", giving you a way to start looking at large correlations across that space in fewer dimensions.

I've talked with a couple of people who've known what the technique is, probably known the math better than I, and yet who start asking questions which show that they clearly don't understand using it for data set analysis: "How many dimensions did you reduce the data to?" and the like. I don't want to cast unwarranted stones at Mann, Bradley, Hughes and those who've built on their work, but it is extremely important to be able to understand what the tool is doing, and not just assume that some abstract notion of what the tool is doing means it's sufficient for your purposes.