It is a happy day to be a statistician, as bloggers and columnists are bragging about many correctly predicted victories in an age in which traditional survey methodologies have been made out of date. Mark Blumenthal at the Huffington Post reminds us that one role of statistics is to temper personal bias. He gives a shout out to several pollsters, but I think Nate Silver at 538 is due special mention.
More than just a pollster, Silver is a great statistics educators. And readers of 538 have learned several important lessons that all Citizen Statisticians should know. For instance, that data-based decisions and predictions are based on models, and all models rest on assumptions. Silver examines and challenges these assumptions, and the success of his blog and the success of his predictions results from his willingness to probe the robustness of the polls, and to examine the resulting scenarios. I’m not sure if other pollsters due this. (Blumenthal suggests that Huffington Post has software that automatically merges polls, but does not suggest that they check the assumptions underlying the various polling models.)
Silver’s willingness to talk about the assumptions behind the models and to explicitly address his methodology is a type of transparency that we should all practice. We talk much about data transparency–which is important–but transparency in analysis is equally important, and vital to scientific reproducibility. Really, it is a type of sharing, and isn’t that one of those lessons we learned in kindergarten?
This testing of assumptions is something we should all teach, at all levels. I fear I don’t do it well enough. Certainly, I teach the conditions/assumptions underlying statistical models, but I’m not sure I do enough to show students what goes wrong if assumptions fail. Sometimes I’ll say a few words, but students need the experience of making failed predictions to understand the importance of assumptions. And they need the tools to tune their models to adjust to inappropriate assumptions.
Incidentally, Silver’s new book (which should be flying off the shelves after last night) has a chapter about political pundits. Pundits are not paid to be correct; they are paid to be provocative. And last night shows that they’ll keep their paychecks.
