Last Saturday the Mobilize project hosted a day-long professional development meeting for about 10 high school math teachers and 10 high school science teachers. As always, it was very impressive how dedicated the teachers were, but I was particularly impressed by their creativity as, again and again, they demonstrated that they were able to take our lessons and add dimension to them that I, at least, didn’t initially see.
One important component of Mobilize is to teach the teachers statistical reasoning. This is important because (a) the Mobilize content is mostly involved with using data analysis as a pathway for teaching math and science and (b) the Common Core (math) and the Next Generation (science) standards include much more statistics than previous curricula. And yet, at least for math teachers, data analysis is not part of their education.
And so I was looking forward to seeing how the teachers performed on the “rank the airlines” Model Eliciting Activity, which was designed by the CATALYST project, led by Joan Garfield at U of Minnesota. (Unit 2, Lesson 9 from the CATALYST web site.) Model Eliciting Activities (MEA) are a lesson design which I’m getting really excited about, and trying to integrate into more of my own lessons. Essentially, groups of students are given realistic and complex questions to answer. The key is to provide some means for the student groups to evaluate their own work, so that they can iterate and achieve increasingly improved solutions. MEAs began in the engineering-education world, and have been used increasingly in mathematics both at college and high school and middle school levels. (A good starting point is “Model-eliciting activities (MEAs) as a bridge between engineering education research and mathematics education research”, HamiIton, Lesh, Lester, Brilleslyper, 2008. Advances in Engineering Education.) I was first introduced to MEAs when I was an evaluator for the CATALYST project, but didn’t really begin to see their potential until Joan Garfield pointed it out to me while I was trying to find ways of enhancing our Mobilize curriculum.
In the MEA we presented to the teachers on Saturday, they were shown data on arrival time delays from 5 airlines. Each airline had 10 randomly sampled flights into Chicago O’Hare from a particular year. The primary purpose of the MEA is to help participants develop informal ways for comparing groups when variability is present. In this case, the variability is present in an obvious way (different flights have different arrival delays) as well as less obvious ways (the data set is just one possible sample from a very large population, and there is sample-to-sample variability which is invisible. That is, you cannot see it in the data set, but might still use the data to conjecture about it.)
Before the PD I had wondered if the math and science teachers would approach the MEA differently. Interestingly, during our debrief, one of the math teachers wondered the same thing. I’m not sure if we saw truly meaningful differences, but here are some things we did see.
Most of the teams immediately hit on the idea of struggling to merge both the airline accuracy and the airline precision into their ranking. However, only two teams presented rules that used both. Interestingly, one used precision (variability) as the primary ranking and used accuracy (mean arrival delay) to break ties; another group did the opposite.
At least one team ranked only on precision, but developed a different measure of precision that was more relevant to the problem at hand: the mean absolute deviations from 0 (rather than deviations from the mean).
One of the more interesting things that came to my attention, as a designer or curriculum, was that almost every team wrestled with what to do with outliers. This made me realize that we do a lousy job of teaching people what to do with outliers, particularly since outliers are not very rare. (One could argue whether, in fact, any of the observations in this MEA are outliers or not, but in order to engage in that argument you need a more sophisticated understanding of outliers than we develop in our students. I, myself, would not have considered any of the observations to be outliers.) For instance, I heard teams expressing concern that it wasn’t “fair” to penalize an airline that had a fairly good mean arrival time just because of one bad outlier. Other groups wondered if the bad outliers were caused by weather delays and, if so, whether it was fair to include those data at all. I was very pleased that no one proposed an outright elimination of outliers. (At least within my hearing.) But my concern was that they didn’t seem to have constructive ways of thinking about outliers.
The fact that teachers don’t have a way of thinking about outliers is our fault. I think this MEA did a great job of exposing the participants to a situation in which we really had to think about the effect of outliers in a context where they were not obvious data-entry errors. But I wonder how we can develop more such experiences, so that teachers and students don’t fall into procedural-based, automated thinking. (e.g. “If it is more than 1.5 times the IQR away from the median, it is an outlier and should be deleted.” I have heard/read/seen this far too often.)
Do you have a lesson that engages students in wrestling with outliers? If so, please share!
