There are two main obstacles students have to overcome to learn statistics: one mathematical one philosophical. The math is difficult, and will be new to a high schooler, and (philosophically) it is rarely obvious what is the true, underlying cause and what is the random accident behind the statistical variation. This philosophical confusion (cause and effect, essence and accident) is a background confusion in the work of even in the greatest minds. Accepting and dealing with it is the root of the best research, separating it from blind formula-following, but it confuses the young who try to understand the subject, The young student (especially the best ones) will worry about these issues, compounding the difficulty posed by the math. Thus, I’ll try to teach statistics with a problem or two where the distinction between essential cause and random variation is uncommonly clear.
A good case to get around the philosophical issue is gambling with crooked dice. I show the class a pair of normal-looking dice and a caliper and demonstrate that the dice are not square; virtually every store-bought die is uneven, so finding an uneven one is not a problem. After checking my caliper, students will readily accept that after enough tests some throws will show up more often than others, and will also accept that there is a degree of randomness in the throw, so that any few throws will look pretty fair. I then justify the need for statistics as an attempt to figure out if the dice are loaded in a case where you don’t have a caliper, or are otherwise prevented from checking the dice. The evenness of the dice is the underlying truth, the random part is in the throw, and you want to grasp them both.
To simplify the problem, mathematically, I suggest we just consider a crooked coin throw with only two outcomes, heads and tails, not that I have a crooked coin; you’re to try to figure out if the coin is crooked, and if so how crooked. A similar problem appears with political polling: trying to figure out who will win an election between two people (Mr Head, and Ms Tail) from a sampling of only a few voters. For an honest coin or an even election, on each throw, there is a 50-50 chance of throwing a head, or finding a supporter of Mr Head. If you do it twice, there is a 25% chance of two heads, a 25% chance of throwing two tails and a 50% chance of one of each. That’s because there are four possibilities and two ways of getting a Head and a Tail.
After we discuss the process for a while, and I become convinced they have the basics down, I show the students a Pascal’s triangle. Pascal’s triangle shows the various outcomes and shows the ways they can be arrived at. Thus, for example, we see that, by the time you’ve thrown the dice 6 times, or called 6 people, you’re introduced 64 distinct outcomes, of which 20 (about 1/3) are the expected, even result: 3 heads and 3 tails. There is also only 1 way to get all heads and one way to get all tails. Thus, it is more likely than not that an honest coin will not come up even after 6 (or more) throws, and a poll in an even election will not likely come up even after 6 (or more) calls. Thus, the lack of an even result is hardly convincing that the die is crooked, or the election has a clear winner. On the other hand there is only a 1/32 chance of getting all heads or all tails (2/64). If you call 6 people, and all claim to be for Mr Head, it is likely that Mr Head is the favorite. Similarly, in a sport where one side wins 6 out of 6 times, there is a good possibility that there is a real underlying cause: a crooked coin, or one team is really better than the other.
And now we get to how significant is significant. If you threw 4 heads and 2 tails out of 6 throws we can accept that this is not significant because there are 15 ways to get this outcome (or 30 if you also include 2 heads and 4 tail) and only 20 to get the even outcome of 3-3. But what about if you threw 5 heads and one tail? In that case the ratio is 6/20 and the odds of this being significant is better, similarly, if you called potential voters and found 5 Head supporters and 1 for Tail. What do you do? I would like to suggest you take the ratio as 12/20 — the ratio of both ways to get to this outcome to that of the greatest probability. Since 12/20 = 60%, you could say there is a 60% chance that this result is random, and a 40% chance of significance. What statisticians call this is “suggestive” at slightly over 1 standard deviation. A standard deviation, also known as σ (sigma) is a minimal standard of significance, it’s if the one tailed value is 1/2 of the most likely value. In this case, where 6 tosses come in as 5 and 1, we find the ratio to be 6/20. Since 6/20 is less than 1/2, we meet this, very minimal standard for “suggestive.” A more normative standard is when the value is 5%. Clearly 6/20 does not meet that standard, but 1/20 does; for you to conclude that the dice is likely fixed after only 6 throws, all 6 have to come up heads or tails.
From xkcd. It’s typical in science to say that <5% chances, p< .05. If things don’t quite come out that way, you redo.
If you graph the possibilities from a large Poisson Triangle they will resemble a bell curve; in many real cases (not all) your experiential data variation will also resemble this bell curve. From a larger Poisson’s triange, or a large bell curve, you will find that the 5% value occurs at about σ =2, that is at about twice the distance from the average as to where σ = 1. Generally speaking, the number of observations you need is proportional to the square of the difference you are looking for. Thus, if you think there is a one-headed coin in use, it will only take 6 or seven observations; if you think the die is loaded by 10% it will take some 600 throws of that side to show it.
In many (most) experiments, you can not easily use the poisson triangle to get sigma, σ. Thus, for example, if you want to see if 8th graders are taller than 7th graders, you might measure the height of people in both classes and take an average of all the heights but you might wonder what sigma is so you can tell if the difference is significant, or just random variation. The classic mathematical approach is to calculate sigma as the square root of the average of the square of the difference of the data from the average. Thus if the average is <h> = ∑h/N where h is the height of a student and N is the number of students, we can say that σ = √ (∑ (<h> – h)2/N). This formula is found in most books. Significance is either specified as 2 sigma, or some close variation. As convenient as this is, my preference is for this graphical version. It also show if the data is normal — an important consideration.
If you find the data is not normal, you may decide to break the data into sub-groups. E.g. if you look at heights of 7th and 8th graders and you find a lack of normal distribution, you may find you’re better off looking at the heights of the girls and boys separately. You can then compare those two subgroups to see if, perhaps, only the boys are still growing, or only the girls. One should not pick a hypothesis and then test it but collect the data first and let the data determine the analysis. This was the method of Sherlock Homes — a very worthwhile read.
Another good trick for statistics is to use a linear regression, If you are trying to show that music helps to improve concentration, try to see if more music improves it more, You want to find a linear relationship, or at lest a plausible curve relationship. Generally there is a relationship if (y – <y>)/(x-<x>) is 0.9 or so. A discredited study where the author did not use regressions, but should have, and did not report sub-groups, but should have, involved cancer and genetically modified foods. The author found cancer increased with one sub-group, and publicized that finding, but didn’t mention that cancer didn’t increase in nearby sub-groups of different doses, and decreased in a nearby sub-group. By not including the subgroups, and not doing a regression, the author mislead people for 2 years– perhaps out of a misguided attempt to help. Don’t do that.
Dr. Robert E. Buxbaum, June 5-7, 2015. Lack of trust in statistics, or of understanding of statistical formulas should not be taken as a sign of stupidity, or a symptom of ADHD. A fine book on the misuse of statistics and its pitfalls is called “How to Lie with Statistics.” Most of the examples come from advertising.