enter the fray: our reader discussion forum
Search in:
Advanced
View:FlatThreaded
Statistically significant? Nope. Therefore, worthless
by Sakura
+2/-1 Reply

Kinsley epically fails in this article by refusing to check to see if his results are statistically meaningful. Afterall, he is only looking at ten presidents (6 Reps, 4 Dems), so the sample size is very thin. His "conclusions" are something akin to saying "We flipped a red coin six times and got four tails. We flipped a blue coin four times and got three heads. Clearly, the blue coin is better if you like heads!". Obviously, any reasonable person can see the flaw with this argument - the result may well be random chance. Indeed, people have done these calculations, and not surprisingly, it is not close to to statistically meaningful. Therefore, any fair-minded person knows that trying to draw conclusions from such a limited set of data is invalid.

I will give Kinsley credit for doing a "one year offset", trying to compensate for the fact that a president's policies don't come into affect immediately. Unfortunately, this is wholely inadequate. The effects of presidents' policies persist for decades after they are passed. Today's economy is "Bush's" economy...it is an economy built by every president since FDR. Indeed, FDR's policies could arguably be having a bigger impact than Bush's! This has the unfortunate consequence of completely scrambling the data, making this type of analysis fairly worthless anyway.

A somewhat more interesting analysis is to look at how our economy does relative to the other G7 nations. We might be hurting now, but so is everyone else. Are we hurting more or less? If we do better than Britain during President A's slowdown but do worse than Britain during President B's slowdown, that might be fairly revealing. It helps to "cancel out" some of the noise caused by world-wide economic swings.

The long and short of it, however, is that there is no statistically-significant relationship between which party is in power and economic performance. Anything statement going behond this is no more than reading tea leaves.

Re: Statistically significant? Nope. Therefore, worthless
by Sanjait
I agree that this kind of analysis is kind of bogus, but I notice that in criticizing Kinsley for not checking if his results are statistically significant, you also didn't bother to check if the results are statistically significant. Where's the P-values? Are you sure they aren't significant? And before you even calculate those, how do you justify counting entire presidential terms of up to 8 years one sample, rather than using annual numbers?
Re: Statistically significant? Nope. Therefore, worthless
by Archarito

Your thoughts are right on. I can't help but feel Bush has taken orders throughout his presidency. When he was sitting in the classroom after the first plane hit on 9/11 it seems like he was thinking he was wishing it wasn't happening, but; he understood what it meant and what to expect.

I don't believe he and McCain are of the same cloth. Who does one trust going forward? That is the question. I strongly lean towards McCain/Palin believing NObama's imperatives are disasterous much as I would like to dream they would work, I know they won't therefore I do not trust him for proposing fake solutions just to get power for power's sake. NObama would be the epitomy of ABSOLUTE POWER CORRUPTS ABSOLUTELY!

I wonder who Colin Powell will support?

Re: Statistically significant? Nope. Therefore, worthless
by Sakura
I didn't want to get that technical. I have seen calculations of p-values for this general set of data, which come in around .25....far from signficance. I don't remember exactly what the assumptions and start/end dates were for that number, though. You are correct that it would be better to treat each 4-year term seperately, but that still only increases your sample size to 13. Therein lies the problem - sample sizes of thirteen are just not very useful for detecting minor changes with so many complicated factors.
Thin sample?
by northwoods

Should we go back to the Grant administration?

I doubt that the publicans of today would like to be compared to the Hoover administration, for example.

Or am I wrong?

Re: Statistically significant? Nope. Therefore, worthless
by Sanjait

P-values of .25 ... for what? We have a number of comparisons here. Which one is that for?

And I was not arguing we should use four year terms, I was asking you how you justify doing that rather than using each individual year, which would bump our sample size up to 48. If we are going to take this kind of analysis seriously, statistically, then we'd have to justify that.

I can think of a reason not to take it seriously though. Basically, each individual year or term isn't an independent variable. Years and terms are linked to past years and terms. The seeds of both good times and recessions are often sewn years in advance.

Re: Thin sample?
by Sakura

That's why you can't go too far back. At some point, the parties of the past no longer bear close relationship to the parties of the present. Also, if you go back far enough to include the Depression, the statistics from those years tend to overbear everything else due to the wild swings.

Re: Statistically significant? Nope. Therefore, worthless
by Sakura

Why not go to every minute, and really increase your sample size?

The problem with doing this is that your variables are no longer random. The president right now and the president one minute from now are not independant. Indeed, there is a supremely good correlation between the two, except for one every four years. Only then are your "dice rolled", giving you a chance to do do your statistics.

Re: Statistically significant? Nope. Therefore, worthless
by Sanjait

Year by year makes sense because that's the budgetary cycle we use in the United States. One could easily argue that they are discrete samples.

But of course, as you point ou and I already said, they aren't independant. But guess what ... either are economic conditions in every presidential term. The economic indicators are the independant variables here, but year to year they are correlated, whether that year boundary is a presidential term boundary or not. There's no more idependant re-roll of the (economic) dice every four years than there is every one, which is what I'm trying to point out to you.

You can't infer causality from a t-test when your samples are taken in serial, and that applies whether you are sampling minute by minute, year by year or every four years, which is what makes this whole exercise fundamentally unsound.

Re: Statistically significant? Nope. Therefore, worthless
by Sakura

Obviously, you have to make a number of simplifying assumptions to work this with kind of data. In the end though, an easy way to visualize it is to imagine dropping five blue (Democratic) balls randomly into 13 "presidential term" boxes, each of which contains a piece of paper with a random number from -0 to 6% (the economic growth rate) for that term. What are the odds that those five boxes contain an average of 4% or more? Less than half, obviously...but much greater than 5%.

It is fairly irrelevant anyway. Even if you did find a statistically robust relationship, the causation might very well run the other way: economic conditions most definitely effect who wins elections.

Re: Statistically significant? Nope. Therefore, worthless
by Sanjait

You still haven't given anything that justifies using four year samples instead of one year, other than that it helps your argument that we supposedly have a sample size problem. Neither one year nor four year sets are independent "die rolls" by any stretch of the imagination. I don't know why you can't concede that, because you correctly note this lack of independent sampling makes the whole exercise flawed, while still inexplicably sticking with the assertion that somehow there's a reason to base df on our independent variable rather than our dependent.

Either way though, Kinsley's assertion is basically unjustified, except the part where he admits it isn't strong evidence of causality.

View as RSS news feed in XML