Friday, November 5, 2010

A significant change of preference?

[Aside: I first posted this on my work research blog, condensed concepts, but thought I would put it here as well since it may be of interest to some readers here too].

I am puzzled by something. Suppose I ask five people if they prefer tea or coffee? Three say coffee, two say tea. Four years later I ask them again. One of the five people has changed their mind, now two prefer coffee and three tea. Suppose the sample size is much larger, but still only one in five people change their preference. I would not say there was a "massive swing" in preference or that drinks "preference did an almost complete turnabout".

Furthermore, suppose also that those five people had all previously said they did not wish to register as having a definite opinion on their preference. Then it should hardly be surprising if one in five changed their opinion.

Hence, I am puzzled by an article in the Wall Street Journal by Gerald F. Seib, Unaligned votes tilt rightward en masse. It states:
A massive swing by independent voters propelled the Republican Party to a series of key victories.... 
In House races nationally, Republicans won the votes of independents—voters who said they aren't affiliated with any party—by a 55% to 40% margin, a compilation of exit polls from across the country showed.
In other words, independents' preference did an almost complete turnabout over the last four years: They favored Democrats by 18 points then, Republicans by 15 points Tuesday.


  1. But suppose you chose your weekly meeting place based on people's preferences (it might be clearer if we said beer versus coffee, and the bar versus the coffee shop). One person's change of preference would take on greater significance, because it would impact the whole group.

    From that perspective, the swing feels bigger.

  2. Ross, I'm not sure that your '1 in 5' illustration is helpful here. If one person in a group of 5 people changes an opinion over 4 years, then you have a statistically insignificant sample of 5 with a single person changing their mind. If research was done that demonstrated that over a 4 year period there was a swing towards coffee from 2 in 5 to 3 in 5 Queenslanders then I would suggest that there had been some significant environmental change in Queensland. Wouldn't you?

    Also, the swing is a 33% one according to your 18 point to 15 point comment, am I right? If I'm not getting something completely wrong, that's one in three, not one in 5.

    Or am I confused because it's late at night?

  3. I would... Which makes me wonder whether I've missed something also.

    Sure, from a group of 5 I wouldn't be at all surprised if 1 changed their preference. I don't know the factors in these people's lives, so as far as I'm concerned it's a random fluctuation. If there were 1 million groups of 5 people though, I would expect, if it were indeed a random fluctuation, that (roughly) for every group where one person swung from coffee to tea, there'd be another group with one person swinging the other way, and the final percentage swing would be (roughly) nil. (And the same argument applies even if you only include people who like both "pretty much equally").

    So if I had a million groups of 5 and in every single group 1 person swung toward tea, that would disprove my 'random fluctuation theory', and I'd have to admit that there has been a significant swing.

    Of course, all I'm doing is elucidating Kutz's 'statitiscally insignificant' comment. Which makes me think I must have missed something as well!