Monday, January 03, 2005

Fun with Surveys (part 3)

I say seems to have done as they don’t report Ns for any of their data other than the grand N of 715. Then they point out that they weight for party ID, race, and sex. Weighting is a neat statistical trick that can be quite helpful but can also cause big problems. In a simple nutshell, imagine you conduct a survey and of your hundred respondents 80 are men and 20 are women. Well, you know the proper split is 50-50. So you weight by four the female responses to get them to match the male N (it is actually more complicated than this but not by much). Click read more.

The problem with this, though, is that you are now reporting a sample size of 160 or a sample size of 80 women, when in fact the true sample size is 100 (20). So their grand N of 715 looks really impressive, but we don’t know what percentage was truly R or D, black or white, male or female. The male/female thing is not likely to be an issue. The R or D thing is a huge issue. Two of the central lessons of surveying uncovered in 2004 revolve directly around this issue.

First, what do we “know” is the proper weight of Rs and Ds? This issue of hang ups. It is commonly assumed by researchers that republicans are more likely to hang up than democrats and that they are harder to catch on the phone. If so, the Rs were weighted which means that they are undersampled to begin with. No problem if the Rs they didn’t catch are no different than the Rs they did catch, but that is a huge assumption. Next Page.


Post a Comment

<< Home

Listed on Blogwise