Can REDET poll claims be trusted?

Hans Hoogeveen and Tim Harris


For a few weeks now the country has been abuzz about the likely election outturns for the President and MPs, following the release of a REDET Opinion Poll on April 27. But how much confidence can we have in the results of an opinion poll of 2,600 people in a country of 38 million people? Can we trust the result that President Jakaya Kikwete is likely to be re-elected? Or the finding that most sitting MPs are in trouble?

All polls are estimates based on a small subset of the electorate. Intuitively more responses or a larger sample size increase the precision of a survey. If instead of asking few people, the entire population would be asked, the results would be very precise. This is precisely what elections do! But until the full election is held, we have to rely on the results of opinion polls like those of REDET. But how big a sample do we need in order to have confidence in its findings?

Suppose in a country with 10 million voters, there are 7 million supporters of the incumbent president. Then the true (but unobserved) support is 70%. If we take a random sample of 1,000 voters, we presumably find that the percentage of supporters of the president is close to 70%. But rarely would one find 70% exactly. If we took several samples of 1,000 persons, we are likely to obtain different values for the support for the president each time depending on which people happen to be selected in the random sample. Sometimes the support will be 68%; at other times it could be 73%. The range between 68% and 73% is called ‘sampling error’ – but we can be pretty certain the true result lies somewhere between these figures.

What does this imply for the REDET survey?  Basically the conclusion on the Presidential vote is a sound interpretation of the poll. But the survey is simply not big enough to make the claims about the prospects of individual MPs.

In the case of support for President Kikwete, REDET finds a 67% support. Taking the sampling error into account, the true support for the president is very likely to be between 65% and 69% (which is the sampling error range). But saying this may confuse those who are not statistically inclined. And, in any case, the conclusion that there is support for President Kikwete is the same whether the high or low estimate is used. So the researchers took the acceptable shorthand in stating that support for the President is 67%. In this case the sample size of 2,600 is large enough to enable REDET to make the claim it has made.

But the situation is completely different for the MPs. The researchers seem to have stretched the survey far beyond its technical limits. When sample sizes become small, sampling errors increase dramatically. The results for individual MPs are based on samples of only 50 respondents per district. This is simply too small to say much with any degree of confidence.

Take the case of ChakeChake for instance. The REDET researchers claim that 54% of the respondents would not re-elect their MP. But, with the sampling error, the percentage who do not support the sitting MP could be as low as 40% or as high as 68%.

When the range of possible results is so wide, it amounts to saying that we really don’t know the likely winner. In the case of the MPs, the REDET findings are statistically imprecise, and should be treated with a great deal of caution.

Hans Hoogeveen manages Uwazi InfoShop at Twaweza; Tim Harris is a statistician working as consultant from the UK.

Read more: Hans Hoogeveen



You might also like...