Margin of Error

[stripey cat]

In most polls the margin of error is given as three per cent.

Taking this into account, it's hard to see anything exciting happening in levels of party support this year.

For example look at tomorrow's poll in the Irish Times:

Fianna Fáil are up 3% since June to 24%, while Fine Gael drop 3% and are also on 24%. Labour have gained 4% to take a commanding lead on 33%.

RT News: FG drops further behind Labour - poll

I reality, this means that Labour can definitely be said to have risen, very slightly, in popularity, and that's all it means. The other parties may well have risen or fallen, but just as easily they may not have moved at all.

Last Sunday the Business Post published the latest in a series of Red C polls, and gave us a graphic of the previous polls this year:


If you exclude the first column, which shows support from the 2007 election, you can see that most parties aren't seeing much actual change in their support, once you take into account the margin of error.

For instance, look at Fine Gael's support since March. They never get very far from around 33 per cent, once you realise that each poll could be out by a margin of three points. They are effectively stuck, as are most of the other parties. The only party showing any movement is Labour, and even that seems to have leveled off in the last few months.

Why does the media get so excited about polls, when the numbers aren't changing that much at all?


.

[baldur0300]

The latest MRBI poll has FG at 24% and Lab at 33%. An MRBI poll in January had FG at 32% and Lab at 24%. Now you can flatten those numbers all you want by invoking the margin of error, but an 8 point drop for one party and a 9 point increase increase for another in 9 months is very much statistically significant.

[stripey cat]

The latest MRBI poll has FG at 24%, i.e. anywhere between 21 and 27.

Lab is at 33%, that is, anywhere between 30 and 36.

An MRBI poll in January using a different system had FG at 32%, that is, anywhere between 29 and 35, and Lab at 24%, i.e. anywhere between 21 and 27.

So realistically, using these figures Labour could have been on 27 in January and be on 30 now, and FG could have been on 29 in January and be on 27 now.

[cyberianpan]

Most posters here do not understand margin of error


Consider this simple example, for a "2 horse race", which shows the probability of the leading candidate winning for various combinations of %age lead and margin of error.

http://www.politics.ie/elections/349...dumb-term.html


cYp

[GJG]

The latest MRBI poll has FG at 24%, i.e. anywhere between 21 and 27.

Lab is at 33%, that is, anywhere between 30 and 36.

An MRBI poll in January using a different system had FG at 32%, that is, anywhere between 29 and 35, and Lab at 24%, i.e. anywhere between 21 and 27.

So realistically, using these figures Labour could have been on 27 in January and be on 30 now, and FG could have been on 29 in January and be on 27 now.

You need to learn more about statistics. The threshold for certainty here is, I think, 95 per cent, but the great bulk of that 95 percent is at the centre of the range, which is at the figure quoted. That's why they quote it.

For a party to be right on the boundary of the margin of error, that is a one-in-twenty event. For what you are saying to be the case, then four separate one-in-twenty events would have to fall exactly the way you need them to.

The odds of that are not 160,000-to-1, because some of those events (Labour being oversampled) would overlap with others of them (FG being undersampled) but it's not far off it.

[Panopticon]

I reality, this means that Labour can definitely be said to have risen, very slightly, in popularity, and that's all it means. The other parties may well have risen or fallen, but just as easily they may not have moved at all.

This is not true. If FG falls by 2% in a poll with 3% margin of error, it is much more likely that they are less popular than that they are more popular or just as popular. It is also possible that FG fell by 5% or even 10% in total, though not very likely.

I don't have an intuitive explanation to hand, because I work with this stuff for a living and it's hard to explain probability to the layman. Canadian newspapers do a good job of explaining what MOE means in their polling articles.

You can disregard the "3%" part unless you care about either comparative polling or probabilistic analysis of dubious polls. The margin of error number is specific to relatively high poll ratings - obviously, a 2% figure in a 3% MOE poll can never mean true support of -1%!

[Harry Hayfield]

In 2007, FF polled 42% of all FPV vs 27% for FG and 10% for Labour. Am I correct in assuming that if a poll was published putting all three on 20%, then the MOE would suggest that all three could be as high as 23% or as low as 17% or as high as 21.5% or as low as 18.5%?

[hammer]

Greens could potentially have -1%, and even that is too high

[devnull]

In 2007, FF polled 42% of all FPV vs 27% for FG and 10% for Labour. Am I correct in assuming that if a poll was published putting all three on 20%, then the MOE would suggest that all three could be as high as 23% or as low as 17% or as high as 21.5% or as low as 18.5%?

It's difficult to say what effect one party having more support than polled has on another's chances because we have to worry about where the extra votes came from.
As an extreme example, since there are almost zero voters who are choosing between SF and FG, the two parties support levels might be completely independent of eachother. OTOH, FG and the PDs were fishing in the same pool and losses for one were always gains for the other.

Even if we ignore the problem of dependence, given the theoretical accuracy of Irish polls it's very unlikely that any of the 3 parties would be 3% above its poll numbers, never mind all 3 of them.

=======

The margin of error Irish polls usually give you is
"If we've done everything correctly then there should be
a 95% chance of the actual number being within plus or minus 3% of what we say
in the worst case scenario of the electorate being evenly split on this question".

Voters are rarely evenly split on questions, so the 3% MoE is usually a bit conservative
(the further a party's support is from 50%, above or below, the more accurate its poll numbers should be).

If just over 1000 people were polled and exactly 20.0% said they were going to vote for party X then _in_theory_ there would be
a 25% chance of X's actual support being between 19.6% and 20.4%,
a 50% chance of X's actual support being between 19% and 21%,
a 75% chance of X's actual support being between 18.5% and 21.5%,
a 90% chance of X's actual support being between 18% and 22%,
a 95% chance of X's actual support being between 17.5% and 22.5%,
a 99% chance of X's actual support being between 17% and 23%,
a 99.5% chance of X's actual support being between 16.5% and 23.5%,
a 99.99% chance of X's actual support being between 15.25% and 24.75%,
a 99.999% chance of X's actual support being between 14.5% and 25.5%.

Of course, there also are a load of judgement calls pollsters have to make about stuff like how likely people are to vote, how likely they are to lie, how skewed the data collection process was, etc. that have an unknown effect on the numbers.

Irish people tend to get too hung up on plus or minus 3% - it's a good way of giving an idea of how accurate a poll is, but shouldn't be taken too literally.

edit: P.S. in answer to your question, 20% with an MoE of 3% would give a range of 17% to 23%.

[holymoley]

It's difficult to say what effect one party having more support than polled has on another's chances because we have to worry about where the extra votes came from.
As an extreme example, since there are almost zero voters who are choosing between SF and FG, the two parties support levels might be completely independent of eachother. OTOH, FG and the PDs were fishing in the same pool and losses for one were always gains for the other.

Even if we ignore the problem of dependence, given the theoretical accuracy of Irish polls it's very unlikely that any of the 3 parties would be 3% above its poll numbers, never mind all 3 of them.

=======

The margin of error Irish polls usually give you is
"If we've done everything correctly then there should be
a 95% chance of the actual number being within plus or minus 3% of what we say
in the worst case scenario of the electorate being evenly split on this question".

Voters are rarely evenly split on questions, so the 3% MoE is usually a bit conservative
(the further a party's support is from 50%, above or below, the more accurate its poll numbers should be).

If just over 1000 people were polled and exactly 20.0% said they were going to vote for party X then _in_theory_ there would be
a 25% chance of X's actual support being between 19.6% and 20.4%,
a 50% chance of X's actual support being between 19% and 21%,
a 75% chance of X's actual support being between 18.5% and 21.5%,
a 90% chance of X's actual support being between 18% and 22%,
a 95% chance of X's actual support being between 17.5% and 22.5%,
a 99% chance of X's actual support being between 17% and 23%,
a 99.5% chance of X's actual support being between 16.5% and 23.5%,
a 99.99% chance of X's actual support being between 15.25% and 24.75%,
a 99.999% chance of X's actual support being between 14.5% and 25.5%.

Of course, there also are a load of judgement calls pollsters have to make about stuff like how likely people are to vote, how likely they are to lie, how skewed the data collection process was, etc. that have an unknown effect on the numbers.

Irish people tend to get too hung up on plus or minus 3% - it's a good way of giving an idea of how accurate a poll is, but shouldn't be taken too literally.

Great explanation of stats there. I think what confuses most people are questions related to - "who would you like to see as the next taoiseach" when people don't actually vote on this. Of course the leader of party is likely to influence who you vote for, it should already be taken into account of in the main polls.