Register to Comment
Page 1 of 12 12311 ... LastLast
Results 1 to 10 of 114
Like Tree144Likes
  1. #1
    statsman statsman is offline
    statsman's Avatar
    Join Date
    Feb 2011
    Posts
    51,183

    Opinion Polls: Caveat Emptor

    Every time there’s a new poll published, we on P.ie go into an entirely predictable pattern of posting:

    Great, accurate poll. My party is doing well.
    Crap, unreliable poll. My party is not doing well.

    To a certain extent, this is understandable, but could be avoided by standing back and asking a few simple questions.

    • What questions are asked?
    • When?
    • What is the sample size?
    • How was the sample selected (random, quota, self-selecting)?
    • What was the mode (face-to-face, phone, e-mail, other)?
    • What is the margin of error?
    • Who did it?
    • Who paid for it?

    If you want to know if a pot of soup needs more salt, you just need to stir it well and then taste a spoonful, not eat the whole pot. If you want to gauge the opinion of an entire population, you don’t have to ask everyone, just a representative sample. Quota samples will reflect the composition of the larger pot by pre-selecting quotas of groups to ask (gender, age, social class, geography, etc.) to fit the census breakdown along those lines. So, if 30% of the population are working-class women over 40, then the sample should reflect this. Random samples will just ask 1,000 (or whatever the sample size is going to be) people, and then adjust to reflect. So, if they find that only 200 of the sample are working-class women over 40 but it should be 220, then they weigh the responses of that 200 so that each of them counts as 1.1 person. Small representative samples will always be better than big, self-selecting ones.

    Then they work out the margin of error (MoE), which is always an approximation as it is based on an ideal situation. For example, statistical theory confirms that for a sample of 1,000 with a 100% response rate on a question where the population is split 50/50, the poll will have an MoE of +/- 3% 19 times out of 20. Clearly, this never happens, but the theory acts as a guideline for an MoE that isn’t too far off.

    Despite what some people think, genuine, serious polling companies have no interest in being wrong, so polls that are conducted by one of them are more likely to be right than polls conducted by randomwebsire.ru. Equally, most serious media organisations want to be as accurate as possible, so, like them or not, a poll commissioned by RTE is more likely to be accurate than one commissioned by a company that wants to sell a product.

    So, why do polls on, say, voter intention in Ireland, that are published within a week of each other often show very different trends? We’re back to these questions:

    • What questions are asked?
    • When?

    Clearly, the question asked will have a strong influence. Just think about the difference between being asked ‘Which party is currently doing the best job?’ and ‘Which party would you vote for if there were a GE tomorrow?

    Equally, time plays a role. Would you expect the same response rate the day before a big scandal as you’d get the day after?

    So, most polls are accurate enough to within the MoE for the questions that were asked at the time they were asked. If you don’t like the result, then the problem is probably more with you than with the poll. And always remember, polls are not predictors of election results, they are samples of how salty the soup is at the moment you test it.

    If you’d like more detail, this link is very useful:

    A Journalist
    Last edited by statsman; 19th April 2017 at 02:03 PM. Reason: Subject verb agreement.
    Sign in or Register Now to reply

  2. #2
    gerhard dengler gerhard dengler is online now
    gerhard dengler's Avatar
    Join Date
    Feb 2011
    Posts
    43,742

    Good effort for an OP. It makes a pleasant change here to read an OP which shows some thought and effort.
    Sign in or Register Now to reply

  3. #3
    hollandia hollandia is offline

    Join Date
    Sep 2012
    Posts
    19,526

    Quote Originally Posted by statsman View Post
    Every time there’s a new poll published, we on P.ie go into an entirely predictable pattern of posting:

    Great, accurate poll. My party is doing well.
    Crap, unreliable poll. My party is not doing well.

    To a certain extent, this is understandable, but could be avoided by standing back and asking a few simple questions.

    • What questions are asked?
    • When?
    • What is the sample size?
    • How was the sample selected (random, quota, self-selecting)?
    • What was the mode (face-to-face, phone, e-mail, other)?
    • What is the margin of error?
    • Who did it?
    • Who paid for it?

    If you want to know if a pot of soup needs more salt, you just need to stir it well and then taste a spoonful, not eat the whole pot. If you want to gauge the opinion of an entire population, you don’t have to ask everyone, just a representative sample. Quota samples will reflect the composition of the larger pot by pre-selecting quotas of groups to ask (gender, age, social class, geography, etc.) to fit the census breakdown along those lines. So, if 30% of the population are working-class women over 40, then the sample should reflect this. Random samples will just ask 1,000 (or whatever the sample size is going to be) people, and then adjust to reflect. So, if they find that only 200 of the sample are working-class women over 40 but it should be 220, then they weigh the responses of that 200 so that each of them counts as 1.1 person. Small representative samples will always be better than big, self-selecting ones.

    Then they work out the margin of error (MoE), which is always an approximation as it is based on an ideal situation. For example, statistical theory confirms that for a sample of 1,000 with a 100% response rate on a question where the population is split 50/50, the poll will have an MoE of +/- 3% 19 times out of 20. Clearly, this never happens, but the theory acts as a guideline for an MoE that isn’t too far off.

    Despite what some people think, genuine, serious polling companies have no interest in being wrong, so polls that are conducted by one of them is more likely to be right than polls conducted by randomwebsire.ru. Equally, most serious media organisations want to be as accurate as possible, so, like them or not, a poll commissioned by RTE is more likely to be accurate than one commissioned by a company that wants to sell a product.

    So, why do polls on, say, voter intention in Ireland, that are published within a week of each other often show very different trends? We’re back to these questions:

    • What questions are asked?
    • When?

    Clearly, the question asked will have a strong influence. Just think about the difference between being asked ‘Which party is currently doing the best job?’ and ‘Which party would you vote for if there were a GE tomorrow?

    Equally, time plays a role. Would you expect the same response rate the day before a big scandal as you’d get the day after?

    So, most polls are accurate enough to within the MoE for the questions that were asked at the time they were asked. If you don’t like the result, then the problem is probably more with you than with the poll. And always remember, polls are not predictors of election results, they are samples of how salty the soup is at the moment you test it.

    If you’d like more detail, this link is very useful:

    A Journalist
    Excellent post. Unfortunately, on poll threads there is very little analysis of the actual numbers, nor the methodology, nor the consistent variances between the various polling companies some give a detailed party by party breakdown, others lump everyone else into "others" for example.

    Because no one (or very few, I do and a few others do) actually analyses the trends, the swings etc, we get threads that descend into spamming, shilling and abuse.

    Unfortunately, only one like is available for the OP. And as a reward for your effort, here's a nice graph:



    (Note to self - add some trend lines by party for clarity next time)
    Sign in or Register Now to reply

  4. #4
    PBP voter PBP voter is offline

    Join Date
    Sep 2015
    Posts
    4,680

    People just need to remember that +or-3% can make a huge difference.

    FF 25%
    FG 25%
    SF 14%

    was the result last time.

    Technically a polling company could produce this before the election and be with the margin of error.

    FG 28%
    FF 22%
    SF 17%.

    Yet the reactionary media and plebs who cannot or will not listen will be telling us how wrong they got it.
    Sign in or Register Now to reply

  5. #5
    bokuden bokuden is offline
    bokuden's Avatar
    Join Date
    Sep 2010
    Posts
    7,146

    I think a broad average taken across all major polls over a period of time can give a general idea of how parties are doing. Individual polls are pretty meaningless TBH.
    Sign in or Register Now to reply

  6. #6
    Dame_Enda Dame_Enda is online now
    Dame_Enda's Avatar
    Join Date
    Dec 2011
    Posts
    35,164

    Irish polls tend to be more accurate in elections but not at all accurate in referendums. And I say that as a poll-junkie.

    I think the lesson of the US and UK elections are that White voters are voting along educational and to a lesser extent class lines. Meanwhile most non-White voters continue to primarily vote for the Left especially in the US because of the Lefts support for multiculturalism and pandering to Identity Politics. The state of US inner cities calls into question whether they are voting in their best interests though.
    Last edited by Dame_Enda; 19th April 2017 at 01:10 PM.
    Sign in or Register Now to reply

  7. #7
    Clanrickard Clanrickard is offline
    Clanrickard's Avatar
    Join Date
    Apr 2008
    Posts
    38,459

    Quote Originally Posted by gerhard dengler View Post
    Good effort for an OP. It makes a pleasant change here to read an OP which shows some thought and effort.
    Makes a change from his meaningless bland one liners.
    Sign in or Register Now to reply

  8. #8
    bokuden bokuden is offline
    bokuden's Avatar
    Join Date
    Sep 2010
    Posts
    7,146

    Quote Originally Posted by PBP voter View Post
    People just need to remember that +or-3% can make a huge difference.

    FF 25%
    FG 25%
    SF 14%

    was the result last time.

    Technically a polling company could produce this before the election and be with the margin of error.

    FG 28%
    FF 22%
    SF 17%.

    Yet the reactionary media and plebs who cannot or will not listen will be telling us how wrong they got it.
    I think that's broadly accurate.
    Sign in or Register Now to reply

  9. #9
    Levellers Levellers is offline

    Join Date
    Apr 2011
    Posts
    11,571

    Polls are not worth that much. The trend of polls over a period of say two years would be of some interest.

    Polls can be manipulated as well as they were before the last British general election when they deliberately under estimated the Tory vote in order to get the Tory vote out.
    Sign in or Register Now to reply

  10. #10
    locke locke is offline

    Join Date
    May 2007
    Posts
    6,192

    Quote Originally Posted by Dame_Enda View Post
    Irish polls tend to be more accurate in elections but not at all accurate in referendums.
    Possibly due to low turnout in referendums.
    Sign in or Register Now to reply

Page 1 of 12 12311 ... LastLast
Sign in to CommentRegister to Comment