Let's Get Some Reality into Political Polls

pn526

The hoo-ha about the latest Colmer-Brunton and Newshub polls shows either that media commentators need to take a course in STATS101 and brush up on what can —and what cannot— be deduced from a poll, or that they need to be taken to task for deliberately misleading the public to sell copy, please advertisers, and put their spokes into the political process in favour of National.

One might expect such shortcomings from Opposition leader Simon Bridges ("What this shows is a government that isn't doing anything, that doesn't know what it's doing..") and the likes of Mike Hosking ("They  [Labour] are a mess. lots of noise, not a lot of action...NZ decides they are not up to it")  but not from an informed, balanced (and it has to be said privately-owned!) media whose  misleading dramatic headlines read:

National surges
Labour takes a tumble, heading downhill
Innate messiness of coalition politics
A seismic shift, erosion of support
Voters increasingly unhappy
National and ACT have numbers to form a government
National the winner, surges to highest level of support since 2017
National in winning position to form government
"Make Ardern go away" hats make a comeback on TradeMe
Double whammy of bad news (for Labour)

 Nothing of the sort has happened.

The "double whammy" is the Colmar-Brunton and Reid polls, commissioned respectively by NZTV1 and TV3's Newshub. The two polls were conducted at the same time.  Colmar-Brunton's 47% for National and Labour 40% was "significant"; Newshub's respective  43.9%  and 41.6% was not. A reinforcement of a trend perhaps, a possible contradiction, but not a separate whammy.

Colmar Brunton shows National just able to form a government, Newshub shows Labour hanging on.  But neither allows for the "Winston" factor that could change results if NZFirst crosses the 5% threshold.   

The "surges" and "erosion" reported by the media are the products of media inventiveness; not sound judgements based on statistical analysis. All the other preferred party results are meaningless because they could be due to variations in the poll samples.

For the second time in a year the Colmar Brunton poll results are (just) "statistically significant". Earlier, there was a "significant" increase for Labour following the Christchurch massacre; now the increase favours National following a flood of negative media reporting on Labour.  

The polls clearly demonstrate the massive role the media plays in influencing results.

What is  most disappointing is that  Jack Tame on state-owned TVOne's Q&A gave a ten-minute slot to Simon Bridges, no time to Labour,  and neither he nor his two commentators made any mention of that important word, margin of error. The poll results were taken as fact, and little mention was made of the "background" news or contradictory result (see my 4th point, below) that could have influenced, or made more sense of, the poll result.

There are several things to note about the political polls:

First, core voters always vote for the same party and do not influence poll results. Changes or trends occur because of "swing" voters, those who might vote this way or that and differently at the next poll.  In this case, they totalled 130 of the 1,003 people polled. And they are most influenced by the latest news!

Secondly, NZFirst was  excluded from the results because it polled below the 5% threshold. But past performance shows NZFirst to rally before elections. Had they been included in the polls, the results would have been very different.

The third  thing to note is that the poll also excluded the 18+ percent of eligible voters who pollsters hoped to interview but who said they did not know or refused to answer.  I'm not sure whether this is 180 of the sampled 1,003 or 212 if they are in addition to the sample, but either way that's a large number of people whose stance must detract from the accuracy of the poll result.  Both pollsters do excellent and impartial jobs with the data they obtain. But a poll is a poll based on a sample, and nothing more.

Fourthly, the media made much of Jacinda's apparent decline as preferred  PM  (down 3% to 38%, "the lowest since April 2017") but they did not comment on how strangely at odds this 38% was with  the party results. How could some 45% vote National while only 14% wanted  Simon or Judith as their Prime Minister, while 38% preferred Jacinda?

Most interestingly and barely commented on by the media is that for the first time Colmar Brunton introduced a PM approval rating, (in which disapprovals are subtracted from approvals) on  how well Jacinda and Simon handled their jobs. 

Jacinda rated +33 (better than Australia's Scott Morrison on +14) while Simon Bridges was on -22 (worse than Donald Trump's -16 and Boris Johnson's - 18).

Draw your own conclusions.

-- ACW