The People Have Spoken: Poll Methodology Sound

Jenny Hayward-Jones

Wow!! This has really put the cat among the pigeons. I had to pinch myself when I read the news. And my guess is that whatever they say publicly, John Key and Julia Gillard will still be pinching themselves.

An independent poll has shown two-thirds of Fiji adults think Bainimarama is doing a “good job” as Prime Minister and that Fiji is heading in the “right direction.” And that from both major races. These were the opinions of 75% of Indo-Fijians polled and 60% of ethnic Fijians. In December 2008 only 48% thought so highly of Bainimarama and the ethnic divide (Indo-Fijians 64%; ethnic Fijians 35%) was quite wide. Nearly three years on, he has nearly doubled his support among ethnic Fijians.

Further, while most Fijians have warm feelings towards Australia and NZ, over eighty percent think they should lift sanctions and leave Fiji to sort out its own problems. Over 76% disagree with Fiji's suspension from the Commonwealth and 79% with its suspension from the Pacific Forum. So here we are with our PM and his Aussie cousin (whose polls shows her to have less than one-half of Bainimarama's support!) working towards a “democratic” outcome that the people of Fiji say they do not want.

This cannot be. The poll must be wrong. Bainimarama is a dictator; we are told he persecutes the Methodist Church, tramples on the trade union movement and infringes on the human rights of Fijians. Our media believes those who speak out against him like Pita Waqatiarewa of the Australian-based Fiji Democracy and Freedom Movement. Bainimarama, he says, “has ordered the detainment and beating of hundreds of people, the murder of others and has brought economic and social hardship on thousands of people he has forced into poverty.” Hyperbole, of course, but still believed by some.

Results faulty due to faulty methodology?

So, if we oppose the Bainimarama Government, what should we do with this new, unwelcome development? The first and most obvious thing is to dispute its results on the basis of faulty methodology. And this is precisely what Dr Steve Ratuva at Auckland University has done, claiming that “a researcher would have a field day” with the results. Well, I'm a researcher and I've been conducting sample surveys in Fiji since the 1970s. I've even written a book on research methods still used by postgraduate students in Fiji. Steve is a political scientist whose published writings analyze secondary data. I am uncertain about his survey experience.

I say this not to disparage Steve but to highlight my own expertise in Fiji, where different survey methods are needed than in New Zealand. Our more reliable surveys mainly use electoral and other rolls and registers, a sampling of telephone numbers or street addresses. These methods would be difficult to use in Fiji where many houses have no numbers and many households no telephone.

Steve has three complaints about the methods, all of which I think unfounded. First, he says the poll may capture people's perceptions but perceptions are not reality. He's right of course, but an opinion poll is about what people think; it is not about reality, and in this respect the Fiji poll is no different than any other. There are other ways to test reality.

Secondly, he talks about the so called “climate of fear” that could have influenced the results. His caution is understandable but there is no evidence that it did. Indeed, the poll organizers said people were open in their opinions, and the results show this to be so. One-third thought Bainimarama's performance was poor or only average. Hardly the level of support a real dictator would be happy with.

Lastly, he thought there could be problems with translation: “It's problematic to translate heavily-loaded political terms such as 'democracy' and 'elections' into Fijian and Hindustani.” The poll was conducted in English, Fijian and Fiji Baat; many questions did not deal with “loaded” abstracts but dealt directly with issues such as education, health, the Methodist Church, the military and the actions of foreign governments, and the poll organizer said a pilot survey, carried out before the main survey, tested the use of words in translation and changes were made where necessary.

While Steve reasonably questions research methods, for faulty method input will produce faulty result outcomes, there is also one other big concern: the independence and credibility of the organizers. The poll was so “independent” that its organisers were “concerned” and obviously disappointed with the results.

The Australian privately-owned think tank, the Lowy Institute sponsored the poll that was overseen by Jenny Hayward-Jones, the director of its Melanesian programme and a long-time critic of the Bainimarama government. It was conducted by Tebbutt Times, Fiji's most reputable and experienced pollsters, who set up business in 1992. They said they had used “international best practice.” In my view, their methodology and handling of the results were fair and reasonable. **

The Poll Methodology

Following on from the pilot survey that tested the questions to be asked, the survey proper involved 10-15 minute interviews (longer and interest may have flagged) in 1,023 households in the urban and the peri-urban areas of Nadi, Lautoka and Greater Suva. The survey can therefore only be taken to convey opinions in urban Viti Levu (though Rev Akuila Yabaki thought a survey of the outer islands would have shown even more support for Bainimarama).

The  households interviewed were chosen from randomly selected “starting points.” Ten percent of interviews were supervised by the organisers. This was a sample stratified according to age, sex, ethnicity and location. The proportion of those needing to be interviewed in each strata was determined from their proportions at the 2007 census, projected to 2011.  One person over the age of 18 in the selected households was chosen for interviewing.  and where the final stratifications were found to be under or over-represented, the results were weighted to produce the correct proportions.  

The organisers did not overstate the results. They said a truly representative random sample (their sample was the best that could be done in the Fiji circumstances but it was not “truly” representative) would have produced results that were correct +- 3 percent at the 95% probability level. In other words, a result of 50% would be between 47 and 53% accurate 95% of the time. Bainimarama's 66% “good job” result would therefore probably be between 63 and 69%. ***

Is the Lowy/Tebbutt poll accurate? As accurate as most polls, but that is not quite the question to ask because it would have to be very inaccurate for it not to be the most reliable indicator we have of public opinion in Fiji. Will it make any difference to our leaders' positions on Fiji? Not openly. That would mean a loss of face but they would be unwise to ignore it. We could hear a new tune from John Key on “no progress towards democracy” but again not any time soon.

The last word to Jenny Hayward-Jones: “For the first time we have a clear voice of the Fijian people. It is a voice to be heard.”

– Crosbie Walsh

** I notice Tebbutt credentials have been questioned because its principal Cez Tebbutt is a friend of Sharon Smith-Johns, the Permanent Secretary of Information.  This is par for the course in Fiji.  While it is conceivable that the friendship could have influenced the results (some would see it as a cheap shot. It could just as probably have resulted in extra caution), it would take a concerted effort involving several people to substantially alter the results, and  the possibility would not have escaped the attention of the Lowy Institute that hired Tebbutt and checked both the methodology and the results. 

*** Compare this with recent polls on Julia Gillard (70% of Australians are now disappointed with her performance) and Barack Obama (53% of Americans now disapprove of his leadership.) John Key fares much better with 57% seeing him as the preferred prime minister. It's not quite the same question that Fijians answered about Bainimarama, of course, but it's a useful indicator for comparison.