The People Have Spoken: Poll Methodology Sound

Jenny Hayward-Jones
Wow!! This has really put the cat among the pigeons. I had to pinch myself when I read the news. And my guess is that whatever they say publicly, John Key and Julia Gillard will still be pinching themselves.

An independent poll has shown two-thirds of Fiji adults think Bainimarama is doing a “good job” as Prime Minister and that Fiji is heading in the “right direction.” And that from both major races. These were the opinions of 75% of Indo-Fijians polled and 60% of ethnic Fijians. In December 2008 only 48% thought so highly of Bainimarama and the ethnic divide (Indo-Fijians 64%; ethnic Fijians 35%) was quite wide. Nearly three years on, he has nearly doubled his support among ethnic Fijians.

Further, while most Fijians have warm feelings towards Australia and NZ, over eighty percent think they should lift sanctions and leave Fiji to sort out its own problems. Over 76% disagree with Fiji's suspension from the Commonwealth and 79% with its suspension from the Pacific Forum. So here we are with our PM and his Aussie cousin (whose polls shows her to have less than one-half of Bainimarama's support!) working towards a “democratic” outcome that the people of Fiji say they do not want.

This cannot be. The poll must be wrong. Bainimarama is a dictator; we are told he persecutes the Methodist Church, tramples on the trade union movement and infringes on the human rights of Fijians. Our media believes those who speak out against him like Pita Waqatiarewa of the Australian-based Fiji Democracy and Freedom Movement. Bainimarama, he says, “has ordered the detainment and beating of hundreds of people, the murder of others and has brought economic and social hardship on thousands of people he has forced into poverty.” Hyperbole, of course, but still believed by some.

Results faulty due to faulty methodology?
So, if we oppose the Bainimarama Government, what should we do with this new, unwelcome development? The first and most obvious thing is to dispute its results on the basis of faulty methodology. And this is precisely what Dr Steve Ratuva at Auckland University has done, claiming that “a researcher would have a field day” with the results. Well, I'm a researcher and I've been conducting sample surveys in Fiji since the 1970s. I've even written a book on research methods still used by postgraduate students in Fiji. Steve is a political scientist whose published writings analyze secondary data. I am uncertain about his survey experience.

I say this not to disparage Steve but to highlight my own expertise in Fiji, where different survey methods are needed than in New Zealand. Our more reliable surveys mainly use electoral and other rolls and registers, a sampling of telephone numbers or street addresses. These methods would be difficult to use in Fiji where many houses have no numbers and many households no telephone.

Steve has three complaints about the methods, all of which I think unfounded. First, he says the poll may capture people's perceptions but perceptions are not reality. He's right of course, but an opinion poll is about what people think; it is not about reality, and in this respect the Fiji poll is no different than any other. There are other ways to test reality.

Secondly, he talks about the so called “climate of fear” that could have influenced the results. His caution is understandable but there is no evidence that it did. Indeed, the poll organizers said people were open in their opinions, and the results show this to be so. One-third thought Bainimarama's performance was poor or only average. Hardly the level of support a real dictator would be happy with.

Lastly, he thought there could be problems with translation: “It's problematic to translate heavily-loaded political terms such as 'democracy' and 'elections' into Fijian and Hindustani.” The poll was conducted in English, Fijian and Fiji Baat; many questions did not deal with “loaded” abstracts but dealt directly with issues such as education, health, the Methodist Church, the military and the actions of foreign governments, and the poll organizer said a pilot survey, carried out before the main survey, tested the use of words in translation and changes were made where necessary.

While Steve reasonably questions research methods, for faulty method input will produce faulty result outcomes, there is also one other big concern: the independence and credibility of the organizers. The poll was so “independent” that its organisers were “concerned” and obviously disappointed with the results.

The Australian privately-owned think tank, the Lowy Institute sponsored the poll that was overseen by Jenny Hayward-Jones, the director of its Melanesian programme and a long-time critic of the Bainimarama government. It was conducted by Tebbutt Times, Fiji's most reputable and experienced pollsters, who set up business in 1992. They said they had used “international best practice.” In my view, their methodology and handling of the results were fair and reasonable. **

The Poll Methodology
Following on from the pilot survey that tested the questions to be asked, the survey proper involved 10-15 minute interviews (longer and interest may have flagged) in 1,023 households in the urban and the peri-urban areas of Nadi, Lautoka and Greater Suva. The survey can therefore only be taken to convey opinions in urban Viti Levu (though Rev Akuila Yabaki thought a survey of the outer islands would have shown even more support for Bainimarama).

The  households interviewed were chosen from randomly selected “starting points.” Ten percent of interviews were supervised by the organisers. This was a sample stratified according to age, sex, ethnicity and location. The proportion of those needing to be interviewed in each strata was determined from their proportions at the 2007 census, projected to 2011.  One person over the age of 18 in the selected households was chosen for interviewing.  and where the final stratifications were found to be under or over-represented, the results were weighted to produce the correct proportions.  

The organisers did not overstate the results. They said a truly representative random sample (their sample was the best that could be done in the Fiji circumstances but it was not “truly” representative) would have produced results that were correct +- 3 percent at the 95% probability level. In other words, a result of 50% would be between 47 and 53% accurate 95% of the time. Bainimarama's 66% “good job” result would therefore probably be between 63 and 69%. ***

Is the Lowy/Tebbutt poll accurate? As accurate as most polls, but that is not quite the question to ask because it would have to be very inaccurate for it not to be the most reliable indicator we have of public opinion in Fiji. Will it make any difference to our leaders' positions on Fiji? Not openly. That would mean a loss of face but they would be unwise to ignore it. We could hear a new tune from John Key on “no progress towards democracy” but again not any time soon.

The last word to Jenny Hayward-Jones: “For the first time we have a clear voice of the Fijian people. It is a voice to be heard.”
– Crosbie Walsh
** I notice Tebbutt credentials have been questioned because its principal Cez Tebbutt is a friend of Sharon Smith-Johns, the Permanent Secretary of Information.  This is par for the course in Fiji.  While it is conceivable that the friendship could have influenced the results (some would see it as a cheap shot. It could just as probably have resulted in extra caution), it would take a concerted effort involving several people to substantially alter the results, and  the possibility would not have escaped the attention of the Lowy Institute that hired Tebbutt and checked both the methodology and the results. 

*** Compare this with recent polls on Julia Gillard (70% of Australians are now disappointed with her performance) and Barack Obama (53% of Americans now disapprove of his leadership.) John Key fares much better with 57% seeing him as the preferred prime minister. It's not quite the same question that Fijians answered about Bainimarama, of course, but it's a useful indicator for comparison.


sister saras said…
The people have certainly spoken, and they will speak again in 2014. Fiji is in the best hands at the moment. A new Fiji is being born and the silent majority support the current admin. We really need not go any further. Those who want democracy, me included, there it is for you, the majority have spoken. Those who dispute this, saying that a fraction of a percent have been involved in the survey, should also question the 101% voter turnout in 2006 election. Now, that is called rigging.
Anonymous said…
What methodology was employed for this poll? Which company conducted the poll? You state that the methodology is sound but give no details?
Proud fijian said…
The Anti Regime forces are frothing in the mouth. Call Kiribati President a communist and Cuban supporter and dictator.

Thanks Croz for the article - well written.

Again well above the articles at C4.5 - full of rumors and scaremongering.
yes sir we love you said…
One of the answers in the poll was “Fifty-three per cent said the Fijian Government was doing a good job preparing to draft a new constitution”

Considering we are told time and time again that there will be no work on the constitution until next year that seems implausible.

It seems to be that over half the people taking part in the survey would say the Government is doing a good job no matter the question. One of the possible answers for that is people all feel intimidated to say good things about the government.
poll does not equal vote said…
Not sure about your heading Croz. It seems to suggest Frank is valid because the people shown support. You might be keen to know the former prime minister actually received even more support ! But polls don't validate self appointed leaders. Time for a election. But first the PER has to be lifted and military needs to stop the threats and questioning of people with alternate views.
Anonymous said…
Stiven Ratuva was my supervisor -temporarily, for a masters thesis and told me that i could go and interview all the population sampel ( about 780 people) for my research. later, this miscopnception was clarified by professor Ian Campbel who clarifed that only a percentage of the sample needed to be randomly selected to represent the overall population and this random sample was adequate for my research. Steven was 'not sure', 'check with that lecturer' , check in that publication... to me at the Masters student researching on poverty amongst the ethnic Fijian community within a specified demographic area, this attitude was enought to tell me about 'HOW MUCH STEVEN KNEW ABOUT QUALITATIVE RESEARCH TECHNIQUES'. Mayeb he is tryign to please the Government that he currently serves that he goes against what every secodn person is sayign on the streets in Fiji .. THAT THE BAINIMARAMA GOVERNMENT IS THE ONLY GOVERNMENT WITH GUTS AND BRAISN TO MOVE THE COUNTRY FORWARD DESPITE MANY ATTEMPTS BY THE POWER HUNGRY RICH I-TAUKEI AND OTHERS TO SABOTAGE HIS EFFORTS. Sure, not only God, but more than two third of the people fo Fiji are with him.....

Popular posts from this blog

Lessons from Africa

The Ratu Tevita Saga, Coup4.5, Michael Field, the ANU Duo, and Tonga