Do 38% of South Africa’s youth believe that the policies recently announced by Julius Malema’s Economic Freedom Fighters will win the party support? Would 26% of the country’s youth vote for the disgraced former ANC Youth League president’s new party? Do 42% of young people believe that foreigners have a negative impact on South Africa? And do as many as 52% of young black South Africans truly believe the Democratic Alliance will bring back apartheid if they win an election?
These are some of the startling claims made by the “consumer insights company”, Pondering Panda, which boasts that its use of “mobile technology” to conduct youth surveys allows it to “deliver much larger samples, faster and more cost-effectively than traditional research suppliers”.
Courting controversy

As a result, Pondering Panda has garnered hundreds of thousands of rands – possibly millions – in free publicity and has secured some of South Africa’s best-known brands as its clients. The company’s CEO, Butch Rice, has also made a name for himself with his highly critical and controversial comments about traditional research and polling methods and those who use them.
The research industry in South Africa is a "Jurassic Park", he was quoted as saying at a conference in May this year. "The traditional research industry still clings stubbornly to invalid analytical techniques, inefficient research designs, and cumbersome and expensive reporting, which takes an inordinately long time to deliver."
The tagline on Pondering Panda’s website promises: “A new wave of research. Finally.” The company boasts that its surveys have “faster turnaround times” with “typically 2,000 responses in a 24-hour period”.
But do these surveys accurately reflect the views of South Africa’s youth? Are they reliable? And are the media being critical enough when electing to publish their findings?
Opting-in to social media

What the press releases do not say is that all their research is done using an “opt-in system” that surveys users of the MXit social media platform. (It is an odd omission, given that the Pondering Panda website states that it has “exclusive access to MXit”, which it describes as “Africa’s biggest social network”.)
As an example, we decided to look a little more closely at the Pondering Panda survey which found that the “majority” of young blacks in South Africa believed the Democratic Alliance (DA) would bring back apartheid if they came to power.
The survey gave the company headlines, and lots of free publicity, after its release in April this year. Reporters who wrote about it made little or no attempt to seek further comment or to interrogate and contextualise the findings. Entire paragraphs were lifted from the Pondering Panda press release or attributed to its spokesperson.
Pondering Panda states that “full results are available on request”. When Africa Check requested the results of the DA survey, we were sent a spreadsheet document. It indicated that the survey was conducted over six days and that there were 3,009 respondents aged between 15 and 34. It did not give a detailed explanation of how the data was collected and weighted. The only mention of MXit in the document was the statement: “All data gathered over MXit social network”.
The devil in the detail

Herein lies a problem: MXit, like many social media tools, does not require people to use their real names when signing up. Many don’t.
Rice claimed Pondering Panda had checks and balances in place to ensure the platform yielded “valid and reliable data”, including checking a respondent’s identity against details they supplied when signing up to MXit and cross-comparing it to information supplied when they complete surveys.
How effective these checks and balances are remains unclear. An Africa Check researcher signed up for MXit under a false name, provided a false date of birth, downloaded the Pondering Panda app and completed a number of surveys. One of them asked for details on race and geographic location. The Africa Check researcher claimed to be a black male, aged 18 to 24, from the Northern Cape. (The age correlated with the date of birth given on signing up.) The researcher is, in fact, a 38-year-old white male from Gauteng.
A further problem is that there is some confusion about the number of active MXit users in SA. Pondering Panda claims on its website that MXit has 6.5-million active users in SA. But stats published by MXit in October 2012 stated there were almost 50-million registered users and 9,346,806 active users. Yet six months later, in March 2013, Vincent Maher, vice president of growth and product strategy at MXit, said the “latest active users’ statistics” for South Africa was 6.3-million per month.
Questions and answers

The question is problematic because different people will interpret it differently, said Derek Luyt, a researcher, analyst and director of the Centre for Accountable Governance.
For instance, some respondents may think they are being asked whether it is true that “some people” think the DA will bring back apartheid, not whether they themselves hold that view. Respondents could also have very different ideas about what is meant by the word “apartheid”. Some of them were probably born after apartheid was scrapped, or were too young to remember exactly what it was like.
“It’s not quite as bad as the old, ‘Do you prefer apartheid or socialism?’ question, but it’s not a question that is particularly meaningful either,’ Luyt told Africa Check.
"Unquestioning" journalists

Rice, a co-founder of Research Surveys, now TNS South Africa, lectured in statistics and marketing research at the University of Cape Town for 10 years. He disparages “just about all” other research methods. Despite questions about the accuracy of data collection on MXit, compared to more conventional door-to-door surveys where people are interviewed face-to-face, he told a marketing conference: “The future of market research is digital, and the digital future is mobile. It is an indictment of market research in South Africa that companies are still investing in door-to-door interviews and focus groups”.
Luyt said he felt that “a large part of the problem is the unquestioning attitude of the media”. “Instead of dissing others, Pondering Panda would probably do the media more of a favour by trying to enlighten journalists about research methods rather than trying to dazzle them with sexy headlines.
"There is a definitely a place for the type of surveys Pondering Panda does, but it is also important to accept the limitations and not to claim to do more than you actually do. You cannot compare it to the work of researchers who do deeper, longer-term research and try to understand things in greater depth and detail.”
Media consultant Gordon Muller believes that research using social media platforms is becoming increasingly important for the quick insights they give.
“To dismiss all other kinds of research as irrelevant and wasteful is wrong...They all have a place. From a macro point-of-view, using a dipstick approach is where social media comes in. But I would urge caution to any researcher who says they are talking on behalf of all youth in South Africa.
“My real issue is why trained journalists are just picking it up and running with it, without examining it critically and asking questions?”
Conclusion - Surveys do not represent views of SA youth

Pondering Panda's suggestion that it conducts cellphone "interviews" is also misleading. There are no interviews in the true sense of the word. Respondents who have downloaded the Pondering Panda app choose which surveys to fill out and tick off the answers to questions.
The real blame, however, lies with the media outlets that have unquestioningly swallowed Pondering Panda’s headline-grabbing press releases. There are probably a host of explanations: budget cutbacks, overworked and inexperienced reporters and an insatiable appetite for cheap, cut-and-paste web content.
The reporting on Pondering Panda, for the most part, has not been journalism. Sadly, far too many of the articles that Pondering Panda so proudly link to on their website are classic examples of “churnalism”. And the reporters have become little more than public relations tools, unquestioningly and unthinkingly churning out copy that has a gimmicky appeal but very little merit.
Edited by Julian Rademeyer
Add new comment