I’ve frequently criticised the newsmedia for getting their reporting about statistics completely wrong, or simply drawing the wrong conclusions. An NZPA story carried by Stuff is yet another example.
The story was headlined: “Nearly half of Kiwis support Aus-NZ union” and was about results of a poll that will be released tomorrow on TVNZ’s “Q+A” programme. The story, however, doesn’t support the headline: It turns out the survey found that 41% of New Zealanders thought the idea was “worth debating”, and 58% did not.
First, and most importantly, thinking that union of the countries is “worth debating” is hardly the same thing as supporting that union actually happening. It’s also debatable whether 41% is really “almost half” of anything (I’d say it definitely is not); after all, almost 50% more people were against debating the idea than were for it.
Many ordinary people don’t understand statistics, which makes it worse when the newsmedia make conclusions that are completely unsupported by the data—or let political activists get away with doing the same. The newsmedia must try harder—a large percentage of people are counting on them.
Update 14 March: The full poll results have now been posted, and it shows the reality is very, very different than the original story suggested. The focus of that original story was that a large minority were okay with discussing not a NZ/Aus union, as the story suggested, but New Zealand becoming the seventh state of Australia, which is very different. But 71% opposed that idea, and a mere 24% favoured it—which is pretty decisive by anyone’s reckoning. Had the original story said that, there’d be no incentive to watch the report. Clearly they focused on the other statistic to generate interest in the broadcast, so they also didn’t mention that 71% said it was not time to discuss the idea (even if a minority felt it was an idea “worth debating” at some unspecified time).
So, it’s pretty evident now that the original story was intended only as a promo for the TV programme.
7 comments:
Don't know i it's true in NZ - heck, I don't even know if it's true online in the US - but the headline writer in US print newspapers often has nothing to do with the story writer. And more often than you'd imagine, I have difficulty with the headline vis a vis the story.
There is really two problems at play here:
1) Journalists arent trained in understanding what the polls mean and for that matter nor are the pollsters - they often have other motivations.
2) News media around the world have had their newsrooms shrunk and the journalists just don't have time to investigate - just do a quick google search (actually some don't even do that)
Roger: That's often been true here, with headlines written by sub-editors. Many of those positions have been eliminated as part of downsizing, or outsourced. That's part of the problem. See also next reply…
Mark: For a very long time I've criticised journalism for what you're getting at in point one: Journalists are taught to be journalists, and that's it. I believe they should be required to have a good generalist education first—heavy emphasis on history, social science, science, statistics. I'm dreaming, I know, but I think it's almost unreasonable to expect journalists to understand things they've never been taught. What I'm really saying is that major news organisations ought to have SOMEONE who can correctly interpret statistics.
And you're also right in your second point. For example, New Zealand's sub-editing work is being outsourced to Australia, which has issues of its own. Newsrooms are short-staffed making real research harder to do (there's not enough time). But, again, if they just had people to be subject experts (like having one person who understands statistics) would improve the situation.
Of course now the Herald's picked it up and shouted it to the rooftops too. Polls in general are dodgy but the ones down here are as weak as any I've seen I think.
This is a very interesting item on ABC Radio National tonight
Journalism just a PR exercise?
Is rather depressing reading - original is here on crikey
Nik: Exactly. I've written a couple posts about one polling organisation in NZ—always reported as if it's true and news—whose polls are worthless. As bad as polls can be in the US, I can't recall writing a similarly critical post about any of them.
Mark: Yes, very depressing. I downloaded the MP3, btw. Radio New Zealand's "Mediawatch" has covered similar subjects (there's a link under "Parade of Podcasts", because it's available as a podcast).
I see that Crikey covers drug company's "influence" over health reporting, and it's something I've been critical about, too.
Oh, and this has nothing to do with anything, but I love the way designed their report.
Post a Comment