By now it is widely understood among Sanders supporters that the mainstream media has been relentlessly pro-Clinton in its coverage of the Democratic race.
Twitter hashtags such as #BernieBlackout regularly trend on nights when Sanders wins a caucus or primary, only to have the cable news networks ignore his victory speech or cut away halfway through.
Most media failed to cover a recent major #BernieBlackout protest outside CNN’s Hollywood, California offices over lack of coverage.
However, in addition to this media bias is an increasingly clear and worrisome phenomenon in which polling organizations and data analysts consistently underestimate Sanders’ relative strength in a given caucus or primary. The most notable example is the Michigan Primary, where polls showed Clinton with a 20-30 point lead just days before the race, which Sanders ultimately won by 1.5%.
I have already written about how obviously biased in favor of Clinton many of the data analysts that use statistical modeling and poll aggregation to make predictions have been. Some are even open about their preferences. Perhaps the most prominent is Nate Silver, the founder and editor in chief of FiveThirtyEight, who in a February editorial shamefully dismissed Bernie Sanders as “another old white guy.”
Had Silver used similar language to describe Clinton he would have been pilloried and perhaps fired. Nevertheless, at this point one might reasonably ask how voters can expect objective analysis and some modicum of accuracy from data analysts if the Data King himself is so anti-Sanders.
This fact seems to point to a larger phenomenon, which is that it appears many of the pollsters and data analysts are plagued by confirmation bias in their polling and analysis.
Confirmation bias is defined as “the tendency to search for, interpret, favor, and recall information in a way that confirms one’s beliefs or hypotheses, while giving disproportionately less consideration to alternative possibilities.”
To be clear: I am not suggesting that all pollsters and data analysts suffer from this cognitive malady. In the case of Michigan, there were several legitimate factors put forth to explain Sanders’ shocking win after trailing so heavily. These include an overreliance on landlines rather than cell phones to conduct polls, which would clearly undercount Sanders’ young supporters, as they are the ones more likely to have only cell phones.
As one prominent pollster noted: “There’s a huge divide in the Dem race between people who have landlines and people who don’t, and your polls are going to massively overstate Clinton’s support if you don’t contact the people without them.”
Furthermore, younger voters turned out in much larger numbers than expected, bolstering Sanders’ support. He also performed better among African Americans than he had in the Southern states. And he decisively won Dearborn, the city with the highest concentration of Arab Americans and Muslim Americans in the US, surprising media pundits who had assumed Muslims would not support a Jewish candidate. However, given the destruction Clinton’s policies have wrought in the Arab-Muslim world, it’s hard to see why pundits were surprised.
Some have also speculated that pollsters and analysts’ increasing use of robo-calling and complex statistical modeling, both of which avoid human interaction, can help to explain notable poll failures. Moreover, some of the smaller, less established polling organizations employ methods such as pop-up online surveys that are surely subject to selection bias and lack scientific rigor.
If Michigan had been an anomaly, perhaps the upset could have been chalked up as simply a bad day for polling. But then Wisconsin happened. This time the polls got it right, but they were way off on the margin of victory. On Saturday, April 2nd, RealClearPolitics listed a poll showing Clinton with a 6-point lead. On Sunday, another poll — this one by CBS News and YouGov — had Sanders with a 2-point lead. This poll was widely cited in the major media as the most authoritative predictor of Tuesday’s race. On Monday, an Emerson poll had Sanders up by 8 points, but the poll was much less prominently featured in the media heading into Tuesday’s contest.
When voters finally had their say, Sanders’ margin of victory — 13.5% — was well beyond what even the most Sanders-favorable polls had predicted.
Contrary to Michigan, little ink was spilled — either digitally or broadcast — on the failure to predict Sanders’ wide margin of victory in Wisconsin. Certainly Michigan was a bigger upset. And to be sure, the polls have been reasonably accurate in a significant portion of the other contests thus far.
Yet Sanders has also previously scored other major — though barely reported — poll-defying victories in Minnesota, Kansas and Colorado. According to one analyst, Sanders actually averaged +35% above the most recent polls in Michigan, Minnesota, Kansas and Colorado combined, although there appears to have been fewer polls available for the latter three states.
Considering the frequent underestimation of Sanders in polls, the open support for Clinton expressed by data sites such as FiveThirtyEight, and dismissive Sanders coverage in other data sites such as The New York Times blog “The Upshot,” perhaps there is something to the confirmation bias phenomenon. Is it fair to ask if pollsters and data analysts consciously or unconsciously make assumptions and employ methods that will favor Clinton?
There is no easy answer to this question. Maybe it’s as simple as letting the dominant media narrative of Clinton’s surefire ascension to the presidency influence the strategic and analytical choices organizations make about the polling and data analysis they perform.
Looking ahead, the New York Democratic Primary is Tuesday, April 19th. A Friday April 8th poll (Emerson) measured Clinton’s lead at 18-points. A Monday April 11th poll (Monmouth) showed Clinton’s lead down to 12 points, with a notably large margin of error of plus or minus 5.6 percentage points. The large margin of error is likely due to the relatively small sample size (the smaller the sample size, the larger the margin of error). It is also worth noting that only 54% of respondents in the April 11th Monmouth poll said they were “certain” about their vote. The rest have either a strong preference but are still willing to change, a slight preference, or remain undecided.
Patrick Murray, the director of the April 11th Monmouth poll, said in the poll’s press release: “I’m sure the Clinton camp was hoping for a much bigger lead in her adopted home state, but any such advantage appears to be limited against Sanders.”
What the last few weeks of New York polling clearly reveal is that Sanders has made significant inroads into what was once a 48 point-lead for Clinton back in mid-March. Given the surprises in Michigan and Wisconsin, and in Minnesota, Kansas and Colorado earlier, as New Yorkers prepare to head to the polls tomorrow, no one can credibly dispute the idea that Sanders can win the Empire State.
Furthermore, regardless of tomorrow’s outcome, it would seem that perhaps from now on we should require polling agencies and data sites to issue mandatory disclaimers informing the public of which candidate they support. We could create something like the letter-based rating system for video games and movies, with a [C] for Clinton-skewed organizations, an [S] for Sanders organizations (if any can be found), and a [?] for “honestly we have no freaking idea, but we like Hillary!”
[UPDATE: A Sunday April 17th CBS News/YouGov poll showed Clinton with a 10-point lead, with a margin of error of plus or minus 4.4 percentage points. And a Monday, April 18th Emerson poll revealed a 15-point lead for Clinton, with a margin of error of plus or minus 4.6 percentage points.
Senator Sanders himself weighed in on the polling bias issue on Monday, alluding to the latest polls and telling NBC’s Today: “Those are the public polls. The bottom line is, let’s look at the real poll tomorrow. Generally speaking, polling has underestimated how we do in elections.”
He also referenced the Michigan upset, noting that his campaign was down by 25-points in polls before its upset March 8 victory. And at a Prospect Park rally for the candidate on Sunday, some of his supporters also expressed their distrust of the polls, stating that polls were “rigged” and that Sanders would prevail on Tuesday.]
Brilliant analysis, powerfully supported by the facts.
Great stuff!
Good stuff here, with a rousing finish! Besides whatever cognitive biases may be at work, I think another factor is that there are some many different ways of getting and spreading information these days, what with social media and all, that relying on any one method (e.g. polling via landlines) is bound to give an inaccurate picture
Glad you liked it! You are right about the manifold ways of consuming and spreading information, including social media. It appears that some of the lesser known, smaller polling organizations often rely on methods such as online pop-up surveys, which, while enabling the ability to reach larger numbers of people than phone polling, are surely susceptible to sample bias and lack of scientific rigor. But as you said, it’s a brave new world of information, so perhaps polling tactics need to evolve too.
Informative! Helps me understand how the major polls are contributing to the campaign madness.
Rather plays into the overall narrative of Clinton’s corporate ties is her strongest card vs Bernie who is challenging the corporate status quo with the popular vote.
Not sure of the overall value polling adds, in this specific context. Can it change someone’s vote? I don’t know.
You are right, Clinton definitely is the preferred candidate of Wall Street and corporate America, and most of them clearly don’t like Bernie, because he’s a fundamental threat to their bottom line.
Regarding polling, I do think polls influence some people’s choices about whether they will show up to vote, for example, based on whether their preferred candidate needs their support or is way ahead, etc. Similarly, they influence who some people will support based on the relative positioning of the field.
“Why Polls Undercount Sanders” …
… because politicians and “the media” like their free rooms and golf from the Orange Idiot of New York …
We need to keep repeating this message over and over again. The TV is always WRONG. Even if the the facts are correct the story is about something designed to distract you from what is important. Younger people are learning to stop believing the TV is truth. I can see my parents are completely brainwashed by it. They repeat the nonsense as if it is gospel. Just turn it off and encourage your friends to do the same. You don’t need corporate media in your life.
I think TV news still has a place, and some types, such as investigative stories, can be worthwhile. However, when it comes to the central question of power in America — who has it and who wants it — the networks circle the wagons for their preferred corporate candidate, as we have seen in the 2016 election.
In my mind, the polling industry as a whole lacks any standards or discipline to accurately predict any response to a question from a group of people.
Years ago, when Nate Silver of fivethirtyeight was switching from prognosticating baseball to politics, I suggested his talents would be better used to create and organize an association that polling organizations could join as long as they met a quantifiable, objective and scientific based effort for their polling. Kind of like an ISO 9000 accrediting agency for the polling industry.
That’s an interesting idea. It seems like some of this happens through natural selection – the media tends to give more attention to the larger, more reputable polls and polling organizations. But you are right, there should be more uniform standards. I’ve wondered about the US inviting international election monitors to come in and certify the whole process. Obviously the powers that be would never go for it, but it would be interesting.
Excellent article. What really puzzles me is that, given the landline-cellphone problem, they don’t try other ways of polling younger voters. For instance, simply counting Facebook likes and pages. Or for instance, simply going out and asking people on the street. I can think of many ways to do this. They could pay students to do surveys. It seems like they’re really not interested in getting accurate polls, but just defending the status quo.
There’s another issue. I’ve come to the conclusion that the news organizations simply no longer have the financial resources necessary to do the kind of comprehensive polling we’re all expecting, much less so than just 8 years ago. That’s why they are only doing landline polling, mostly using robocalls. It’s not particularly accurate, but it’s extremely cheap.
Thanks. I agree that polling methods should be expanded. The problem with using online surveys, or measuring Facebook likes, is that often these will yield unrepresentative samples that are demographically or otherwise skewed. Phone polling allows for the the use of geographic, statistical sampling to reach specified confidence intervals with defined margins of error, etc.
I definitely agree that news orgs don’t have the resources they once did, and this can affect the quality of polling.