Tuesday, August 27, 2024

Opinion Today: These 3 things about the polls keep me up at night

Social science can be messy.
Opinion Today

August 27, 2024

An illustration that includes a photo of a hand picking up the handset of a rotary telephone.
Illustration by The New York Times. Photograph by Getty Images
Author Headshot

By Kristen Soltis Anderson

Ms. Anderson, a contributing Opinion writer, is a Republican pollster and a moderator of Opinion's series of focus groups.

Each Tuesday here at Opinion Today, from now through Election Day, contributing Opinion writer Kristen Soltis Anderson will assess the latest polls to shed light on how voters see the 2024 campaign, the stakes for the country and the issues that matter most to them.

Sunday night was an unexpectedly exciting time in the Anderson household: Just after dinnertime, my husband got called and asked to participate in an election poll. I was in the middle of getting my toddler ready for bed when he rushed into the room, holding our baby in one hand and his phone in the other, now on speaker so I could hear the questions.

There's a real thrill you get as a pollster when you get to peek behind the curtain and see how someone else in your field is asking voters what they think. But it was also a reminder of how messy social science can be. Here was my husband, telling this very nice interviewer what he thought about, among other things, immigration and Tim Walz, all while pacing the hallway bouncing a bedtime-bound infant. I can only imagine what others are up to while answering survey questions.

As we near Election Day, Americans' interest in polls will only increase. Election forecasting models are being heavily scrutinized and debated by commentators and reporters. Small shifts in polling results can trigger big headlines. As we start this post-Democratic convention week, when analysts will be combing for any hint of a bump for Kamala Harris, it's worth keeping in mind that there are lots of things that make polling art as much as science.

Pollsters try to get it right, but we don't know what we don't know, and sometimes in our industry it can feel you're fighting the last battle. In 2012 some pollsters missed the mark and overestimated Mitt Romney's vote share, in part because they were excluding less-likely voters or not calling enough voters on cellphones. By 2016, those issues were largely addressed, but new problems reared their heads, like some pollsters overrepresenting college graduates in their surveys, yielding poll results that were overly favorable toward Hillary Clinton, particularly in key Upper Midwest states.

Four years after that, pollsters had mostly fixed the issues that plagued them in 2016, yet in 2020 many polls "featured polling error of an unusual magnitude," according to the American Association for Public Opinion Research's task force on the matter. The task force was circumspect in its report on what went wrong and didn't propose a definitive solution. And troublingly, the industry did not come away with a clear conclusion as to what went wrong, so that it could be remedied.

As a pollster, I'll be the first person to tell you that I have a measure of apprehension about the polls this year. I mentioned last week that a couple of warning lights are starting to blink on the 2024 polls. And so an accounting of a few things that keep me up at night:

  • The Trump factor. Contrary to the after-the-fact narrative, overall, the polls in 2022 were fairly good. Same in 2018. But years when Donald Trump is on the ballot simply may introduce a type of voter who participates only as a way to support or oppose him and who is, therefore, for whatever reason, unpollable.
  • Unlikely likely voters. This is a related concern. In some of those prior cycle polling misses I mentioned, one problem was pollsters failing to adequately factor some groups of unlikely voters into their samples. In 2012 those voters favored Barack Obama. In 2016 and 2020 they presumably favored Mr. Trump. In a shaken-up 2024 race, it remains an open question who will benefit most from unlikely voters.
  • Poll fatigue. It seems like the number of public election polls being conducted has risen as barriers to entry into the field have fallen. "Technology has disrupted polling in ways similar to its impact on journalism: by making it possible for anyone with a few thousand dollars to enter the field and conduct a national poll," wrote Pew Research Center's Courtney Kennedy four years ago. And that's not counting the private polls being conducted out of public view for campaigns and clients. Getting a call from a prominent polling outfit is a big to-do in my home, but it isn't hard to imagine a voter in a battleground state getting to late October of this year, looking at the caller ID and begging for the pain to stop.

Before we all get too deep into this election cycle, I'm imploring everyone to keep their expectations tempered, to expect the unexpected and to treat horse race polls with a level of healthy skepticism. And if a pollster does call you, consider yourself a bit lucky. Please be kind and answer.

Here's what we're focusing on today:

Editors' Picks

Kamala Harris saving to crowds, as seen over waving U.S. flags in the foreground.

Patrick Healy

How 14 Undecided Young Voters See Kamala Harris After Her Speech

They were skeptical and even suspicious of Ms. Harris before the convention. Her speech shifted some of their thinking.

By Patrick Healy

More From Opinion

An illustration of a worm poking out from a sea of press microphones in the foreground.

Pamela Paul

A Worm's-Eye View

After digesting Kennedy's thoughts, his motivations become clearer.

By Pamela Paul

The faces of Kamala Harris and Donald Trump superimposed on graph paper.

Guest Essay

The Short-Sighted Economic Populism of Harris and Trump

It's worrisome to see the candidates talk down to voters.

By Roger Lowenstein

A black-and-white photo collage showing President Biden, standing at left and looking to the right, and Abraham Lincoln, seated at right and looking to the left.

Guest Essay

Abraham Lincoln's Wiliest Trick Should Be a Lesson for Biden

A model of how a lame duck can nobly serve the nation.

By Michael Vorenberg

An old computer keyboard and a monitor in a field with many stones.

Guest Essay

Breaking Up Google Isn't Nearly Enough

To end Google's monopoly on search, the company should be required to disgorge its data on what we're searching for.

By Julia Angwin

The Opinions

The 'Sad Missed Opportunity' of the Democratic Convention

Kamala Harris needs to do more to assure voters on Gaza.

play button

10 MIN LISTEN

A withered banana peel with a faded sticker on it knocks down blue chairs at a breakfast table. The table has pancakes and orange juice on it.

Guest Essay

How Much Bloodshed Is Behind the Bananas You Eat?

In June, a Florida jury found Chiquita Brands liable for the deaths of eight men killed by a Colombian paramilitary group the company had financed.

By Ignacio Gómez G.

A black-and-white photograph of Butch Wilmore and Suni Williams in spacesuits.

Faced With the Boeing Starliner's Flaws, NASA Gets a Couple of Things Right

Redundancy, it seems, has paid off.

By Neel V. Patel

A person shown from behind at the New York Stock Exchange watches a TV screen that shows the Federal Reserve chair, Jerome Powell, speaking. Stock exchange data is on a screen above it.

Paul Krugman

Inflation Is Fading, Statistically and Politically

The cost of living may not be the G.O.P.'s, um, trump card.

By Paul Krugman

A black and white photograph of Gus Walz, overcome with emotion and clapping.

Gus Walz Brought Out Both the Joy and the Cruelty

He deserved the kind of treatment any person does.

By Pamela Paul

An illustration depicting three figures, two orange and one blue, pulling on an orange rope as if in a tug of war.

Peter Coy

What Kamala Harris Needs to Do to Take On China

Washington cannot win "the competition for the 21st century" against Beijing by itself.

By Peter Coy

We hope you've enjoyed this newsletter, which is made possible through subscriber support. Subscribe to The New York Times.

Games Here are today's Mini Crossword, Wordle and Spelling Bee. If you're in the mood to play more, find all our games here.

Forward this newsletter to friends to share ideas and perspectives that will help inform their lives. They can sign up here. Do you have feedback? Email us at opiniontoday@nytimes.com.

If you have questions about your Times account, delivery problems or other issues, visit our Help Page or contact The Times.

Need help? Review our newsletter help page or contact us for assistance.

You received this email because you signed up for the Opinion Today newsletter from The New York Times.

To stop receiving Opinion Today, unsubscribe. To opt out of other promotional emails from The Times, including those regarding The Athletic, manage your email settings. To opt out of updates and offers sent from The Athletic, submit a request.

Subscribe to The Times

Connect with us on:

facebookxinstagramwhatsapp

Change Your EmailPrivacy PolicyContact UsCalifornia Notices

LiveIntent LogoAdChoices Logo

The New York Times Company. 620 Eighth Avenue New York, NY 10018

No comments:

Post a Comment