Let the Landlines Ring: One Easy Trick to Make Your Poll Results Pop

If nothing good happens after 2 A.M., then by inversion’s logic, at 3 P.M., even if one was eavesdropping on a retiree at an Arizona golf course mere weeks after Trump’s inauguration, some good can arise. And thanks to some chance positioning at the driving range, and my strained hearing efforts, on that fateful 2017 day I learned two things: that Bill O’Reilly had a new book out called Old School, and that in this fellow’s estimation it had a “really good” section early on, in which one could determine, by way of a brief but scientifically rigorous quiz, whether one were “Old School” or a “snowflake.”

For a few weeks I would tell this story and think little of it, a brief oasis of silliness in that strange era when the biggest issue was finding time to March for Science. But then, during one lazy afternoon when I was neglecting my work, I had an epiphany: If this quiz were in the introduction, or close to it, then thanks to the previewing magic of Google books, I could probably take it myself. Thirty seconds later, I learned the comedic power of “landline.”

Most Popular

Suffice to say, the quiz is a disaster. Across much of the preamble and its five questions, it’s equal parts offensive and unsurprising, sprinting through a paint-by-numbers display of transphobia, jingoism, and basic aversion to human difference that any midlife hack comedian could provide. For the average Fox News viewer, it is perhaps the closest thing they have to soul food.

I think “landline” is different. All of these questions are misguided, but “landline” is such a targeted but impotent jab at the soft millennial underbelly, it always makes me laugh. I can’t fathom that something this innocuous is one of the Old School crowd’s five biggest grievances, let alone batting first in the lineup, and then I realize that there are nearly two hundred more pages of presumably lesser gripes. Its placement in the book provides insight into the collective psyche of this population—anything representing change, no matter how much more powerful and dynamic and accessible, must be bad—but it also offers the left rhetorical ammunition against that generation. If conservatives can deem any and all opponents “snowflakes,” then in turn I suggest we term them “landlines”: dusty, obsolete, and far more dependent on government infrastructure than they’d like to admit.

Outside of O’Reilly’s impotent polemic, though, there is one context in which landlines still play any outsized role of importance: political polling. As phone technology has developed, not just through the proliferation of mobile phones but also caller ID, silent mode, and the growth of telemarketers, response rates have plummeted. By the second half of 2018, well over half (57.1%) of American homes possessed only wireless phones; for young adults, that proportion was north of three-fourths. And yet, a survey that no one answers is a bad survey; as long as people need to answer the phone, landlines will play an important role for pollsters, for better or for worse.

This article by Hassen Morad details the diminished yet continued significance of landlines in polling: Across 71 different kinds of polls, landlines accounted for about a quarter of all survey respondents. As Morad documents, the plurality of polls reach participants through both landlines and cell phones.

Today, there are polls of all shapes and sizes. There are neat trendlines, visualizations of the range of uncertainty conveyed by margins of error; there are nuanced grades of polls, mincing distinctions made between those targeting registered voters and likely voters. Even if no one knows what to do with any of it, we are in a golden age of data. But to my knowledge, so far no one has tried to answer this rather pressing question: Who’s winning the landline vote?

That is, until now.

Introducing…The Landline-Indexed Evaluator (LIE)

The LIE is calculated thusly:

—First, we copy the national and early state polling numbers from RealClearPolitics—because with this foray into political predictions, we will not be sending any traffic to our new sworn rival, FiveThirtyEight.com.

—Then, we dig into the methodology for each poll included in those averages. Unfortunately, none of the polls I encountered break out their panel data by manner of contact: In other words, for a poll that has both landline and cell phone respondents, we can’t filter to only landline respondents.

—Instead, we use the next best estimator: What percent of interviews did they conduct via landline? This percentage will act as our multiplier for whether we trust a given poll’s results—the higher the percentage, the better (for our purposes) the poll. Online-only surveys, like YouGov’s, can take a hike.

For example, this Monmouth poll from New Hampshire (showing Bernie Sanders leading by 4 points) featured 377 landline respondents and 400 cell phone respondents: So its landline multiplier is 377 / (377 + 400), or .4852.

—After we have the multiplier for every poll, we can then properly weight each poll, and by summing (and re-scaling) across these adjusted polls, we have our measure.

With that in mind, let’s take a look at how the RCP averages stack up with our LIE averages.

In our pool of five national polls, we completely removed three online-only polls from our dataset and re-weighted the remaining two. However, this view doesn’t clearly reflect which candidates received the biggest bump / decline; to do that, we add the percent change. (To keep this chart manageable, we’ll focus on just those candidates polling above 1 percentage point.)

It appears that focusing on, or at least amplifying, the responses of landline voters doesn’t greatly affect any of the current frontrunners, with Amy Klobuchar seeing the sole bounce north of 10%. (In aggregate terms, Sanders receives the biggest bump of 1.7 percentage points, while Michael Bloomberg suffers the most, falling 2 points.)

We can conduct similar analyses based on the final set of polls in Iowa (albeit sans the Des Moines Register poll that was infamously spiked at the last-minute, and sans a Focus on Rural America poll that didn’t quantify its number of landline respondents):

And in New Hampshire:

When I first envisioned this analysis, I imagined we’d see massive jumps for Joe Biden as we prioritized landline voters, and correspondingly big declines for the candidates stereotyped as having hyper-online followings (Sanders and Andrew Yang). But that’s not at all the case. Neither Biden nor Yang have wide disparities between their LIE averages and their RCP averages, and Sanders’ numbers actually improve both nationally and in these two state-specific polls. The most consistent effect appears to be a boost for Klobuchar, which maybe points to her middle-of-the-road, scolding, “landline” appeal, or maybe points to the many flaws in this methodology.

You might be asking, 1,100 hundred words deep: Lucas, why would you bother doing this? Indeed, I breezed through the portion of this post where a genuine data journalist might put a statement of purpose, and that was no accident. While I thought the exercise would be more striking and worthwhile—and perhaps it will be in the general election—the truth is that this was a coping mechanism, a stalling tactic to avoid writing something of importance. I simply want Tuesday’s New Hampshire primary to arrive and supply a clear outcome, so that I can stop reading about delays and diversions, shadowy apps named Shadow, error-laden tabulation, and outsized momentum swings built on erroneous conclusions.

This past week of politics has felt surreal, and so much of that stems from the juxtaposition of what we know and what we don’t. Polling is the same way. With their limited transparency, polls only offer a picture that we can see with enough clarity to tell it’s blurry. That’s not good enough: If polls feature mixed methodologies, they should break out the top-line results by survey method. That is, we should be able to see how landline respondents responded, how mobile users did, and so forth.

Here’s the conundrum of voting in the modern era, in the aftermath of the shock of the 2016 election: No predictive numbers are ironclad. Nothing can sufficiently ward off our niggling doubts. And a surplus of dry, public knowledge about the state of things can foster complacency (or spark desperation) in voters that directly mitigates the already-inauthentic narrative of the polls. I don’t know if more information is the solution, but I know that with more information we can at least salvage something amusing—a debilitating oversight in methodology, a telling divide in how we think, a respite from the overwhelming stakes of these next ten months. I want to see how 90 percent of Biden’s support comes from people who use Netscape accounts. I want to track the meme-based polls that swing for Yang; to scan the surveys of Young Republicans that put Pete Buttigieg in the lead; and to hunt down whatever arcane, mobius-stripped cross-sections manage to capture Michael Bennet voters.

Then again, that desire to look to statistics for solace perhaps indicates my own biases. I’m nervous for Tuesday, and not just given everything that transpired in Iowa: My phone battery is quickly deteriorating, and I don’t have a landline.