Nothing can be more fallacious than to found our political calculations on arithmetical principles.
— Federalist 55, Hamilton or Madison.
What happened with Nate Silver? The man was no hack, but a warrior of might and magic. Yet turn around, and Trump teeters on the verge of accepting our universe’s nod as GOP Chaos God. According to Silver, this should never have happened. Silver predicted a new era of refinement in Moneyball politics; a way to quantitatively understand how the game works and predict its outcomes.
His data politics seemed impressive, like the Guardians of Ga’Hoole, but it failed us, like for-real owls, because the methodology is too in love with itself and captures a false view of the world; Silver’s models explain the past but cannot predict the future. The weakness of Silver’s system parallels other contemporary problems of economics and data, and helped with the fall of Sanders and the rise of the Orangeman.
WHO NATE SILVER IS AND WHAT HE DID
It was barely eight years ago, during Obama’s candidacy that promised to Change All Reality Forever. The Republicans had pulled paper bags over their heads and declared that it was night everywhere. Suddenly, towering above all other pundits was the Math Wizard himself, Supreme Mugwump of Stats, Nate Silver. The man had come from statistics, and poker, and proved to be a deft hand at both. Silver stared over the punditocracy, and said to them, You lot have been eating scorpions and hallucinating that you have some insight into politics. I am Nate Silver, and I shall never die. That is a slight paraphrase.
Nate Silver! Caller of elections! Silver’s first taste of fame came from designing a math model named PECOTA, a program which let users make forecasts about baseball. But he struck his real buzz bonanza during Election 2008, when he correctly predicted the outcome in 49 of 50 states. He made all of the relevant most-influential lists the following year, went on to guess the 2012 election — nabbing all 50 states this time around — and started up an award-winning political blog, FiveThirtyEight, which was hosted at the Times before ESPN bought it.
The Guardian, and the kind of people who read The Guardian, and the kind of people who want to be the people who read The Guardian – they all frankly adored him. Carole Cadwalladr wrote:
“Nate Silver is a new kind of political superstar… In America, punditry has traditionally been about having the right kind of hair or teeth or foaming rightwing views. Silver has none of these. He just has numbers …. on the night of the US presidential election, they were proved to be right in quite spectacular fashion.”
Rachel Maddow declared that Silver had really won the election, and his book sales shot up by 800%. The usual parade of cranks and wingnuts despised him, for obvious and delicious reasons. xkcd was predictably smug.
This bare recitation of facts does not adequately describe Silver’s real impact at the time. It is easy to say “He was the big figure in the rise of data journalism,” but that isn’t an accurate capture either.
Over the past three decades, Americans witnessed the injection of hordes of shallow speakers into the media – quasi-fascist talk radio hosts, pedantic basement war-bloggers, the whole Who’s Who ecosystem of conservative nepotists: John Podhoretz, Bill Kristol, Jonah Goldberg, and the squirrel-infested tree of the Kagan family. These and other assorted nuts had had their turn upon the stage and eventually sank back into the think tanks that spawned them, muttering fabricated Founding Father quotes and clasping Hayek and The Purpose-Driven Life all the way down.
There seemed to be an unending line of them. They would mouth humble-bee platitudes about the vast mysterious Other dooming us – depending on the moment, these could be homosexuals, socialists, teens, immigrants, ungrateful blue-collar voters, Muslims — and always, of course, the beastly liberals, simultaneously ineffective and immensely clever. When Nate was proved right on Election Night 2012, seeing Karl Rove breaking down on Fox News made up for years of watching him succeed. But Silver’s appeal really went beyond peeved leftists.
The real curse of pundit journalism has never been the obvious mouth-breathers on the Koch payroll, but the scrim of well-paid news network centrists who keep us from seeing the truth. After Silver turned out to be right, the world was treated to the spectacle of the pundit corps floundering around like gibbons suddenly catapulted into the mesosphere. Indeed, the talking heads almost seemed to fade into the evening haze in the manner of Jedi ghosts. They sought solace in fell bars beneath Washington landmarks while they groped each other and considered mutual murder pacts: the age of the gigantic con of political commentary was over, we thought. A new summer was coming in. The age of entitled, puffy tools in studios recycling talking points and prattling about in vague, unfalsifiable generalities, would come to an end.
We understood what Nate Silver was doing was data-driven, quantitative politics, a politics which promised to end our reliance on the specious gut-feeling notions of hack pundits. No more leering middle-aged hucksters from the Beltway basing their calls on intestinal palpitations or whichever way their wallet’s interest lay. After the Bush years of truthiness, the rejection of science across wide swathes of the American mindshare, and the vague miasma of horseshit that surrounds American punditry, there would be a royal road to political understanding, and Silver was the Moses to take us up them trails.
We loved him for it. Hell, I loved him for it. I bought his book, and after Romney was skewered and the whole army of Republican cardsharps became gaping, fish-faced reaction gifs, I photoshopped a picture of Silver in the style of the X-Men’s “Magneto was Right” meme and uploaded it across the web.
Silver showed us a shining path, all right. He was the Big Data man, in the Big Data age. He went on from there from strength to strength. Would there be no end to his incredible powers?
SILVER’S WAY
Silver used a combination of poll and probability analysis and demographic study. His book The Signal and the Noise sold, and his blog was tremendously popular. He could do no wrong. Which is why when he fell, he fell hard.
We read Nate Silver as pre-Darwin naturalists used to catalog dead jellyfish, hoping to gain some secret of the cosmos through measurements. So what happened? How could the god of math lead us all awry?
Again and again, Silver told us Trump had no chance, none at all. Because Silver had gathered up all the broken pieces of pundithood and pasted them together in a madly correct prediction back in 2008 and 2012, we trusted Cassandra. But he was wrong, wrong, wrong.
Some people had never put stock in him, or lesser figures like Chris Cillizza, who also claimed to be a “quant,” or quantitative political thinker – a pundit who alleged specializes in hard data. Virgil Texas, who writes columns under the guise of a fictional data journalist named Carl Diggler (whose fake quant site is called SixThirtyEight), criticized Silver, noting that he “enjoys the imprimatur of scientific authority … [Silver] insists that hard mathematical analysis is more objective and more accurate than old-school pundits with their capricious instincts. ‘DataLab,’ the name of FiveThirtyEight’s blog, conjures up wonks in white lab coats chalking authoritative equations.”
However, Virgil wrote, “Good science is falsifiable. Silver’s horserace predictions are not.”
Silver missed Trump, and didn’t understand Bernie. He was wrong, but his wrongness is not the end of the affair. After all, some men, such as Bill Kristol, have made a career out of being wrong. With Silver, there was and is a deeper problem.
After the fact, there were lots of lots of articles debating just what Silver had gotten wrong. All of them had some level of crowing over how the quants had been outflanked by reality.
Well, what went wrong? First, Trump was shown in poll after poll to be ahead of his rivals. As the writer Leon Neyfakh termed it, the polls were plentiful and consistent. Silver never bought into this narrative, and repeated many times how inconsistent they were. There was some precedent here: after all, it’s only natural for polls in the early stage of a primary to identify some far-out new face as the shining light of the field. Silver explained, but most of it landed squarely in Black Swan territory.
The polls, Silver said, didn’t have much weight. Voters in primaries wait until the last minute, right? And folks who answer poll questions may not be the people who vote. While it is true that Silver has forgotten more about polling than most human beings shall ever know, what was arbitrary about Silver was his method of picking which polls to disregard. When polls reliably showed the Orangeman holding up the lead, Silver said no way. This was the basis of his “Trump is winning the Polls but Losing the Nomination.” When journos said Trump was a rising power, Silver said it was punditocratic sham.
Silver made an assumption that Trump would not be the nominee. He kept this assumption even when it was clear that Trump was still rising in the polls. I’ve read his writers. This is not a hack who hides under a river of math-wash to fool the buffoons. Silver believes in the reasoned, be-conscientious-of-your-words school of writing, and that’s the tragedy of his twenty-one gun salute of misfires. He had all the right in the world to gloat about calling two national elections, and he doesn’t in his book. Silver wrote about how the early polls did not give Trump a major lead necessarily, but disregarded all the future indicators that he was, indeed, in the lead. This assumption preyed on him early, took fast hold of the quant, and never let him go. That is the first knot on which Silver’s axe broke.
THE PARTY DECIDES
As for the second: Nate pledged his heart at the fraternity of The Party Decides: Presidential Nominations Before and After Reform, a highly influential 2008 book by four political scientists: Cohen, Karol, Noel, and Zaller. The authors argue a candidate must successfully romance the party elite to be nominated. Endorsements are the key to victory. Candidates win their party’s nod by capturing certain influencers in the party, Pokémon-style. Silver leaned on this faulty crutch for months. Rather than prefer cold stat calculation, Neyfakh wrote that “in repeatedly citing The Party Decides, he [Silver] was relying on a theory about how political parties work—one that’s been embraced by some of the very same pundits that Silver has defined himself against.”
Nobody believes that the nomination process is not at some level a cover up for an elite love circle at the highest level, or that the best candidate wins, or that democracy is pure here, or that money is not the predominant factor. But one thing I have learned in over twenty-five years as a watcher of politics is that although money may thwart the will of the people at every level of the political process, it cannot make a terrible candidate into a good one. It can make a Status Quo Bro like George W. Bush seem like a first-rank pageant contestant for a while, but it is hard to fool the people entirely for very long.
And yet … Silver screwed himself to the sticking place and held onto The Party Decides model even when it became apparent it wasn’t working. He presumed that the palest of the pale at the top of the Republican pyramid would deep-six the Orangeman, because that is what he, eminently rational Nate Silver, would do in their position. Neyfakh writes: “Indications in December that GOP leaders were either powerless against Trump or unwilling to go after him struck Silver as ‘perplexing’—precisely the emotion you would expect from a quantitatively inclined thinker confronted with a reality that hasn’t conformed to his calculations.”
“Even when all of Silver’s models for a given race turn up wrong, it never seems to be FiveThirtyEight’s fault,” wrote Virgil. “… Their mea culpa after Michigan’s Democratic primary, which Sanders won by 1.5 percentage points even though Silver’s model gave Clinton a greater than 99 percent chance of winning, was titled ’Why the Polls Missed Bernie Sanders’s Michigan Upset.’ … what’s the point of a prediction if you can’t stand by it, and if it doesn’t, well, predict? This ridiculous backpedaling has a glib, Brechtian tone to it: ‘The models are right. It’s the voters who are wrong.’”
Even if the nomination landed in Trump’s hand, FiveThirtyEight thought the party establishment would still sniff out a way to throw momma from the train. Again, Silver is not like many people who I discuss, con men who deserve to be dragooned without a trial. This is not a schmuck who pretends to be rational. Rather, this is a man who earnestly thought he was rational, but was very wrong.
WHAT SILVER MIGHT HAVE BEEN THINKING
Some people might find Silver arrogant. I don’t, and neither should you. Silver’s fall is the fall of a certain kind of worldview. Silver himself was modest, but his belief in the model was not.
“Nate Silver has the Bernie Sanders campaign figured out,” wrote the historian Rick Perlstein, reporting on Bernie’s campaign in the early days. Silver argued that journalists should ignore Sanders’ results in Iowa and New Hampshire, or so “the ‘data-driven’ prognostication wizard wrote back in July, when Sanders was polling a healthy 30 percent to Clinton’s 46 percent in both contests.”
Silver argued the results were irrelevant to the larger contest. After all, the Dem primary voters in those places were liberal and white, the foundation of Bernie’s army. Silver had a Bernie chart which ranked the percentages of pale leftists in all the states in the Union. According to this graph, Bernie should’ve been run out of Texas on a rail.
But no—Bernie was warmly received there. Silver tabulated the numbers of comfortable Caucasian cosmopolitans and assumed that was the extent of the Prophet Sanders’ draw. But, Perlstein reminds us, when Nate Cohn of the Times actually studied the data from Iowa, an alternate narrative emerged: Sanders did better with poorer whites than Obama did back in 2008. This usually does not happen with leftist party outsiders, which hints that something had changed. Perlstein again: “That’s Sanders’s argument: that he’s uniquely positioned to assemble a cross-racial coalition of lower-income voters.”
Counting matters a great deal in politics. Lyndon Johnson’s father used to tell him that if he did everything, he’d win, and a lot of everything turned out to be counting right. “Johnson said the first lesson of politics is to be able to count,” Hubert Humphrey said. In his book about LBJ, Master of the Senate, Robert Caro wrote:
“Johnson’s counting was not a social pastime but an exercise in hard political reality — and he was still ‘a great counter.’ To a staff member who, after talking with a senator, said he “thought” he knew which way the senator was going to vote, he snarled, ‘What the f___ good is thinking to me? Thinking isn’t good enough. Thinking is never good enough. I need to know!’ Often he didn’t know … But if he didn’t know, he didn’t guess: the lines flanking the senator’s name stayed blank.”
Silver was not counting. Silver’s specialty, it turns out, is thinking, not knowing. Silver was governed by assumptions. As Virgil pointed out, when you favor one data set over the other, you have brought bias into the system. Silver’s system took in information that turned out to not matter at all.
To give a very simple example of what is wrong with Silver’s graph, suppose you’re a member of Congress who wishes to put together a coalition opposing the NSA violations of privacy. If you are Nate Silver, you would probably look for liberals, or representatives from liberal states, like Feinstein from California. But the Republican Congressmen Sensenbrenner and Amash are far more progressive on this issue than she is.
ASSUMPTIONS
Silver’s beliefs were governed by implicit presumptions: Trump simply could not win, because he was not the kind of elite, worker-hating culture-war spewing warlock the party loved; he was a different kind of gold-plated boob, one who did not play by their rules. Silver turned to history to justify his beliefs: the party didn’t usually nominate tiny-fingered deviants like Trump, so why would it happen this year? In a later article, Silver tried to explain away his predictions by noting how:
“Unlikely events like the Trump nomination are, by their very nature, impossible to predict.”
And:
“One is that the primary system (like the mortgage market) is extremely complex—in Silver’s words, ‘among the most complex systems that I’ve studied.’ Predictive modeling is much, much better in more controlled systems like sports, where Silver got his start.”
And:
”… had Silver relied more heavily on his limited historical data set, he might not have done much better than he did.”
But this is the problem. It is not just that Silver, the nearly-infallible predictor of the 2008 and 2012 results, was wrong. Everybody is wrong from time to time. As the godly Hamilton wrote in Federalist 21, “in political arithmetic, two and two do not always make four.” It is not even that Silver is going to great lengths to avoid blaming himself, or his system — it is that he cannot see.
“Unlikely events like the Trump nomination are, by their very nature, impossible to predict.” But many people predicted it. Trump had serious polling numbers. This was not a black swan or a dinosaur-murdering space rock. Reasonable people could and did foresee the rise of the hair goon. Every single issue in the Silververse revolved around the unarticulated rule that Trump would seize up and collapse in the doomed ink pit of populist death spiral. Silver forgot the numbers, which brought him to success.
Silver tabulated 2008 and stuck the landing because he could look past all the worthless assumptions that governed the hack Luftwaffe. Conjecture was his bosom companion this year, and it showed. Economics, like sports, builds models for small contained universes. But even in the laboratories, there are errors, coincidences, mistakes, misreadings, correlations conflated with causation, and vice versa. If this is true in the tiny, confined, artificial and hugely simplified world of the controlled lab, what does it say about our world? Great as science is, it is not philosophy, queen of all rational thought. Philosophy does not split atoms or untangle DNA, but it does tell bad arguments from good; the heart of the discipline is the careful digging-up and investigation of assumptions. Nate never got around to that part.
Big Data loves behaviorism, inputs and outputs, using computation without insight. We have very clever machines, but they are advanced decision trees and pattern-finding devices. The field of Big Data uses complicated stat methods to hunt for data regularities, but this is not the way to render up discoveries. It is true that knowing every kind of shell in the Atlantic or the age of every citizen of Constantinople can tell you certain facts about the world, but true knowledge demands the patience of farmers and the insightful guesses of artists. It requires abduction, the skill of making useful theories.
Data mining will not save us, any more than Hoover’s FBI files ended crime. Before 1859, how many beaks and dog breeds had been studied by how many scientists in how many places? But it took Darwin, and his masters Malthus and Humboldt, to unweave that particular rainbow. Silver searches for patterns in the collective past and tries to extrapolate our common future. And if this was the same world over and over again, unchanging time after time, this would be unobjectionable. But it isn’t, and it’s not.
Speaking of the objectionable, the most-favored maw of chaos had a few choice words on the subject. “They’ve been wrong so much,” Trump said. “Nate Silver, I watched him. It destroyed his career. He was this big guru that never missed a call. He wasn’t even close. And he actually did say a few months ago, Trump is a whole phenomenon that’s a hard thing to figure. He sort of gave up.”
MEASURING
We, however, must not give up. To paraphrase an English author, quantitative politics has not been tried and found wanting; it has been tried and found difficult.
If I have one theme I return to, it is to argue for philosophy and politics in the business of society. Philosophy does not disagree with numbers. There are always numbers. Philosophy tells us if the arguments we make from those numbers is good. There is nothing wrong with making models of life. Models are enticing, because they sacrifice the possible for the certain. But the art of politics can show us how those models will work in a normal world, where not everything proceeds rationally. Abandoning politics means abandoning the competing dialogue of persons and interests—which is the reality of politics—for facsimiles which have nothing to do with the world. Silver’s failure does not mean that numbers are wrong, that counting is wrong, or that a great unified theory of politics is impossible. Rather, it argues that the models and theories we have now are childish, and we need to do better.
Our culture worships measurement, but we have no idea what to measure. We designed perfect scales, but can’t figure out what to weigh. Some things cannot be measured easily, or at all. We know the cost, but the value eludes us. This leads to the worship of the map, not the terrain. We ought to be better than Alexander, who when he approached India, wept, thinking there were no more worlds to conquer; he imagined he had reached the end of the world but had only approached the end of his narrow understanding. We should not weep that there are no more worlds to count. We have not even begun to count this one properly.