Varun Sivaram

Energy, Security, and Climate

CFR experts examine the science and foreign policy surrounding climate change, energy, and nuclear security.

Print Print Cite Cite
Style: MLA APA Chicago Close


FiveThirtyEight’s Data Problem

by Michael Levi
March 24, 2014


Nate Silver’s new FiveThirtyEight has been catching a lot of flak since it launched last week. Perhaps the harshest has been directed at the site’s retention of the often-contrarian climate analyst Roger Pielke Jr., with everyone from Paul Krugman to the Center for American Progresspiling on. The onslaught is disturbing. I’ve disagreed with Roger often, but he is genuinely well intentioned. People who care about getting good policy should want more thoughtful voices, not fewer, proposing options – and organized campaigns to run heterodox thinkers out of town are awfully ugly.

But that doesn’t mean I’m impressed with the new FiveThirtyEight. Indeed it’s another energy post – “U.S. and Chinese Current Accounts Converging” – that’s troubling.

The post starts out with an arresting chart (reprinted at the top of this post) that shows an impressive improvement in the U.S. current account balance. The author then explains what’s happened:

“In the U.S., a natural gas boom is cooling demand for imported petroleum, and oil represents a huge share of American imports. The dollar, meanwhile, has depreciated, boosting American exports.”

This is certainly the conventional wisdom. But what one expects from FiveThirtyEight is a data-driven interrogation of that conventional wisdom. Alas that would have produced a different result.


Here’s another chart that combines the data presented by FiveThirtyEight (the blue line – it looks different because I’m showing absolute numbers rather than percentage of GDP) with three other series. Start by comparing the blue and red lines. These seem to move together beginning in 2009. In particular, what you’ll see is that the big shift in the current account comes in 2008-9. Indeed the decline in the oil trade balance (the red line) accounts for a little more than half the decline in the current account balance.

So far so good: changes in oil trade seem to explain a lot of what’s happening with the current account.

But now look at the dashed lines.

The green dashed line shows that net petroleum and product imports have been declining steadily since about 2007. This raises a first problem with FiveThirtyEight’s analysis: if falling oil imports explain so much of the lower current account deficit, why hasn’t the current account deficit continued to plunge alongside oil imports? The answer is pretty straightforward: oil prices have gone up. The United States imported 64 percent as much oil in 2013 as in 2009, but the average cost of an imported barrel was 66 percent higher.

It’s the purple dashed line, though, that’s the most damning. What that line shows is that in the span that the U.S. current account balance really shifted – 2008-9 – U.S. oil production had barely begun to pick up. It was only later, beginning in 2011-12, that output really took off. But that’s not the period when the big change in the current account balance appeared. (Had I added in natural gas liquids, which started rising earlier, the comparison would look a little better, but the basic problems would remain.) Had FiveThirtyEight actually juxtaposed the current account balance with oil trade data, it wouldn’t have ratified the conventional wisdom.

Indeed one shouldn’t be surprised with this result. As Robert Lawrence showed in a paper that my program published in January, there are strong theoretical reasons to be skeptical of claims that falling U.S. oil imports and rising U.S. oil production will substantially reduce the U.S. current account deficit.

Which gets to the heart of FiveThirtyEight’s challenge. They want to do theory-free data analysis. But without at least some sort of theory, you at best need to investigate a much larger volume of data in order to get useful results. (That’s what Nate Silver did so well with election predictions and baseball statistics.) At worst, if your data isn’t good enough, theory-free analysis leaves you with nothing. Here’s hoping that FiveThirtyEight will be disciplined enough to stick to analyses where its unusual approach works.

Post a Comment 4 Comments

  • Posted by Peter

    I find it interesting that the drop in US oil imports, and therefore the narrower trade deficit, is always attributed solely to the boom in oil production since 2009 or so. Yes, indeed the increase in US oil production has been crucial, and substantially so.

    However, between 2005 and 2012 US oil demand fell about 12%, well over 2 million b/d. Yet this trend on the consumption side is rarely referred to as having any impact at all on the fall in imports, and therefore of lower petroleum trade deficits. It also has probably contributed to refiners being able export oil products, with domestic demand falling and then remaining stagnant between 2005 and 2012.

    Often trends are not attributed to just one factor.

  • Posted by Doug Cotton

    There can be no correlation found between temperature data and carbon dioxide, because it cools only by about 0.1 degree. You need to come to grips with the new 21st century paradigm shift in climate science based upon the gravito-thermal effect.

    This is how absurd the old 20th century paradigm of greenhouse radiative forcing gets. They claim that you can work out Earth’s surface temperature by adding together the radiative flux from both the Sun and the colder atmosphere, and then bunging this total value into the Stefan-Boltzmann equation and out pops your answer 287K or 288K. Well it might well do if you fiddle the back radiation and then use the emissivity value instead of the absorptivity.

    But there’s absolutely no physics to support the calculations. When you consider that about 70% of the surface is a thin transparent water layer, it is obvious that the solar radiation which mostly (like over 99%) passes through this layer into the thermocline is not what is determining the temperature of that thin surface layer. In fact the mean temperature of the thermocline is obviously less, and the mean temperature of all the ocean water is less again.

    Oh, and the back radiation doesn’t even enter the surface layer – it just raises electrons between quantum energy states momentarily, and then those electrons immediately emit another photon which climatologists think is energy coming from the kinetic energy in the surface molecules, but it’s only electro-magnetic energy from the back radiation being thrown back in their red faces.

  • Posted by David B. Benson

    I deeply doubt that the anti-RPJr campaign is actually organized. It is rather than many on many are able to recognize the errors of his statistical ways.

  • Posted by Stephen

    In addition to your critique, what about the simple fact that natural gas and oil are not substitutes for one another? You imply this by focusing on the uptick in domestic oil production (the dotted purple line), but their initial assertion was that “In the U.S., a natural gas boom is cooling demand for imported petroleum, and oil represents a huge share of American imports.”

    Maybe if they had focused on oil instead of gas, they would’ve stumbled onto your conclusion that the increase in oil production occurred too late to explain the earlier decline in the current account deficit.

Post a Comment

CFR seeks to foster civil and informed discussion of foreign policy issues. Opinions expressed on CFR blogs are solely those of the author or commenter, not of CFR, which takes no institutional positions. All comments must abide by CFR's guidelines and will be moderated prior to posting.

* Required