Varun Sivaram

Energy, Security, and Climate

CFR experts examine the science and foreign policy surrounding climate change, energy, and nuclear security.

Print Print Cite Cite
Style: MLA APA Chicago Close


Thoughts on a New Methane Study

by Michael Levi
August 6, 2013


A new paper in press at Geophysical Research Letters (GRL) claims to show methane leakage of between 6.2 and 11.7 percent in Uintah County, Utah. This is the same study that got pre-review press in January after a co-author claimed, in a conference presentation, to have observed 9 percent leakage. The study team, which includes many of the same people who claimed last year to have observed massive methane leakage in Colorado, once again uses their results to question the “bottom-up” estimates upon which the EPA relies, saying that those lowball actual emissions.

The measurements and data analysis in the new study are far better than those in the previous one. The previous study, some readers of this blog will recall, did not hold up well under scrutiny. In particular, once reanalyzed, the observations in that study indicated an amount of methane leakage consistent with bottom-up estimates.

The new study used a different methodology that requires fewer assumptions and produces more reliable estimates of total methane leakage. Most important, unlike the previous study, this one includes real and serious uncertainty estimates. There’s nothing on the surface of those that should lead one to question their validity.

My concerns with this paper are subtler. In particular, contrary to some coverage, I don’t think that the paper actually says much about emissions from “fracking” per se, and I don’t think its results are seriously inconsistent with the bottom-up estimates that are already out there, particularly once one reads the uncertainty estimates correctly.

The paper’s authors have a challenge in attempting to convert their observations of absolute methane emissions to a percentage leakage rate. They only have three hours of observations, and no direct way of knowing whether those observations are representative of methane emissions over longer periods of time. Yet they only have monthly gas production records. They therefore need to assume that the observed emissions are representative in order to obtain a percentage leakage rate.

The authors thus make two arguments for why their observed emissions are representative. First, they argue, emissions appear to be dispersed, rather than coming from a few concentrated sources. This, they note, suggests that emissions aren’t due to a handful of activities whose intensity might vary substantially from hour to hour or day to day. Let’s assume that this is correct. This immediately yields an important conclusion: the emissions aren’t primarily due to well completions (“fracking”), of which there were only four on the day that the team made their observations. Instead they’re due to a more dispersed set of activities. We’ll come back to this in a moment; it turns out that there’s good reason to believe it’s true.

The second argument that the paper makes, which allows people to conclude that fracking itself may be the problem, is less persuasive. The paper argues that well spudding and completion activity during the week surrounding its observations is normal. What it fails to mention is that well spudding and completion activity on the day of its observations is more than two times the normal level (all data obtained from the Utah DNR). If anyone wants to claim that fracking itself (rather than activity surrounding oil and gas production in general) is generating the observed methane leaks in Uintah basin, they need to contend with and account for this fact.

There is additional reason, however, to believe that most emissions in the basin are due to gathering, processing, and transmission, rather than fracking per se. The GAO, in a 2011 study that’s cited in the new paper, published estimates for methane emissions in the Uinta basin. (The new paper conflates these Uinta basin estimates, which cover five counties, with their own Uintah county observations, but I’m going to follow their lead and assume for now that the difference isn’t too important.) The GAO study estimates that a full 93 percent of fugitive methane emissions in the area come from pneumatic devices and glycol dehydrators – equipment used in gathering, processing, and transmission, not in fracking itself. Indeed the GAO estimates only 4 percent of emissions come from well completions.

Which brings me to the final important point. The authors acknowledge that the GAO study found an unusually high methane leakage rate for the Uinta basin – 5.1 percent – using standard bottom-up methods. (The authors of the new paper claim that this could be partly due to flaring; the fine print in the GAO report shows that that is insignificant at best.) This lies outside the paper’s reported 6.2-11.7 percent leakage (though if the GAO estimate were to come with even a modest uncertainty range, as would undoubtedly reflect reality, there would likely be some overlap). But the reported leakage in the paper is, inexplicably, described using a 1-sigma confidence interval, which means that there’s a one-in-three chance that the actual methane leakage lies outside the reported range. Using a more conventional 2-sigma interval, which gives 95 percent confidence instead, one finds a range of 3.5-14.5 percent leakage. That’s entirely consistent with the previous bottom-up estimates.

Looking at the data this way allows one to avoid hasty extrapolation of the Utah results to other fields. For example, the same GAO study that estimated 5.1 percent leakage in Uinta estimated 2.1 percent leakage in Denver-Julesburg using a basic bottom-up method. That number – perhaps this isn’t a surprise anymore – is generally consistent with the correct interpretation of the earlier top-down emissions observations of that field. It’s not, however, consistent with the more outlandish (and highly publicized) claims of massive leaks.

To be clear, leakage in the neighborhood of 5 percent is a bad thing, even if it still doesn’t come close to making gas worse for climate change than coal. The new study usefully confirms that previous estimates are in the right neighborhood. There are straightforward technological requirements that can be imposed on gathering and processing systems that would greatly cut down on this leakage. But continuing to fixate on fracking itself, and particularly well completions, distracts from this problem. And continuing to attack the way EPA estimates methane emissions, to the extent that such attacks are ill founded, only serves to confuse policymaking and public debate.

Post a Comment 3 Comments

  • Posted by David McCabe

    Hi Michael,

    You claim that the bottom up EPA inventories are accurate. Have you looked into them very much – the way they have varied over the last couple years, or some of the underlying assumptions?

    Between 2010 and 2011 EPA raised their estimate of the emissions from production (wellpads and gathering systems) by more than a factor of 8. Between 2012 and 2013, they decreased their estimate by about 40%. Your confidence in the accuracy of the bottom up inventory seems strange just in light of that fact.

    Take a look at some of the details of the inventory. The massive table A-124 in Annex 3 of the inventory (available at has lots of interesting stuff in it. Consider that the inventory only lists ~120 large compressors in gas gathering, nationwide. Or, compare the emissions factors for liquids unloading from different regions. It’s quite clear that there are real uncertainties in the emissions factors and activity data that EPA uses. As the huge changes year-to-year mentioned above show, those uncertainties matter at the macro level.

    By the way – ambient measurements have been indicating that there seems to be more ethane (which is mostly from nat gas) in the air then one would expect based on the EPA methane inventories, for some time. The Katzenstein paper that Karion et al cite noted this some years back. A group from Harvard (with some of the same authors as the Katzenstein paper) published a paper in 2008, based on different data, that came to a similar conclusion. These observations aren’t that new.

    The WRAP inventory, which the GAO report used, is a bit different – based on operator surveys. But, those data are (essentially) a black box, since the survey results are not available at a granular level.

    Regarding the GAO study, what is your basis for claiming that “the fine print in the GAO report shows that that is insignificant at best?”

    Finally, using 2sigma uncertainty bands is fine. Based on this study, they mean there is an equal probability of a 14.5% leak rate as a 3.5% leak rate, on that day in this basin. Error bars go both ways.

  • Posted by Jim Welke

    “…continuing to fixate on fracking itself, and particularly well completions, distracts from this problem.”

    One inevitably fixates on fracking; that’s where the vast majority of new production comes from and thus new leaks. Fracking is the problem, at least as far as ecologically sustainable energy production is concerned. Improved efficiency and distributed renewables are the proven, cost-effective, environmentally safe, workforce favorable solution. See:

  • Posted by Chris Busch

    Regarding the Colorado study that you write, “did not hold up well to scrutiny.” You cite your October 2012 blog on the topic. Since that time, the authors of that study have countered your criticism.

    Pétron, G., et al. “Reply to comment on “Hydrocarbon emissions characterization in the Colorado Front Range—A pilot study” by Michael A. Levi, J. Geophys. Res. Atmos., Vol.118, 236–242

    I would be curious to know how you would reply to their response to your critique. Thanks.

    [ML: See my peer reviewed response in JGR:

Post a Comment

CFR seeks to foster civil and informed discussion of foreign policy issues. Opinions expressed on CFR blogs are solely those of the author or commenter, not of CFR, which takes no institutional positions. All comments must abide by CFR's guidelines and will be moderated prior to posting.

* Required