A free-market energy blog
Random header image... Refresh for more!

Poorly Defined Climate-Change Questions Lead to Meaningless Poll Results

The American Geophysical Union’s house organ, Eos, has an article entitled “Examining the Scientific Consensus on Climate Change,” written by Peter Doran and Kendall Zimmerman of the University of Illinois at Chicago. (h/t Roger Pielke, Sr.)

The paper explains that the two “primary questions” asked were:

1. When compared with pre-1800s levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?

2. Do you think human activity is a significant contributing factor in changing mean global temperatures?

The article reports that over 90 percent of the responders agreed with the statement that mean global temperatures have “risen” relative to the pre-1800 levels (Question 1). But this is unremarkable. Everyone knows that in the 1700s and early 1800s the world was in the grip of the Little Ice Age, so a recovery toward normalcy alone would lead to global warming. No one I know — and I know plenty of skeptics (not yet a criminal offense, I hope) — has ever disputed this point. So the real surprise to me is that it wasn’t 99 percent–plus!

With respect to the second question, the authors report that 82 percent agreed that “human activity is a significant contributing factor” in the increased global temperature.

But what precisely does this mean? How did respondents interpret the terms “human activity” and “significant contributing factor”?

I am reminded that under the Clean Air Act, an area that has relatively clean air is deemed to be “significantly deteriorated” if the concentration of an air pollutant in that area increases by 2–8% of the National Ambient Air Quality Standard. Under this interpretation, a contribution of 2–8% would be significant! So the question is: What instructions or guidance, if any, was provided to respondents in interpreting what precisely was meant by “significant contributing factor” in the second question? Is a 2% contribution significant? Why not 20%, or 40%, or 60%?

In addition, what exactly is the “human activity” that is supposed to be significantly contributing to global warming? Is it land clearance, paving over surfaces, producing black-carbon emissions, generating well-mixed greenhouse gases, or a combination of all these activities? I claim that most people would interpret “human activity” to encompass “all of the above.”

Moreover, how do the researchers know that there is a uniform understanding of the question, and what precisely is that uniform understanding?

Given that the two key terms are imprecisely defined, a person who believed that well-mixed greenhouse gases have contributed no more than, say, 10% to the global warming since 1800 might be compelled to agree that “human activity is a significant contributing factor” to post-1800 global warming.

From the perspective of policy makers, to whom this article is partly addressed, identifying the precise kind of human activity that contributes significantly to global warming (however defined) is absolutely critical. If well-mixed greenhouse gases (GHGs), for example, carbon dioxide, do not contribute significantly, then no amount of GHG reductions will make a significant difference (however defined) to the degree of future warming. Thus, the imprecise wording of Question 2 precludes this poll from closing the debate among policy makers.

The fact that the article doesn’t raise, let alone address, the issues posed by its inexact question indicates that the Eos article is premature, if not erroneous, in its claim that “the debate on the authenticity of global warming and the role played by human activity is largely nonexistent among those who understand the nuances and scientific basis of long-term climate processes.” It also reflects poorly on the pollsters and Eos reviewers — assuming it was reviewed — that they missed such obvious difficulties with the study.

The article also claims that ‘[t]he challenge, rather, appears to be how to effectively communicate this fact to policy makers and to a public that continues to mistakenly perceive debate among scientists.”

But the real challenge is: How can pollsters accurately communicate their questions to potential respondents so that there is a common and precise understanding on everyone’s part regarding the question posed. Without this, one is condemned to obtain meaningless poll results. And the rest of the world is subjected to reading (and reading about) one more poorly designed study.

5 comments

1 Ed Reid { 01.23.09 at 12:22 pm }

I think you miss the point.

I don’t believe there is any concern that the poll results are meaningless, as long as they support the ongoing narrative.

2 jae { 01.23.09 at 6:33 pm }

Those are absolutely the most inexact and meaningless poll questions I have yet seen. I would bet that the professional pollsters would get a good laugh. I think it is entirely within the realm of possibility (in fact, highly likely, considering the topic) that the questions were deliberately worded this way in order to elicit certain hoped-for results. Climate science seems to ignore some important scientific principles that are demanded by the other sciences. One is that biasing the outcome by flawed procedures is perfectly acceptable (as in the case at hand). So, in paleoclimatology, for example, it seems to be perfectly acceptable to select those proxy series that show the type of response (“temperature signal”) you desire and throw out or truncate those that don’t (which is how the flawed “hockey stick” was produced). For the surface temperatue records, it’s apparently OK to ignore any ancillary information that is important, such as the satellite data, the Urban Heat Island effect studies, missing data, the decline in the number of stations, etc., etc. Also, it appears that there are a lot of brand new (and unproven) statistical procedures being invented by climate scientists. I don’t see how any scientist worthy of the title can stomach this stuff. Perhaps most of them just don’t study the issue enough to realize what is going on.

3 CGB { 01.24.09 at 10:23 am }

A highly touted poll re what the public (Virginians) thinks about climate change may be found in the archives of the Miller Center in Charlottesville, VA. The October 21, 2008 survey report purported to show convincing evidence of public concern and support for climate change mitigation. The Miller Center then devoted a two day conference, December 11-12, 2008 to developing political remedies to mitigate the global warming crisis seemingly identified by the prior report.

The framing of the questions and the lack of definition of “climate change” in the initial survey would seem to fit the profile of the Eos format and make the survey results subject to similar critical comments. Meanwhile the moving finger having writ, it has moved on and has overwritten this survey with the latest Pew Reseach Center survey wherein the fickle public has placed global warming dead last in the 20 things to worry about in 2009.

See: http://millercenter.org/scripps/archive/panel/detail/4051

and http://millercenter.org/scripps/archive/conference/detail/4026

4 David Wojick { 01.26.09 at 10:24 am }

Indur, I am a skeptic who believes we do not know if it has warmed or not, so now you know one. The problem is that the data and goofy area-averaging method used to estimate global temperatures for the last 150 years are not accurate to within the estimated temperature change. The first poll question should have included the option “we don’t know.” As for the little ice age its existence is disputed by the warmers so it is not something everyone knows happened. In short, the climate change debate is really much deeper than you or the poll allow for. Cheers.

5 TokyoTom { 02.04.09 at 12:41 am }

Indur, it seems to me that you’re criticizing the poll for things it does not try to be. It is admittedly a very high-level view of climate scientists on wether human activities influence climate, and does not attempt to inform policy-makers further (for which other resources, such as AR4 are available).

If you want a more useful poll, why don’t you design and run one, and publish the results here? Perhaps the poll by Fergus Brown, James Anna and Pielke Sr – http://climatesci.org/2008/02/22/is-there-agreement-amongst-climate-scientists-on-the-ipcc-ar4-wg1/ – might make a good start, taking into account some of the criticisms noted in comments to Fergus Brown? http://fergusbrown.wordpress.com/2008/02/24/is-there-agreement-among-climate-scientists/

Leave a Comment