In May 2010, the InterAcademy Council (IAC) was selected to “conduct an independent review of the IPCC processes and the procedures by which it prepares its assessments of climate change.” In June, economist David (P. D.) Henderson shared with MasterResource his rather critical comments submitted to the IAC which centered around the IPCC’s lax adherence to their own set of governing principles. In this article, we highlight several other submissions to the IAC that Dr. Henderson thought MasterResource readers may find particularly interesting.
Additionally, we offer a compilation of all other IAC submissions that we could find scattered across the web—a service that the IAC does not itself provide.
The IAC bills itself as “a multinational organization of science academies created to produce reports on scientific, technological, and health issues related to the great global challenges of our time, providing knowledge and advice to national governments and international organizations” and as such has been asked by the United Nations to:
[E]stablish a Committee of experts from relevant fields to conduct the review and to present recommendations on possible revisions of IPCC processes and procedures. In particular the IAC Committee of experts is asked to recommend measures and actions to strengthen the IPCC’s processes and procedures so as to be better able to respond to future challenges and ensure the ongoing quality of its reports.
Such a review has been welcomed by all quarters—and is especially relevant considering the revelations of the apparent shortcomings of IPCC procedures that have been revealed both directly from the contents of the Climategate emails as well as the increased scrutiny it has received as a result of the Climategate revelations.
The IAC invited and received a number of opinions from “knowledgeable experts and thoughtful observers regarding IPCC’s processes and procedures for producing assessments.” However the IAC does not seem to be making the submissions public prior to the release of the its final report (and even then, is planning on stripping the names from the comments). As many of these comments may be of immediate interest, we have collected together the IAC submissions that we could find scattered about the web. We include a link to these comments at the bottom of this article. Undoubtedly there are others that we have overlooked. If any one knows of any that we have missed, please feel free to draw our attention to them (providing links if possible ) in the Comments section of this post, and we’ll do our best to add them to our main collection.
Comments by David Henderson
In his comments to the IAC, shared with MasterResource in the article “Reforming a Flawed Process: The IPCC and Its Clients,” David Henderson was particularly critical of the IPCC’s lack of adherence to its own set of governing principles:
In the ‘principles governing IPCC work’, laid down by the Panel’s member governments, the IPCC is required to conduct its assessments on ‘a comprehensive, objective, open and transparent basis’. This requirement has not been met. The process of preparation of the Assessment Reports is far from being a model of rigor, inclusiveness and impartiality. It has shown itself to be professionally flawed.
Henderson detailed evidence of IPCC flawed procedures and made suggestions for improvement in the areas of disclosure, inclusiveness, and audit.
All sound like good ideas.
David has drawn our attention to two other sets of comments that provide further support for these general ideas—comments by economists Ross McKitrick of the University of Guelph and Gordon Hughes of the University of Edinburgh.
Each has submitted lengthy comments for the IAC’s consideration describing IPCC weaknesses and offered ideas for fixing them. We excerpt from the comments here, and encourage our readers with greater interest to spend some time with the full comments.
Ross McKitrick lays out a 13-step procedure for writing new IPCC reports that notably established a 21-member Executive Advisory Board (EAB) largely made up of qualified individuals whose areas of expertise lie outside the traditional field of climate science, such as “mathematics, statistics, physics, engineering, chemistry, economics, biology, medicine, computing, and other areas.” The EAB would serve as advisors to the IPCC Chapter authors when questions arose on technical issues and when authors and reviewers disagreed. The EAB would serve as a check, keeping IPCC authors from steering the text in a direction that doesn’t well-reflect the full range of scientific knowledge.
McKitrick’s general recommendations to the IAC are well summarized in his response to the IAC’s request “Comment on the sustainability of the IPCC assessment model. Do you have any suggestions for an alternative process?”
He offers these well-reasoned thoughts:
I have outlined an alternative process above. Those suggestions are focused on WGI [Working Group I]. As for WGII and WGIII I see little need for them since the sponsoring governments appear to make very little use of their reports. I would suggest WGIII simply be abolished and WGII be reformed along the lines I suggested for WGI.
I think you should also recognize that the IPCC began before the Internet did, and its structure is now obsolete. It adopted a rigid bureaucratic structure that had some relevance in the days before the internet imposed deep transparency on public organizations. But times have changed, and public expectations have evolved. Henceforth, from the start of the chapter review process, the attention of international bloggers will be intense, and every aspect of the report-writing process will now be done in a fishbowl. Without major reforms to the process, the next Assessment Report will simply explode on impact. All it will take is for one error to be found, or one email to be leaked, or one graph to be manipulated, and the entire report will be discredited.
This is not because there are armies of nasty bloggers out there who are being unreasonable (although even if there are armies of nasty bloggers out there, they are not going away so you need to find a process that can still function in their presence). It is because the IPCC has become one-sided and brittle, and has no real ability to cope with legitimate differences of opinion. That makes it inevitable that there will be growing numbers of critics who see it as biased and insular. The choice is whether simply to press onwards with the hope the IPCC will somehow regain its former glory, or to consider whether the critics actually have a point, in which case the process is in need of correction.
McKitrick’s full set of comments are available here.
Gordon Hughes Comments
Gordon Hughes’ comments offer a bit of a different take than do those of Ross McKitrick and concentrate on the poor handling of the reports from IPCC Working Groups II (impacts and vulnerabilities) and Working Group III (mitigation). Hughes offers up this summary of the IPCC problems and potential solutions:
The problems in the IPCC process stem from a confusion of assessment and advocacy. Some parts of the AR4 report are simply outrageous if it is a body that is expected to provide a careful and dispassionate assessment of the evidence that is currently available. In many areas, the dispassionate conclusion – for the time being – must be that we do not know. Clearly, that answer is viewed as not being acceptable by those who are firmly convinced that something – anything – should be done to address what they believe to be the consequences of climate change. But, passionate opinions must not be treated as evidence.
If the Review Panel considers that it is appropriate for the IPCC to be an advocate for some action designed to address issues of climate change, then let us be clear that this is the way in which it operates and everyone can stop pretending that it provides a dispassionate assessment based upon evidence alone.
On the other hand, if the Review Panel considers that the IPCC’s role is to provide independent assessment rather than advocacy, then the body must step back from any involvement in discussions of how to respond to the possibility of climate change. It cannot, for example, be involved in any negotiations concerning future treaties about climate change other than at a purely technical level. This should apply not only to the IPCC as an organisation but also to any individual who plays a substantial role in carrying out the assessments and writing the reports. This is a standard principle for avoiding conflicts of interest which is routinely applied to judicial bodies and must operate in this context.
David Henderson’s submission has addressed matters of disclosure/transparency, inclusiveness and audit. All of these are important and relevant if the IPCC is to carry out a function of independent assessment. I would add that the IPCC should not attempt to create a consensus when none exists. It can, of course, report differences of evidence and interpretation of that evidence. It might call for additional investigations which could shed light on or even settle such differences.
I would also emphasise that there can be no mixing of advocacy and assessment. This particular well is so polluted with mistrust and confusion that the IPCC has to go down one route or the other. Indeed, it may now be the case that no one will believe that the IPCC is capable of acting as an independent body charging solely with the responsibility for assessing the evidence concerning climate change. If that is, indeed, what the Review Panel believes that its function should be, then something close to a new start is required with a very clear break from the past methods of working and behaviour.
At an absolute minimum, no chapter or Working Group report should proceed to publication unless and until it can be positively demonstrated that the team responsible have made all reasonable efforts to canvas and incorporate the full range of evidence and analysis relevant to the topic of the chapter or report. This is not merely a matter of soliciting such material, but a positive obligation to seek it out and incorporate it in the final text. Reporting that there is an agreement to differ and presenting two or many sides of one or many issues is acceptable, but any failure to report on differences should lead to suspension of publication and, if necessary, replacement of the authors and/or editors of the chapter or report.
It is sad but not entirely surprising that the work of the IPCC has become so contentious. The world does not lack passionate advocates for action to address climate change. But there is no other organisation that is equipped to undertake the essential function of dispassionate assessment. If that is to be the role of the IPCC in future, then a clear set of rules designed to avoid all actual or apparent engagement in activities that might be classed as advocacy must be instituted. Countries with experience of quasi-judicial independent panels or inquiries can provide various models of what is required to avoid conflicts of interest. But it is important to realise that the primary job of a reformed IPCC should not be to reach definitive conclusions but to report on the evidence that has been submitted and to present or clarify matters of dispute. Reporting on the examination of such evidence is part of that job, but the outcome may well be a conclusion that no particular case can be regarded as more likely or proven.
The full set of comments to the IAC from Gordon Hughes are available here.
And for those interested in other viewpoints on how the IPCC should go about conducting itself, here is our compilation of those submissions to the IAC that we have come upon thus far (and thanks in advance for our readers for helping us assemble the most complete list possible).
Comments to the IAC
David Henderson (Academic Advisory Council of the London-based Global Warming Policy Foundation): http://www.masterresource.org/2010/06/reforming-a-flawed-process-ipcc/
Ross McKitrick (Professor of Economics, University of Guelph, Guelph, Ontario)
Gordon Hughes (Professor of Economics, University of Edinburgh)
Richard Tol, Professor of the Economics of Climate Change at the Vrije Universiteit, Amsterdam, The Netherlands
Roger Pielke Sr. (Professor Emeritus of the Department of Atmospheric Science, Colorado State University)
Marcel Crok (science journalist):
Presentations to the IAC
John Christy (Distinguished Professor of Atmospheric Science, Director, Earth System Science Center and Alabama State Climatologist at University of Alabama in Huntsville):
Kiminori Itoh (Engineering Department at the Yokohama National University):
Some other presentions to the IAC are also available at the IAC website, including those by:
Hans von Storch (Professor at the Meteorological Institute of the University of Hamburg, Germany):
Christopher Field (Department of Global Ecology, Carnegie Institution of Washington)