Medium confusion: the IPCC's approach to communicating uncertainty

Rajendra Pachauri, Chairperson of the Intergovernmental Panel on Climate Change. Photo by PopTech via Flickr
Mar 2014

The latest international climate assessment was published today. Jon Turney gives a personal assessment of how the media may interpret the way the new report handles confidence levels and uncertainty.

“How confident are scientists that Earth will warm further over the coming century? Very confident”. That simple declaration comes from the 2014 summary of evidence about global change published jointly by the British Royal Society and the US National Academy of Science.

But while that will do very well as a summary of one facet of the scientific consensus, pretty much everything else about the future is less clear. The Intergovernmental Panel on Climate Change – whose latest mega-report is published today (March 31) – strives to take this into account, and pays close attention to the language it uses to do that. As with the continual improvement in climate modelling, however, it may not always make things much clearer.

The fifth assessment from IPCC Working Group Two (WGII) on Impacts, Adaptation and Vulnerability (a further report from Working Group Three later this year completes the new set) is the latest installment of perhaps the largest literature review in progress ever seen on a specific field. It covers many topics and disciplines, and since the 1990s there has been a sustained effort – owing much to the late Stephen Schneider – to agree a common language for expressing uncertainty in IPCC documents. Most obviously, this has been done by affixing numbers to words such as “likely” to give a better guide to what probability they are supposed to indicate.

Working Group Two have come under pressure to exercise extra care in their latest assessment, after a couple of embarrassing errors made it into their Fourth report – most notably a mistaken date for the possible disappearance of Himalayan glaciers, indicated as 2035 instead of 2350. Part of their response has been to embrace the same linguistic discipline to underpin their effort to communicate the consequences of climate change and how governments might respond to them.

For the first time, the IPCC has decided that the agreed usage of various terms relating to the “degree of uncertainty in assessment findings” should be the same across all three working groups. So WGII uses the quantified probability terms – “extremely likely” means 95-100 per cent probability, for example – although they do not crop up very often in the latest report.

Appearing more frequently are other agreed terms relating to evidence – which is described as limited, medium, or robust and - degree of agreement – which is high, medium, or low. So far, so good. However, there is also yet another scale, said to be a synthesis of the level of evidence and degree of agreement, which denotes the degree of confidence in a statement, from very low to very high).

There are times when this results in confusion. The Summary for Policy Makers does not indicate any numbers to go with these terms, which is a concern as there is now plenty of research to suggest that if readers are asked to interpret words like this they come up with a wide range of quantitative translations. The protocol the WGII authors are following also means that there are paragraphs in the Summary where there is a simple (“high confidence” or “medium confidence”) at the end of some general statement, which just seems like saying, "we think that" or "it is our best judgment that”.

You can see the struggle they have had to be clear about these levels of uncertainty in this paragraph from the final published version, for example.

“Increasing magnitudes of warming increase the likelihood of severe, pervasive, and irreversible impacts. Some risks of climate change are considerable at 1 or 2°C above preindustrial levels... Global climate change risks are high to very high with global mean temperature increase of 4°C or more above preindustrial levels in all reasons for concern… and include severe and widespread impacts on unique and threatened systems, substantial species extinction, large risks to global and regional food security, and the combination of high temperature and humidity compromising normal human activities, including growing food or working outdoors in some areas for parts of the year (high confidence). The precise levels of climate change sufficient to trigger tipping points (thresholds for abrupt and irreversible change) remain uncertain, but the risk associated with crossing multiple tipping points in the earth system or in interlinked human and natural systems increases with rising temperature (medium confidence).”

The judgment about tipping points is crucial because there is an argument being made by some that the IPCC reports taken overall can now be read to indicate that warming this century which does not cross thresholds that tip the global climate system into a new regime will be manageable in most respects. The IPCC’s latest science report, last year, did predict surface warming of 4°C without mitigation, but not until the very end of the century. However, that projection does not take tipping points into account. So persuading some of the case for urgent – and expensive – mitigation efforts rests heavily on the risk of lurching into a much more damaging phase of climate change.

With that in mind, how does this paragraph come across? It seems clear that the IPCC working group’s opinion is that if climate warming increases, the effects get worse. No surprise there. But does the “medium confidence” at the end relate to the fact that we are uncertain about the levels of climate change that might trigger bigger changes, or only to the increase in risk of crossing tipping points as temperature rises? It is hard to be sure.

This may simply arise because the Summary for Policy Makers is highly compressed, and the whole notion of tipping points is quite difficult to explain simply. But it appears in a report which as Fred Pearce explains very well represents an important shift away from detailed predictions and toward highlighting climate policy as part of an overall risk management strategy.

That being so, managing this aspect of the risk where there is a small probability of really bad things happening, is a vital part of the discussion. The limiting cases of risks like this, dubbed zero/infinity risks by analysts – to indicate there is a tiny chance of some catastrophically bad outcome - have been difficult to deal with in many other policy arenas, from nuclear power to bioterrorism.

As Pearce notes, the IPCC report itself emphasises that, “Responding to climate-related risks involves making decisions and taking actions in the face of continuing uncertainty about the extent of climate change and the severity of impacts in a changing world.”

So it is worth thinking about how to improve on this approach in further assessment rounds, and in presentation of climate-related predictions more generally. Several researchers, including the veteran analyst of the psychology of risk Baruch Fischhoff of Carnegie Mellon University and Nick Pidgeon of Cardiff University in a paper in Nature Climate Change in 2011, have called for stronger involvement of social scientists in the work, especially on strategies for communicating about uncertainty.

The current conventions will probably survive now the IPCC has invested a good deal in developing them. James Painter of the Reuters Institute for the Study of Journalism, and author of a recent book-length study of how risk and uncertainty around climate change gets reported reckons the IPCC is on the right track. “The probability assessments are easier to get than the confidence levels, but you do need both”, he says. David Spiegelhalter, Professor of Public Understanding of Risk at the University of Cambridge agrees: “I have a lot of sympathy with the efforts of the IPCC at assessing and communicating both quantifiable probabilities (when appropriate) and 'confidence' in the underlying evidence”.

 Painter indeed suggests that “low probability, high impact events (such as some tipping points)” are a good example of why you need both. He goes on, “If you say that it is low probability then policy makers might say, ‘Oh we don’t need to worry then’.  But if you say that you have high confidence that even though it is low probability, it will have major impacts, then they might sit up and take notice”.

So it looks as if we are likely (high confidence) to be seeing this kind of language in climate reports for the foreseeable future – and it may be a topic Future Earth can include in its expanding agenda with a view to eliminating the kind of ambiguities we have seen in the past. Meanwhile, some may hanker after the simpler approach adopted again by the Royal Society and the NAS. On tipping points, their report just says: “Results from the best available climate models do not predict abrupt changes in such systems (often referred to as tipping points) in the near future. However, as warming increases, the possibilities of major abrupt change cannot be ruled out”. But then, as they also say, while “Such high-risk changes are considered unlikely in this century”, they are “by definition hard to predict”, which presumably denotes a lasting lack of confidence in their absence.

We welcome more perspectives on the IPCC WGII and III reports in the coming days and weeks. Why not add yours in the comments?