Why matching of CMIP5 model-simulated to observed warming does not indicate model skill
Guest post by Nicholas Lewis. Originally published at Judith Curry’s blog Climate Etc.
A well-known Dutch journalist, Maarten Keulemans of De Volkskrant, recently tweeted an open letter to the Nobel-prizewinning physicist Professor Clauser in response to his signing of the Clintel World Climate Declaration that “There is no climate emergency”, asking for his response to various questions. One of these was:
The CLINTEL Declaration states that the world has warmed “significantly less than predicted by (the) IPCC”. Yet, a simple check of the models versus observed warming demonstrates that “climate models published since 1973 have generally been quite skillful predicting future warming”, as Zeke Hausfather’s team at Berkeley Earth recently analysed.
The most recent such analysis appears to be that shown for CMIP5 models in a tweet by Zeke Hausfather, reproduced in Figure 1. While the agreement between modeled and observed global mean surface temperature (GMST) warming over 1970–2020 shown in the Figure 1 looks impressive, it is perhaps unsurprising given that modelers knew when developing and tuning their models what the observed warming had been over most of this period.
Figure 1. Zeke Hausfather’s comparison of global surface temperature warming in CMIP5 climate models with observational records. Simulations based on the intermediate mitigation RCP4.5 scenario of global human influence on ERF through emissions of greenhouse gases, etc. were used to extend the CMIP5 Historical simulations beyond 2005.
It is well-known that climate models have a higher climate sensitivity than observations indicate. Figure 2 compares equilibrium climate sensitivity (ECS) diagnosed in CMIP5 models and in the latest generation, CMIP6, models with the corresponding observational estimate on the same basis in Lewis (2022) of 2.16°C (likely range 1.75–2.7°C). Only one model has an ECS below the estimate in Lewis (2022), and most models have ECS values exceeding the upper bound of its likely range. CMIP6 models are generally even more sensitive than CMIP5 models, with half of them having ECS values above the top of the 2.5–4°C likely range given in the IPCC’s 2021 Sixth Assessment Report: The Physical Science Basis (AR6 WG1).
Figure 2. Red bars: equilibrium climate sensitivity in CMIP5 and CMIP6 models per Zelinka et al. (2020) Tables S1 & S2 estimated by the standard method (ordinary least squares regression over years 1–150 of abrupt4xCO2 simulations). Blue line and blue shaded band: best estimate and likely (17%-83% probability) range for ECS in Lewis (2022), derived from observational evidence over the ~150 year historical period but adjusted to correspond to that estimated using the aforementioned standard method for models.
So, how is it possible that Hausfather gets an apparently good match between models and observations in the period 1970-2020? Does it imply that the models correctly represent the effects of changes in “climate forcers”, such as the atmospheric concentration of greenhouse gases and aerosols, on GMST, and accordingly that their climate sensitivities are correct?
The key question is this. Matching by CMIP5 climate models, in aggregate, with observed GMST changes would only be evidence that models correctly represent the effects of changes in “climate forcers”, such as the atmospheric concentration of greenhouse gases and aerosols, on GMST if resulting changes in their combined strength in models matched best estimates of the actual changes in those forcers. The standard measure of strength of changes in climate forcers, in terms of their effect on GMST, is their “effective radiative forcing” (ERF), which measures the effect on global radiative flux at the top of the Earth’s atmosphere once it and the land surface have adjusted to the changes in climate forcers (see IPCC AR6 WG1 Chapter 7, section 7.3)
It is therefore important to compare changes in total ERF as diagnosed in CMIP5 models during their Historical and RCP4.5 scenario simulations over 1970–2020 with the current best estimates of their actual changes, which I will take to be those per IPCC AR6 WG1 Annex III, extended from 2019 to 2020 using the almost identical Climate Indicator Project ERF time series.
Historical and RCP4.5 ERF (referred to as “adjusted forcing”) in CMIP5 models was diagnosed in Forster at al. (2013), for the 20 models with the necessary data. I take the mean ERF for that ensemble of models as representing the ERF in the CMIP5 models used in Figure 1.
Figure 3 compares the foregoing estimates of mean ERF in CMIP5 models with the best estimates given in IPCC AR6. Between the early 1980s and the late 2000s CMIP5 and AR6 ERF estimates agreed quite closely, but they diverged both before and (particularly) after that period. The main reason for their divergence since 2007 appears to be that aerosol ERF, which is negative, is now estimated to have become much smaller over that period than was projected under the RCP4.5 scenario. Updated estimates of aerosol ERF also appears likely to account for about half of their lesser divergence prior to 1983, with the remainder mainly attributable to differences in ERF changes for land use and various other forcing agents.
Figure 3. Effective radiative forcing (ERF) over 1970–2020 as estimated in CMIP5 models (mean across 19 models) and the best estimate given in the IPCC Sixth Assessment Scientific Report (AR6 WG1). The ERF values are relative to their 1860–79 means.
The IPCC AR6 best estimate of the actual ERF change between 1970 and 2020 is 2.53 Wm−2. The linear trend change over 1970–2020 given by ordinary least squares regression is 2.66 Wm−2, while the change between the means of the first and last decades in the period, scaled to the full 50 year period, is 2.59 Wm−2.
By comparison, the mean ERF change for CMIP5 models between 1970 and 2020 is 1.67 Wm−2. The linear trend change over 1970–2020 is 1.92 Wm−2, and the scaled change between the first to last decades’ means is 1.76 Wm−2.
It is evident that the AR6 estimate of the actual 1970–2020 ERF change is far greater than that in CMIP5 models. Based on the single years 1970 and 2020, the AR6-to-CMIP5 model ERF change ratio is 1.51. Based on linear trends that ratio is 1.39, while based on first and last decades’ means it is 1.46. The last of these measures is arguably the most reliable, since single year ERF estimates may be somewhat unrepresentative, and due to intermittent volcanism the ERF has large deviations from a linear relationship to time. As there is some uncertainty I will take the ratio as being in the range 1.4 to 1.5.
So, CMIP5 models matched the observed 1970–2020 warming trend, but the estimated actual change in ERF was 1.4 to 1.5 times greater than that in CMIP5 models. On the assumption that both the CMIP5 model ERF estimates and the IPCC AR6 best estimates of ERFs are accurate, it follows that:
- CMIP5 models are on average 1.4 to 1.5 times as sensitive as the real climate system was to greenhouse gas and other forcings over 1970–2020
CMIP5 models would have over-warmed by 40–50% if their ERF change over that period had been in line with reality.
It seems clear that the ERF change in CMIP5 models over 1970–2020 was substantially less than the IPCC AR6 best estimate, and that CMIP5 models substantially overestimated the sensitivity of the climate system during that period to changes in ERF. Moreover, the divergence is increasing: the ratio of AR6 to CMIP5 model ERF changes is slightly higher if the comparison is extended to 2022.
In conclusion, Maarten Keulemans’ claim that “a simple check of the models versus observed warming demonstrates that “climate models published since 1973 have generally been quite skillful predicting future warming” is false.
Contrary to the impression given by Zeke Hausfather’s rather misleading graph, CMIP5 models have not been at all skillful in predicting future warming; they have matched the illustrated 1970–2020 observed warming (which was past rather than future warming until the late 2000s, when CMIP5 models were still being tuned) due to their over-sensitivity being cancelled out by their use of ERF that increased much less than the IPCC’s latest best estimates of the actual ERF increase.