8.1.2.2 Metrics of Model Reliability

What does the accuracy of a climate model’s simulation of past or contemporary climate say about the accuracy of its projections of climate change? This question is just beginning to be addressed, exploiting the newly available ensembles of models. A number of different observationally based metrics have been used to weight the reliability of contributing models when making probabilistic projections (see Section 10.5.4).

For any given metric, it is important to assess how good a test it is of model results for making projections of future climate change. This cannot be tested directly, since there are no observed periods with forcing changes exactly analogous to those expected over the 21st century. However, relationships between observable metrics and the predicted quantity of interest (e.g., climate sensitivity) can be explored across model ensembles. Shukla et al. (2006) correlated a measure of the fidelity of the simulated surface temperature in the 20th century with simulated 21st-century temperature change in a multi-model ensemble. They found that the models with the smallest 20th-century error produced relatively large surface temperature increases in the 21st century. Knutti et al. (2006), using a different, perturbed physics ensemble, showed that models with a strong seasonal cycle in surface temperature tended to have larger climate sensitivity. More complex metrics have also been developed based on multiple observables in present day climate, and have been shown to have the potential to narrow the uncertainty in climate sensitivity across a given model ensemble (Murphy et al., 2004; Piani et al., 2005). The above studies show promise that quantitative metrics for the likelihood of model projections may be developed, but because the development of robust metrics is still at an early stage, the model evaluations presented in this chapter are based primarily on experience and physical reasoning, as has been the norm in the past.

An important area of progress since the TAR has been in establishing and quantifying the feedback processes that determine climate change response. Knowledge of these processes underpins both the traditional and the metric-based approaches to model evaluation. For example, Hall and Qu (2006) developed a metric for the feedback between temperature and albedo in snow-covered regions, based on the simulation of the seasonal cycle. They found that models with a strong feedback based on the seasonal cycle also had a strong feedback under increased greenhouse gas forcing. Comparison with observed estimates of the seasonal cycle suggested that most models in the MMD underestimate the strength of this feedback. Section 8.6 discusses the various feedbacks that operate in the atmosphere-land surface-sea ice system to determine climate sensitivity, and Section 8.3.2 discusses some processes that are important for ocean heat uptake (and hence transient climate response).