All Models Are Wrong

"All models are wrong, but some are useful" — George Box (1919-2013). 

Before I continue investigating the impact of climate change on disease spread, I thought it would be useful to pause and review how and why models are used to investigate these questions.

When climate change deniers criticise scientists, they often target their models. Joseph Bast, head of the conservative think-tank the Heartland Institute, claims "the most important fact about climate science, often overlooked, is that scientists disagree about the environmental impacts of the combustion of fossil fuels on global climate". 

The thing is, scientists are well aware that all models are wrong in some way. However, that doesn't mean that they can't be useful. 

Models are abstractions of reality. They are designed to represent complex systems in the simplest way possible that still reflects the essence of the situation. They can be used to make predictions about a system or to understand the phenomena we observe in the world. 

Information needs to be fed into the model for it to generate these useful outputs. The inputs used depends on the purpose of the model. For example, they can be estimates of future carbon dioxide emissions put into a climate model to predict future temperature increases. The diagram below gives an outline of how these steps fit together in the model-building process.

A schematic of the modelling process. Based on guidance from Marion (2008).

This generation of outputs from inputs is usually done using mathematics. Scientists come up with a set of equations based on the variables they want to measure, how those variables interact and the rate at which those variables change. In a simple model of malaria, scientists want to track variables such as the number of humans that are susceptible to the disease, the number of humans that are infected and the density of infected mosquito vectors. They can then track the rate at which certain events happen, such as mosquitos infecting susceptible humans, or demographic events (births and deaths) of both populations.

This example also highlights another benefit of modelling; it allows us to study scenarios for which we cannot run experiments. Scientists can't run large-scale experiments to determine how increasing temperatures will affect mosquito spread, but models allow them to explore many different scenarios by altering inputs to their model. 

Researchers also need to account for other factors when designing the models; in the above 'model', the changing climate, human movement and land-use change must also be considered. Researchers can either use estimates from experiments or real-world data when building this information into the model. 

If all of these details are considered when building a model, then why are models 'wrong'? 

We don't have computers powerful enough to represent any system with perfect accuracy. The real world is too complex. For example, to perfectly model the spread of malaria in a given region, you would have to predict the movements of millions of mosquitos, the movement of people, and how those mosquitos and people interact. This is clearly an impossible task. 

The art of modelling involves striking a balance between accurately representing the detail and giving a useful bigger picture. We may be able to build a model that could very accurately predict the spread of a given disease in a small region of space, but this has very limited uses in the real world. Instead, if we sacrifice some of the detail, we can make some baseline assumptions about how a system operates that allows us to simplify the problem. It is because of these assumptions that models are wrong, and contribute to the uncertainty associated with model outputs. 

What assumptions are made depends on the purpose of the model. Many disease models include the assumption that no vaccine will be developed, that everyone is equally susceptible to the disease, or that all mosquitos are equally likely to transmit the disease. These aren't true, but they simplify the problem enough that models can still produce useful results. 

For models to be useful, we need to be aware of what assumptions were made in their conception. An over-reliance on model outputs could result in poor decision-making, which could have catastrophic consequences if the model is being used to mitigate disease spread. To prevent this, modellers should be open about their assumptions and communicate them clearly to whoever wants to use the model.

Once we accept that, yes, models are wrong, but the picture they give us about reality is extremely useful, we can then utilise these tools to make powerful predictions about our world. 

For those interested, the University Corporation for Atmospheric Research has a (very) simple climate model available online using which you can see the effects that increasing carbon dioxide emissions have on global temperatures and compare these projections with warming limit targets.