Most aren’t even useful

I love the quotation “All models are wrong. Some are useful”. I like to whisper it to myself before I fall asleep. Sometimes I say it enouragingly to the R console in the hopes it will take the inspiration to make a useful one and magically produce the next QRISK2 model. It’s short, pithy, and it’s true.

But I think most people use it freely as a bit of a catch-all for “we have carte blanche to be wildly incorrect”, as a bit of a rejoinder to a comment on where a model has fallen short. And I don’t mean that in a critical way, despite how it sounds. I think it’s very easy to focus on the fact that George Box has said all our models are already wrong. And that some are useful. It’s encouraging. It let’s us know we’re off the hook for being completely correct (right?) but that, if we are careful enough, we might make one that’s useful.

I also enjoy the clinical literature surrounding how messages are delivered to patients about disease risks. These often show that (and I’m paraphrasing here) if you tell someone that they “have an 80% chance of losing a game”, it will be taken as more negative than if you phrase it as there being “a 20% chance of winning”, despite describing the exact same thing. Even swapping between phrases like 20% and 1 in 5 makes a difference. Apparently showing individual risk where you put a plot of 100 little transparent people and only highlight, say, 10% of them as red to show a 10% risk of disease, is more effective than just telling patients the numbers.

All this is to say that I love the phrase “All models are wrong. Some are useful”, but it’s the optimistic version you tell yourself and others. The inverse, “All models are wrong. Most aren’t useful” wouldn’t get as much traction, but it bears repeating so we can appreciate the same clinical effect as that which has been observed when communicating disease risk. Most of the models we make are neither correct nor useful. I think this is more sobering and, whether it’s accurate or not, probably gives me a little bit more pause before I fit yet another model just because I can, rather than because it’s sensible.

So remind yourself: most aren’t even useful. Just don’t tell your R console. That would be mean.