Both economists and data scientists need to learn some basics !
First, deploy simple models to identify the key factors that influence an assessment. A common response to criticisms of the kind we have described above is an offer to add to the model whatever we think is missing. But this is another reflection of the mistaken belief that such models can describe ‘the world as it really is’. The useful purpose of modelling is to find ‘small world’ problems which illuminate part of the large world of radical uncertainty.
Second, having identified the parameters which are likely to make a significant difference to an assessment, undertake research to obtain evidence on the value of these parameters. For example, what value do rail passengers attach to a faster journey? Quantification can often serve as a reality check
Third, simple models provide a flexibility which makes it much easier to explore the effects of modifications and alternatives
In the end, a model is useful only if the person using it understands that it does not represent ‘the world as it really is’, but is a tool for exploring ways in which a decision might or might not go wrong.
But creating good models requires humility, and the willingness to be proven wrong, so you can improve them !
Notes from the book, Radical Uncertainty by Mervyn King