Lies, Damn Lies and Statistics

As scientists, IPNI encourages farmers and advisors to seek out evidence when assessing options such as new fertilizer products or formulations. In a presentation, a graph or table may be shown with a comment like “….it is clear from this that….” and proceed to describe how the data supports the argument or point to be made.

But the truth is not always that clear. We are all very good at fitting patterns to the things we see to fit them into the scheme of things we understand – or think we understand. In fact the human mind is very adept at pattern recognition – just think how subtle are the differences in the faces we pass in the street – but how clearly we can recognize a friend when they appear.

Pattern recognition has its shortcomings though – such as when you mistakenly greet an old friend who in-fact is a stranger who you thought you recognized.

This same problem can be a trap when evaluating evidence. We often try to fit the data into patterns we recognize even though we try to be objective. Statistics can help by fitting trend lines or regressions between an independent variable such as fertilizer applied and a dependent variable such as grain yield. It is easy to jump to a conclusion that what was changed caused the effect measured.

It takes good experimentation to separate associative effects from caused effects – such as making sure there are appropriate controls and that we are sure that we have confidence in knowing the factor that was altered in the experiment. But more than that – we look to experiments to test our ideas and expand our thinking – not just to justify what we already believe.

A line on a graph or a 95% probability function is not proof even if from a well designed experiment. There needs to be a reasoned argument for the response or lack of it.

That is where science diverges from belief – scientists will change their opinion based on evidence. As John Maynard Keynes – one of the foremost economists in the last century said when challenged as to why his opinion had altered. His response was “When the facts change, I change my mind. What do you do, sir?”.

While we need to be skeptical of junk science, we also need to keep our minds open so we can capture the new, novel and innovative. Our task of feeding world demands we do so.