The purpose of data analysis is to reduce uncertainty. By reducing uncertainty, human beings can make wise decisions. In the business world (and beyond), informed decisions are based on the analysis of appropriate activities that are measurable. The purpose of these activities is to try to increase revenues, maximise profitability and reduce risks
All of us are averse to uncertainty
When we have to decide on a particular action, if we have the option of making it with greater or lesser uncertainty, we choose to make it with less uncertainty.
Of course, in business as in life, almost all major decisions must still be made under conditions of partial uncertainty.
This is normal; it is the nature of life and leadership.
Although we would all like to be certain, by accepting partial ignorance, we acknowledge to ourselves how much we do not know, this is a form of awareness that leads to better decision-making.
In fact, uncertainty is essential to avoid excessive risks.
Overestimating uncertainty can lead to delays, in some cases to paralysis, while underestimating uncertainty… well… it is far more dangerous.
uncertainty must be reduced as much as possible
As rational creatures, we must always ask ourselves: is the current reduction of uncertainty sufficient to proceed with a decision right now? Or should we wait and collect more data?
The methods for quantifying uncertainty are well-defined in information theory.
Such methods exist to be applied rigorously to inference problems through the field known as Bayesian analysis.
These methods of data analysis were largely developed by physicists, based on Claude Shannon’s fundamental definition of information.
The basic ideas can be traced back to the pioneering work of E.T. Jaynes, Phil Gregory, David Mackay and Devinderjit Sivia.
The work of these pioneers was the foundation for the development of machine learning (Machine Learning) and artificial intelligence (AI).
More sophisticated companies, such as Amazon and Google, are doing machine learning as a practical realisation of these methods.
Their level of information is one of the reasons for their spectacular success over their less well-informed competitors.