The power and perils of data-driven decision making

Data is everywhere. Governments use it to design policy, businesses use it to allocate resources, and individuals use it to make choices about everything from health to finance. And that’s welcome – up to a point. The world is a complicated place, full of patterns we cannot detect without a statistical helping hand. But while data can be the basis for good decisions, it can also mislead, distort priorities, and create false confidence. The real challenge is not simply to gather more information, but to use it wisely. Here are six hard-won principles for turning data into decisions. 

The power of good data

The first principle is that data is powerful, and the absence of data is dangerous. Without evidence, people can keep making the same bad decisions for astonishingly long periods of time. History offers many examples, but one of the starkest is the death of George Washington. When he fell gravely ill in 1799 with a throat infection, the three doctors treating him repeatedly drained his blood in the hope of easing his breathing difficulties. This was accepted medical practice and Washington himself had requested it. Of course, it did not work – and Washington, a rich and celebrated man, died. 

What is striking is not that the treatment was harmful, but that the medical profession had gone for a couple of thousand years without ever conducting the kind of controlled experiment that would have confirmed that it worked – or that it did not. For generations, doctors had relied on tradition and authority instead of systematic evidence. 

The second principle is that if good data matters, we should actively try to get it. Too often, decision makers treat the lack of evidence as an unavoidable fact. But useful data can often be found, improved, or produced. That may mean collecting better records or conducting experiments. A company unsure whether remote work improves productivity can test different arrangements rather than rely on the HIPPO (highest paid person’s opinion). In medicine, the clinical trial replaced guesswork with structured comparison. The key habit is intellectual humility. Instead of pretending to know, leaders should ask: what data do we need, and how might we obtain it? Good decision making is not just about making the most of the numbers that come to hand – it is about creating the conditions under which reliable evidence can emerge.

Be careful how you use data

If the first two principles are a celebration of data, the next three are warnings. 

Never let data become a substitute for clear goals or strategy. The Vietnam War is notorious for the US military’s reliance on the ‘body count’ metric, by which the US measured its success by tracking the number of casualties it inflicted. The practical and moral flaws with this approach should be obvious, but less well-known is how the US came to set such store by the body count. The basic problem, writes historian Gregory Daddis, was that “the US Army in Vietnam often stumbled through the conflict without a consensus on its strategy”. Lacking a clear definition of what success might look like, or how it would be achieved, military and civilian leaders leaned heavily on metrics such as sorties flown, villages ‘secured’ and territory controlled. The body count was simply the most famous – and, for military men, the most institutionally familiar. All these figures created an impression of progress, but they could not answer the deeper strategic question of whether the war was moving in the right direction. Data cannot tell us what our goals ought to be, but it can falsely reassure us that somehow we have already decided. 

The fourth principle is to beware of targets. The economist Charles Goodhart is famous for Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure”. Once people have a strong incentive to hit a target, they will find ways to hit it that do not reflect the target setter’s original intent. For example, managers at a hospital might be told to try to reduce the time it takes to discharge patients. As long as patients are being discharged only when it is safe to do so, an effort to reduce discharge times is an effort to improve patients’ recovery times. But if fast discharge becomes a high-stakes target, hospitals may simply discharge patients prematurely, only to see them re-admitted as emergency cases. Targets are not always a mistake but they are usually a risk; assume that people will try to game the system.

Don't let numbers become the target

The fifth principle is to avoid what the philosopher C. Thi Nguyen calls ‘value capture’. Value capture is a deeper and more subtle problem than mere gaming. It occurs when a rich, complex set of values is replaced by a thin, easily measured substitute. Over time, the metric does not merely stand in for the real objective; it starts to replace it. Test scores in a school start by being an indicator of learning – and then dominate the idea of education itself. Or academics, in chasing the number of times they are cited in academic journals, may start to forget that they ever had a loftier goal. Spend too much time hoping for likes on a social media platform and it is easy to lose sight of the fact that originally you were on there to connect with friends or to have fun. The risk is not only that data distorts our behaviour, but that it distorts our desires. 

The sixth and final principle is that data always needs context. Numbers stripped of context are an invitation to bad decisions. Context includes where the data came from, how it was collected, what it excludes, what the baseline is, and what comparisons are relevant. Behind every dataset lies a story and without that story the bare numbers are often misleading. Good judgement requires curiosity: where did this figure come from? What does it really mean? Compared with what? Over what period? For whom?

Data is a tool of immense power, but only when used with wisdom. Data can distract us, lead us astray, and distort our values. But it is also precious: without good data, we not only make mistakes but can make the same mistakes, unchallenged, forever. 

Tim spoke at the Walter Scott Research Conference in May 2026. This article was written in February 2026 and forms part of the 2026 edition of our Research Journal.

The views and opinions expressed in this article are those of the author and do not necessarily reflect the position of Walter Scott.