Managers Shouldn’t Fear Algorithm-Based Decisions
Organizations are increasingly favoring algorithms in an effort to make organizational decision-making and judgment more rigorous, be it for setting prices or for selecting candidates to interview for a job. But many professionals remain wary of rule-based decision making, and the interaction between formulas and expert judgment remains a gray area.
When making predictions with data, algorithms tend to be superior to humans. Even for decisions that are traditionally the domain of highly trained experts, decades of studies show that even simple statistical models are often an improvement over human inconsistency.
Many times the professionals take the liberty to override algorithmically generated decision, inspite of its (almost) flawless track record. After all, humans have access to more information, while models are limited to a programmed set of variables. Humans can flexibly update assumptions and understand nuances, while models are rigid in their interpretations of a fixed set of data. Thus, it is but natural for humans to feel the need to review, update or eve change the decisions. However, evidence points in the opposite direction. Because of our access to a seemingly infinite set of variables for consideration in any given problem often works to our disadvantage, as, consciously or not, we are subject to the influence of irrelevant factors that may lead to flawed judgment.
However, humans should be empowered to overrule algorithms in some cases. One clear group of cases is what is referred to as ‘broken leg’ cases. Professionals should ignore the results of a statistical model telling you that someone is going to go to a movie tonight if you learn she just broke her leg. A decent model will do better than you at recognizing how demographic and other variables predict movie attendance; but an instance like this is an opportunity for humans to intervene. Professionals need to be more rigorous and rule-based in their decision-making, which will lead to structured human judgment that is at least somewhat, algorithmically arrived.
There are a few ways to structure human judgment. One is to create a shared reference set that professionals can use for comparison and reference to better ‘program’ human judgement. Another way is to set rules around how much professionals can tinker with algorithms. Managers can allow people to adjust an algorithm’s output, but there should be limits on either the magnitude or frequency of adjustments.
As algorithms get smarter and better, professionals will trust them more, as they will find them to be more discerning and fair. Formulas can, among other improvements, help remediate the hidden scourge of inconsistent human decision so managers should at least warm up to the idea that algorithm-based decision-making maybe a better way to go ahead.