Using AI, the Ultimate Black Box, Effectively—A Management Decision Making Perspective ()
1. Introduction
There can be little doubt that Artificial intelligence (AI) is a transformative and often disruptive technology, particularly its latest version—generative AI. This begs the question of whether AI is a constructive or destructive force for change? To answer this question from a management decision making perspective, we need to answer three relevant questions: What is AI? How does AI work? How can managers use AI to improve the quality of their decisions? To answer these key questions, we need to understand the role of transparency in science and the relationship between AI and Decision Sciences to have a context for understanding how AI fits into the variety of tools used by managers to make effective decisions.
2. The Importance of Transparency in Science
Merriam-Webster (n.d.) defines a black box as “anything that has mysterious or unknown internal functions or mechanisms.” As we shall discover, AI is definitely a black box from a management decision making perspective. Why is this important? Because of the essential role of transparency in science: “Transparency is a defining characteristic of all scientific endeavours. Without it, the integrity and validity of research findings cannot be independently tested and verified, which is necessary to ensure the reliable use of evidence in decision making” (Sampson et al., 2019: p. 2).
Many consider John D. C. Little to be the father of marketing science. In 1970, Little expressed a similar view of the importance of transparency in his groundbreaking work on decision models for managers. He argued that good models for management decision making are simple, robust, easy to control, adaptive, complete on important issues, and easy to communicate with (Little, 1970). On the subject of transparency, Little (1970) wrote:
Managers don’t understand the models. People tend to reject what they do not understand. The manager carries responsibility for outcomes. We should not be surprised if he prefers a simple analysis that he can grasp, even though it may have a qualitative structure, broad assumptions, and only a little relevant data, to a complex model whose assumptions may be partially hidden or couched in jargon and whose parameters may be the result of obscure statistical manipulations.
When asked to revisit his views four decades later, Little (2024) stated, “Guidelines like simplicity, robustness, completeness, and ease of use, as perceived by the user, should not be surrendered easily” (p. 1858). Let’s see how this relates to management decision making.
3. AI and Decision Sciences
In this section, we examine four categories of management decision making tools: decision support systems, data analytics tools, traditional AI, and generative AI.
3.1. Decision Support Systems
Decision support systems help managers make better decisions in high impact areas. An example is John Little’s Brandaid model for making marketing mix allocation decisions related to product, price, promotion, and place on the shelf, a version of which I worked with at a major consumer products company (see Little, 1970).
3.2. Data Analytics Tools
Standard data analytics tools are used to analyze quantitative data and help users make decisions based on the model’s prediction of what is likely to occur for different possible scenarios. Made famous in the book, Moneyball, by Lewis (2004), such tools are used extensively today in businesses of all types. It is helpful for management decision making purposes to distinguish this category of data analytic tools that rely on standard probability and statistics methods and computer programs and do not involve AI from those in the next two categories of management decision making tools, which do.
3.3. Traditional AI
Marr (2023) provided a useful definition of traditional AI:
Traditional AI… focuses on performing a specific task intelligently. It refers to systems designed to respond to a particular set of inputs. These systems have the capability to learn from data and make decisions or predictions based on that data. Imagine you’re playing computer chess. The computer knows all the rules; it can predict your moves and make its own based on a pre-defined strategy. It’s not inventing new ways to play chess but selecting from strategies it was programmed with. …Other examples of traditional AIs are voice assistants like Siri or Alexa, recommendation engines on Netflix or Amazon, or Google’s search algorithm. These AIs have been trained to follow specific rules, do a particular job, and do it well, but they don’t create anything new.
Another example of the effective application of traditional AI-based data analytic tools involves the diagnosis of a patient’s symptoms to identify the latest research and techniques a physician might choose to be more effective in treating the patient. The focus is on finding and presenting existing data, which AI does well, to enhance the capability of the human being at the center of the process. The AI program is not asked to diagnose the patient for the doctor.
3.4. Generative AI
Marr (2023) also provided a useful definition of generative AI:
Generative AI, on the other hand, can be thought of as the next generation of artificial intelligence. It’s a form of AI that can create something new. Suppose you have a friend who loves telling stories. But instead of a human friend, you have an AI. You give this AI a starting line, say, “Once upon a time, in a galaxy far away…”. The AI takes that line and generates a whole space adventure story, complete with characters, plot twists, and a thrilling conclusion. The AI creates something new from the piece of information you gave it. This is a basic example of Generative AI. It’s like an imaginative friend who can come up with original, creative content. What’s more, today’s generative AI can not only create text outputs, but also images, music and even computer code. Generative AI models are trained on a set of data and learn the underlying patterns to generate new data that mirrors the training set.
Pay particular attention to the last line of this definition, as in the words of Shakespeare, “There’s the rub.” In the next section, which focuses on the pros and cons that managers who choose to use AI must understand to make valid decisions, we will examine the significant problems caused by: 1) the way in which generative AI creates its output and 2) the method used to train generative AI models. In a subsequent section, we will address 3) ethical issues related to traditional and generative AI.
4. Pros and Cons of AI for Management Decision Making
To understand how to use AI effectively, it is critical to know the difference between an Intelligent Assistant (IA) and AI. Because of their intentional transparency, decision support systems and data analytics models of the type described previously are intelligent assistants. They help the user make decisions. They are not black boxes that crunch data and provide solutions without providing the user with any insight into how those solutions were arrived at. An analogy may help clarify the difference between IA and AI. Think of an IA as a robot under the control of an operator. The robot magnifies the power of the operator, but the operator is definitely in charge. Think of AI as the ultimate black box into which goes data that is transformed (as in “then a miracle happens”) into answers by the machine.
4.1. IA vs. AI
To understand how to use AI effectively, it is critical to know the difference between an Intelligent Assistant (IA) and AI. Because of their intentional transparency, decision support systems and data analytics models of the type described previously are intelligent assistants. They help the user make decisions. They are not black boxes that crunch data and provide solutions without providing the user with any insight into how those solutions were arrived at. An analogy may help clarify the difference between IA and AI. Think of an IA as a robot under the control of an operator. The robot magnifies the power of the operator, but the operator is definitely in charge. Think of AI as the ultimate black box into which goes data that is transformed (as in “then a miracle happens”) into answers by the machine.
4.2. Traditional AI
The main difference between traditional AI and generative AI lies in their capabilities and application. Traditional AI systems are primarily used to analyze data and make predictions, while generative AI goes a step further by creating new data similar to its training data (Marr, 2023).
An important feature of traditional AI is that it harnesses the power of Big Data for analysis and decision making. Three serious shortcomings of traditional AI are that while applying powerful computing software to large amounts of quantitative data enables the machine to identify significant relationships between variables that might otherwise not be discovered, the results are often questionable due to these factors: 1) the large number of false positives generated, 2) confusing significance with importance, and 3) mistaking correlation for causality.
To understand these three drawbacks of traditional AI, visualize a target hit by a shotgun blast. The computer identifies a significant positive relationship between variables A and B, meaning that B goes up as A goes up and vice-versa, based on data that a scatter plot shows looks remarkably like a shotgun blast. Without the machine’s input, you would likely see no obvious relationship between A and B, but given the tremendous amount of data involved, the statistics suggest that there may be a significant, but very small relationship between A and B. While this may be true, it is possible in cases like this for a very large proportion of true negatives (i.e., there is no relationship between A and B) to be misclassified as positives (i.e., there is a positive relationship between A and B). Another drawback of traditional AI is that users often confuse a significant finding with an important finding. Often the model underlying the apparent linear relationship between A and B, while significant, explains only a very small amount of the variation in the data, making it essentially useless for predictive purposes. Finally, users often confuse correlation with causation. They note that A and B have a significant statistical relationship and incorrectly assume that changes in A are responsible for changes in B. This simple example from Bobbitt (2021) refutes the notion that correlation invariably means causation. It may, but it does not in all cases:
If we collect data for monthly ice cream sales and monthly shark attacks around the United States each year, we would find that the two variables are highly correlated. Does this mean that consuming ice cream causes shark attacks? Not quite. The more likely explanation is that more people consume ice cream and get in the ocean when it’s warmer outside, which explains why these two variables are so highly correlated. Although ice cream sales and shark attacks are highly correlated, one does not cause the other.
4.3. Generative AI
As noted previously (see Marr, 2023), traditional AI systems, which basically run on binary quantitative data at its most elemental level (i.e., 0 s and 1 s), analyze quantitative data to make predictions. Hence, the term computer, or number cruncher. In contrast, generative AI systems deal with words, which are not numbers! Furthermore, they autonomously (i.e., without human control) create new qualitative data (words) similar to the data (words) used to train them. Before we discuss the very serious problems this creates for the credibility of generative AI output, let’s stop to acknowledge the obvious: these generative AI tools are not human! They have no built-in human intelligence. So, how does generative AI (a software program that runs on numbers) create words, and what does this process say about the validity of the words it creates? The answers to these two extremely important, fundamentally intertwined questions is what we will discover next.
While the actual process of transforming data into words is more complex, because of the need to produce what appear to be real sentences, generative AI algorithms essentially try to make the output look like the work of a real person by analyzing the data used to train it to calculate what word has the highest probability of being the next word in a sentence based on the percentage of times that word appears in the training data. There is no thought involved in this process. The result is, therefore, fake information, which is known as hallucination when it occurs. With regard to ChatGPT, a generative AI tool, Warner (2024) argued:
GPT-generated comments are a simulation. They are fake and not genuinely communicative. I’m sure everyone has heard of the problem of AI “hallucination,” where it invents material that is untrue or inaccurate, but it is important to recognize that from the point of view of AI, everything is a hallucination. It has no capacity for separating the real from the fake, the true from the false. That it may occasionally or even frequently hit on good advice should not matter, because there is no genuine intent at meaning or communication behind the generation of syntax.
So, the Achilles heel of generative AI is having to use probability to determine what word comes next in a sentence, rather than using the rules of grammar, which would be impossible for it to do!
Another major problem related to the application of generative AI output to management decision making relates to the method used to train generative AI models. Basically, the only words a generative AI algorithm has to work with in generating its output are those it was trained on. In other words, whatever computer files are available in the training database shape the solutions produced by a generative AI model. If the data is outdated, graphically obscene in part, or otherwise flawed, the generative AI tool trained on such data will include these flaws. This is why companies that develop generative AI models expend considerable effort on neutralizing the inherent flaws of the training data and regularly adding new data to the training database. According to Heikkilä (2024):
The generative AI boom is built on scale. The more training data, the more powerful the model. But there’s a problem. AI companies have pillaged the internet for training data, and many websites and data set owners have started restricting the ability to scrape their websites. We’ve also seen a backlash against the AI sector’s practice of indiscriminately scraping online data, in the form of users opting out of making their data available for training and lawsuits from artists, writers, and the New York Times, claiming that AI companies have taken their intellectual property without consent or compensation.
5. Ethical Issue with Using AI for Management Decision
Making
In addition to the major problems in output validity inherent in traditional and generative AI solutions, users of traditional and generative AI systems must also concern themselves with the ethical/moral implications of implementing the solutions those systems produce.
5.1. Traditional AI
As noted earlier, traditional AI models generally make recommendations based on correlations between large amounts of very noisy data. Not only does this often result in spurious findings, but those findings can also lead to highly unethical decisions if applied in ways that discriminate against a targeted population based on age, race, gender, income, and other demographic characteristics (O’Neil, 2016). Referring to traditional AI models (generative AI models were not on the scene at that time), O’Neil titled her book in a provocative way to emphasize the magnitude of the Big Data problem unethical use of traditional AI represents: Weapons of Math Destruction.
5.2. Generative AI
There are many ethical issues involved in making management decisions based on generative AI model output. Among these are the effect of cleaning the training data on the cleaners, who are often very underpaid people in underdeveloped countries who must identify and eliminate the graphically obscene internet data and other identified flaws in the training database, the effect of decisions made by users who do not understand the distortion of reality caused by hallucination, and the effect on the planet of the enormous amount of power required to run these generative AI models, which threatens to overwhelm efforts to mitigate climate change by decarbonizing the planet. According to Leffer (2024):
Experts warn that AI’s insatiable appetite could derail efforts to ditch fossil fuels and confront climate change. “Decarbonization is at risk,” says University of Chicago computer scientist Andrew A. Chien. “Progress is at risk.”… Generative AI requires tremendous energy because it churns through extraordinarily complex math. Every line of text from a chatbot requires billions of calculations, says Shaolei Ren, associate professor of electrical and computer engineering at the University of California, Riverside. Writing a single email with ChatGPT can use as much energy as driving a two-ton electric vehicle half a mile.
6. Gauging the Effectiveness of AI-Based Decisions: The AI
Decision Quality Index
What is the effectiveness of a decision based on the use of AI? One way to measure the effectiveness is to incorporate two major aspects of a decision, its validity (i.e., the extent to which it can be trusted) and ethicality (i.e., the extent to which the implementation of the decision is ethical) into an index consisting of a validity score multiplied by an ethicality score:
AI Decision Quality Index = (Validity) × (Ethicality)
For example, measuring validity and ethicality on a scale from 0 to 10 would mean that an invalid decision (based for example on the hallucination of a generative AI model) would receive a validity score of 0, and, regardless of the ethicality of implementing that decision, an AI Decision Quality Index of 0. Similarly, a valid decision (based for example on a statistically meaningful relationship between an independent variable and dependent variable arrived by using traditional AI) would receive a validity score of 1, but an overall AI Decision Quality Index of 0 if implementing that decision would be unethical. If the decision were both 100% valid (score of 10) and 100% ethical (score of 10), it would have the highest possible AI Decision Quality Index of 100. For all other combinations of validity and ethicality, the AI Decision Quality Index would be between 0 and 100.
It is important to note that various management-determined weighting and shaping factors could be included in the AI Decision Quality Index to customize it for its intended decision making purpose. These might include weights for the respective variables or transformations of the respective variables. An example of such a more general formula for the AI Decision Quality Index would be:
AI Decision Quality Index = [(WV)(Validity)nV] × (WE)(Ethicality)nE
where the respective Ws represent weighting factors for each variable (i.e., validity or ethicality) and raising a number to the respective power of n results in a transformation of each variable.
7. Guidelines for Using AI to Make Good Decisions
Based on the AI Decision Quality Index and the information on the pros and cons of AI for management decision making described previously, we can draw certain general conclusions about the effectiveness of data analytics and AI models and how to use them to make good decisions:
For traditional AI tools used by a person as an intelligent assistant (IA) to support their decision making, the AI Decision Quality Index can vary from 0 to 100. However, when not used as an IA, the AI Decision Quality Index will be low due to the high probability of false positives. Guideline: Use traditional AI tools exclusively as an IA to support your decision making.
For generative AI tools, the AI Decision Quality Index will be zero to very low due to hallucination. Guideline: Use generative AI tools exclusively as an IA to support your decision making.
Where AI is concerned, the primary decision making rule for managers at all levels who use AI solutions is clear: Let the user beware!
8. Conclusion
In this paper, we addressed three key questions related to the use of AI for management decision making: What is AI? How does AI work? Specifically, we focused on the importance of transparency in science and decision making versus the black box nature of AI, described the pros and cons of four types of models used for management decision making (decision support systems, data analytics, traditional AI, and generative AI), identified the crucial role of decision makers as users of AI in the interpretation and validation of AI output, discussed ethical issues associated with AI, developed a method for gauging the quality of management decisions based on AI (the AI Decision Quality Index), and suggested guidelines to help managers as users of AI make good decisions, the foremost of which is: Let the user beware!