Don’t Let AI Steal Your Tacit Knowledge

A recent, and extremely controversial, Palantir essay advocates for a deep enmeshment of the state with artificial intelligence for the purpose of national security. Karl et. al make the case that those who are unwilling or unable to integrate artificial intelligence into their nation’s security infrastructure will become overtaken by those who do. Businesses and governments who adopt AI with no clear structure are able to burn up limitless credits without gaining much competitive advantage, while those who don’t adopt AI at all are in  an even more uncompetitive situation. Gaining power in our world now requires the effective use of artificial intelligence. While this is necessary where survival comes into play, we risk drifting towards a world where our best ideas remain forever in Plato’s form land. AI allows far more effective execution, and a subset of ideation, but it must be recognized that most machine learning models share one form of intelligence. They are able to offer recommendations given their specific metacognitive personalities, but they are analogous to a single person with a real personality with access to almost limitless knowledge. Regardless of how much that person knows, they will still prioritize and approach problems in the way that is native to them. Much of entrepreneurship and creativity in the world arises from the intersection of different people’s tacit knowledge. Limited in intelligence and information, each person’s unique collection of pre-informational analytical traits allows them to approach the world in unique ways that allow discoveries that could not arise from millions of AI agents that share one fundamental cognitive structure.

AI can be helpful at providing a subset of all the good ideas with which to approach your situation, but often the things that work best in the world at first seem inadvisable or the information needed to make the right decision is radically unknown. AI is able to prototype effectively in some situations dealing with unknowns, but there are some goals and situations that do not have as many reference points  that can be used to draw conclusions about them, and thus in them, there is more space for cognitive uniqueness. While artificial intelligence can certainly offer ideas that seem unique, it is for the most part more tied into situations already seen before. People making the wrong decision can often lead to good results, and missing out on those wrong decisions could lead to a future governed by risk aversion and unable to fail into discovery. Additionally, the legal risks of AI force it to be risk-averse rather than suggest things that might be right for a person in their specific situation. Essentially, AI can help us create good ideas, but it will systematically err towards a certain subset of those ideas, just by the very nature of having one cognitive structure rather than the numerous ones that are found in actual people.

One of the greatest risks of societal AI use is that it can allow people to outsource the pesky little decisions that only they can make. We lose things like risk preference and seemingly unimportant information that is understood by our minds, but not prominent enough to be actively thought and incorporated into a prompt. Even individual preferences are often forgotten when people just want an answer and they think they have given all of the pertinent data. Many more things are significant to a decision than we think, and sometimes the gut reaction that we have is informed by reason that we cannot access when thinking in verbal terms. We risk losing some of this when we try to create situational models for an artificial intelligence system to use. Of course, as the systems gain access to more data, and even situational data about us, they will become better at accounting for these sort of things, but even in that situation, our observations about our situation cannot be extracted into machine readable form.

While using AI, we must not let intellectual laziness create poor outcomes. We must have a clear understanding of where AI can and cannot help us. We must understand that decision-making must ultimately fall to us and learn how to ask questions to LLM’s that can be properly answered by them. Particularly in a business context, there must be clear guidelines for where AI must play second fiddle to our own thoughts. As imperfect and irrational as we are, the human mind is still uniquely creative and insightful, and the much greater informational capability of artificial intelligence systems can only be fully used when the human brains’ uniqueness is embraced. No system can be smart enough to make up for the costs of poor thinking and question framing.