One of the fundamental properties of complex systems is feedback: what happens within the system alters the conditions that will produce what happens next. What we have, then, is not a simple linear chain of causes and effects, but a circuit in which actions, reactions and consequences feed into one another.
This idea is particularly useful for understanding what is happening with generative artificial intelligence.
A Large Language Model produces linguistically plausible responses based on patterns learnt from data. It can do so extremely effectively, often surprisingly so, but this does not mean it has a general criterion for distinguishing what is true from what is merely well formulated. The fluency of the language, however, makes this distinction much less apparent to the user, because a coherent, confident and formally impeccable response is easily interpreted as the product of equally sound knowledge.
This is where what I call epistemia emerges: the confusion between plausibility and knowledge, between the ability to produce a convincing response and the ability to guarantee that it is well founded.
The interesting point, however, concerns more than an individual error or hallucination. If that were the problem, refining the systems or checking their responses more thoroughly would suffice. The issue becomes deeper when millions of people begin to integrate these tools into their own cognitive processes, delegating ever greater parts of their research, writing, study, decision-making and verification to them.
At that point, feedback comes into play. The easier it becomes to produce content, the more content becomes available. The more content becomes available, the higher the cost of verifying it. At the same time, if these same tools are also used to check what other tools have produced, the task of verification risks being delegated to the very infrastructure that made verification necessary.
The paradox is clear. Generation becomes ever cheaper while verification remains costly, because it requires expertise, time, experience and knowledge of the field.
Yet the very availability of systems capable of providing immediate answers can reduce the incentive to develop that expertise. If we gradually stop exercising our capacity to verify because a machine seems to be doing it for us, we risk increasing our dependence at the very moment when our ability to scrutinise what we depend on is diminishing.
This is a problem typical of complex systems: an intervention alters the behaviour of the system’s elements, and the new behaviour in turn alters the effects of the intervention.
Generative AI, then, is not simply a tool entering an unchanged society. It changes the way we seek information, write, learn and make decisions. These changes alter the information environment, the value of expertise and even what we consider sufficient grounds for claiming to know something.
Epistemia therefore does not lie in the fact that machines make mistakes. Human beings have always made mistakes. It lies in the risk that the distinction between knowing and appearing to know will gradually lose its importance, because plausibility becomes cheap, abundant and immediately available.
In a world where producing answers costs almost nothing, value shifts towards the ability to establish which answers deserve our trust. The scarce resource is no longer necessarily information, but the expertise needed to assess it.
And this is where contemporary complexity reveals one of its most interesting features. We have built machines that are extraordinarily efficient at reducing the cost of cognitive production, but their success could make even more valuable what they cannot automatically guarantee: verification, judgement and the ability to distinguish a plausible sentence from well-founded knowledge.
The problem, then, is not only what machines are capable of producing. It is what we become when we begin to regard that production as equivalent to knowledge.