For at least two years, the debate over “Artificial Intelligence: risks and opportunities” has plagued us daily, fuelled also by the sensationalist tone adopted by US Big Tech, which once a month discovers in top-secret tests just how excessively powerful its systems are.
On the one hand, they push for the mass commercialisation of AI systems; on the other, they miss no opportunity to warn governments and institutions worldwide how desperately urgent it is to place limits on AI development. That revolutions are under way is beyond doubt, as is the fact that these revolutions find useful commercial support in the ambiguous and hypocritical pronouncements of whichever Anthropic, Google or OpenAI executive happens to be speaking.
Added to this picture are the camps of the doomsayers and the enthusiasts, but above all the wise, moderate intellectuals who urge us to be neither, in a constant and superficial call for caution.
Rather than jumping on one of the available bandwagons, what I would like to do is take a further step towards understanding why LLMs have nothing to do with the processes of knowledge, nothing to do even with logical validity and, for everyone’s peace of conscience, absolutely nothing to do with thought.
Let us therefore begin at the beginning: what does it mean to know, what does it mean to think, and what is logic? The very premise makes clear how incredibly complex it is to address these subjects. These are questions at the foundations of thought, which have troubled philosophers and intellectuals for over 2,500 years. We will not resolve them here, but will try to ask ourselves a few questions rather than give ourselves a great many answers.
To condense and simplify as far as possible, we might perhaps say that knowledge essentially represents the way human beings are in the world, and the ways in which they experience and inhabit it. And here we already enter our first tautology: the question has implicitly asserted that knowledge relates solely to human beings. Yet it is clear that, until now, we have always understood knowledge in relation to human beings and, more precisely, taking a cross-section spanning over two thousand years, from Aristotle to St Thomas Aquinas, through Locke and Husserl to the present day, we can assert that knowledge resides in two specific human faculties: the senses and the intellect. In the history of philosophy, the balance sometimes tips one way and sometimes the other, with often desperate attempts to reconcile two faculties so apparently distant, though both belong to human beings (read Descartes or Kant).
Nevertheless, we all agree that artificial intelligence systems possess neither of these elements. There is no sensory knowledge in AI, just as there is no “intellectual” knowledge. Besides, purely intellectual knowledge, to paraphrase Kant, is only theoretically (or rather, in a strictly theoretical sense) possible.
We can therefore say, philosophically, that AI is incapable of knowing, since it lacks two characteristics proper to human beings through which they interact with the world. Yet it is capable of connecting already known elements, associating them with one another and generating new elements.
But could the way known elements are associated be the foundation, not of knowledge, but of thought? AI, then, cannot know. But could it think?
And here we should try to understand precisely what we mean when we speak of thought. Thinking does not mean “putting together” known things and processing them in an attempt to arrive at new concepts or definitions. The process of thought is far more intricate. In 1954, a short but illuminating text was published: Was heißt Denken? (What Is Called Thinking?) by Martin Heidegger. In this text, Heidegger seeks to specify in great detail precisely what thought is and what it means to think. For Heidegger, the act of thought is a capacity proper to human beings, and one which, moreover, they do not even already possess a priori. We have reached our second tautological statement: while knowledge may be a faculty that also belongs to animals, thought is characteristically human. The statement itself already rules out the possibility that a machine might be endowed with thought.
For Heidegger, indeed, human beings do not think as a matter of course: rather, thinking is a particular disposition they develop when alone with themselves, or when called to thought by their surroundings. Thought, according to Heidegger’s interpretation but probably also, if implicitly, across the whole Western philosophical tradition, is something that remains “with itself”, something that questions the very activity of thinking.
Let us therefore abandon the idea that “thinking” is something scientific or analytical. To quote Heidegger himself:
“It is indeed true: what we have said so far, and the entire discussion that follows, have nothing to do with science, precisely where our discussion comes closest to thought. The reason for this lies in the fact that science does not think. It does not think because — as a consequence of its procedures and its instruments — it cannot think (…). Science does not think. To the ordinary mind, this is a scandalous assertion. Let us leave the assertion its scandalous character, even though, on the other hand, we immediately add that science, like everything human beings do or refrain from doing, has no recourse other than thought. But the relationship between science and thought is genuine and fruitful only when the abyss separating science and thought becomes visible and is recognised as unbridgeable. There is no bridge leading from science to thought; the only possible passage is a leap”.
Thinking, then, has nothing to do with calculation, nor does it necessarily reach a conclusion. Above all, it serves absolutely no purpose beyond the activity of thinking itself. You will appreciate that this definition is probably as far removed as possible from anything to be found in AI agents, trained always to provide an answer and to make it as useful as possible.
AI, then, does not think. We can, however, say with near-total certainty that it develops logical functions.
But even here, the reality is far more complex than it appears. Without going into overly specific details, it is fairly well known that LLMs solve difficult problems and stumble over trivial ones, making serious errors in simple processes of formal deduction. In fairness, it should be pointed out that commonly used language models are not really trained to arrive at formal deductions; agents developed using symbolic AI are a different case, tending to produce more accurate results. It is no coincidence that, until a few years ago, AI systems were a complete disaster mathematically.
That said, the picture has changed in recent years. Current models are good at solving a large proportion of logical and mathematical problems, including complex ones, even surpassing human beings in many cases. But the real question is: is this genuine intuition or an impressive capacity for calculation? They certainly remain less reliable than a formal system, since correctness is guaranteed not by the method but by a statistical result which, however good it may be, is decidedly not infallible.
This too will sound like blasphemy to many readers, but it can in fact be verified through experiments with the most common AI systems. Their weakness lies in the distance between producing reasoning that seems correct and guaranteeing that every step within that system is valid.
Finally, and I believe this is an absolutely central point, AI systems may know a logical rule, but that does not mean they can apply it correctly in a concrete case. Knowing a rule and applying it “opportunistically” within logical processes are distinct capacities. And at present, while AI may know the rules (probably all the rules), it is not yet endowed with the analytical/mathematical intuition needed to have that creative spark and manage to apply those rules to a specific case, actually drawing a new and unexpected conclusion from them.
Recent developments in AI seem to contradict this thesis; nevertheless (and we return to the earlier point), it is very difficult to establish how much is a matter of intuitive capacity and how much, instead, is due to impressive computational powers.
In conclusion, AI does not know, does not think and is incapable of logical intuitions, at least as these terms have traditionally been understood. So it is useless: what are we worried about?
Let us begin, for now, with the idea that AI gives us access to the greatest computational capacity ever to have existed. We can perform hundreds upon hundreds of complex operations in a span of time hitherto considered unimaginable. Yet for LLM systems, semantic plausibility remains the principal output, substantially accelerating logical formalisation while already containing a structural error within itself.
On this point, probably the most complex, I would not wish to come across as naïve. Semantic plausibility can in fact influence thought: much twentieth-century philosophy (from Wittgenstein to Davidson) has shown how intimately thought and language are connected, and how each can influence the other. Yet this position, much closer to an analytical approach to the processes of knowing, does not consider thought as a meditative process; and in general, though this is only my opinion, I believe linguistic processes go far beyond simple formal elements, creating a “surplus of meaning” that LLMs cannot currently comprehend.
But there is a qualification: with the continuous improvement of AI systems, the refinement of test-time compute systems and of neuro-symbolic systems, the future suggests that AI will probably never think, but our intellectual processes will be decidedly influenced by it, not so much through delegating our thought to AI as through beginning to think “with” AI.
But let us return once more, for a moment, to “what it means to think”. For perhaps the real difference between machine and human being no longer lies in the answer, but in the question. When the problem is given and success is verifiable, our analytical capacities really do tend to be matched, if not surpassed, by those of technology.
Returning to Heidegger and his Was heißt Denken?. The title can mean both “what does it mean to think?” and “what calls us to think?”, since heißen also means “to call”. Heidegger exploits precisely this double meaning.
Perhaps the distinction between human being and machine lies less in the capacity to give answers than in “being called” by the question. A machine can generate questions, but nothing calls it to ask them.
Human beings thus retain dominion over the question, and the capacity to discern whether to give an answer or change the question.
