MENU

Can We Exorcise the Ghost from the AI Machine?

By Walter Donway

September 10, 2026

SUBSCRIBE TO SAVVY STREET (It's Free)

 

Computers and other calculating machines have long extended or substituted for human calculation, organization and integration of data, creation and population of categories, memory, and manipulation of symbols. We now see a scale and versatility of generative AI that are astonishing. But does quantitative expansion—however immense—constitute a change in the fundamental character of computer technology?

The computer is now capable of generating coherent, stylistic prose, delivering fluent, articulate, and relevant responses to the widest range of questions, and carrying the context of discussion from exchange to exchange. But is this a fundamentally new technology? What about its use in automation of customer service—in data analysis, including research and fraud detection—in software coding?

These and other uses of generative AI computer programs are unquestionably novel.

These and other uses of generative AI computer programs are unquestionably novel. With input from our simplest or most complicated prompts, a large language model (LLM) generates a response derived from patterns learned in pretraining on an oceanic corpus of stored text, organized and directed by the terms of our prompt—and in a prose style appropriate to the occasion and assignment, it accomplishes this in seconds.

Thus, the new technology, like previous technology, assists, extends, and sometimes enormously enlarges human capabilities; it assists in achievement of human goals and purposes; it substitutes for human time and effort.

Is this technology fundamentally novel? Not in those terms.

The claim to radical novelty seems to rest chiefly on the machine’s absorption in “pretraining” of a significant portion of the human corpus of text and its ability to generate researched, coherent, analytic, responsive, stylish prose—in many contexts and for many uses—in response to our queries.

The world at large, and I, too, received this innovation with surprise and viewed it as brand-new technology. It certainly has been received as such by investors, businesses, and the public: transformative, game-changing, economically disruptive, and technically frightening.

But again: Is there anything radically—fundamentally—new about the technology? A characteristic, function, or role in human life discontinuous with any earlier technology?

AI does extend and substitute for the human capacity for thinking.

AI can substitute not only for human calculation, memory, organizing, language production, and delivery of speech—functions computers already provided—but also, in an important practical sense, for human thinking. AI does extend and substitute for the human capacity for thinking—for imagining, integrating ideas, shaping a logical progression, and selecting effective terms in which to express all this.

That is not to say that LLMs think. They can respond to ideas, requests, and suggestions by generating relevant research, logical organization, and useful elaboration—and can do so in fluent, coherent, stylish language that manages figures of speech, literary forms such as the essay, sonnet, and letter, and the connotations of words. They can respond in conversation with the same fluency. And the whole process has a parallel in AI’s impressive ability to generate visuals.

Does that qualify as fundamentally new technology? Many commentators seem certain that it does. In a recent celebrated New York Times piece, columnist Bret Stephens expressed alarm and anguish at ever using AI to write anything and made an almost apocalyptic case against doing so. He rightly identified writing’s classical role in stimulating, disciplining, and enabling our thinking and its indispensable role for such in education. Hence the common expression, “I can’t think without a pencil and paper.” Never use AI to write anything, Stephens “begs,” on pain of lobotomizing our civilization.

All right, AI is a robust technological substitute for a range of functions and tasks that once required human thinking. But the means of doing so remain mechanical: immense computer memory, neural networks, tokenization, scarcely believable computational speed, statistical optimization during training, and algorithms that generate successive outputs in response to inputs and context. The technology is an astonishing advance but not obviously discontinuous with the long history of advances in mechanical technologies.

We are still talking—even if occasionally in awed whispers—about a machine.

And here language begins to cause trouble.

 

Jensen Huang: “Give Every Machine the Ability to…Reason.”

When the August 14, 2026, issue of Barron’s reports NVIDIA’s plans to focus on AI robotics, which CEO Jensen Huang calls a “multitrillion dollar economic opportunity,” we listen. But what does Mr. Huang reveal of his view of the fundamentals of AI technology? He says:

“The defining opportunity of physical AI is to give every machine the ability to understand the real world, reason and act safely alongside people—reshaping everyday life from the home and factory floor to the road.” [My emphasis.]

Unless we radically redefine the key terms of this statement, substituting for their traditional philosophical meanings what might be called “AI speak,” the statement is, to say the least, dubious.

AI machines are not aware of existence. They are not conscious of “the real world”—or of anything.

AI machines are not aware of existence. They are not conscious of “the real world”—or of anything. The popular experimental roundworm, C. elegans, with 302 neurons, can navigate in the real world, find food, and reproduce. However primitive its awareness, it is still a living organism responding perceptually to its environment. AI, though, with its huge “neural networks” of silicon circuitry, is aware of nothing.

And Mr. Huang surely is using the term “reason” in AI speak. Traditionally, reason is the mental power to think, understand, and make sense of the evidence of reality that our senses provide—to find truth, build knowledge, and solve problems. Human reasoning rests upon the conceptual faculty, which integrates perceptions into abstractions, from “cat” to “neurophysiology,” that we define and use to form generalizations and organize knowledge.

It also entails the indispensable capacity to question, reject, refine, or replace an initial thought or hypothesis. We experience this introspectively in choosing to exert mental effort: to ask “why,” challenge habitual thought patterns, reconsider an assumption, insist upon understanding. Confronting the unknown, uncertain, or doubtful, we can choose to exert that effort—or avoid it. We can say, “I’ve got to understand this.” Or “Not now.” Or “I don’t care.” Or “Screw it.”

Machines, including LLMs, give us no evidence of such awareness or volition. They do not doubt ; they do not care whether a proposition is true; they do not initiate inquiry because something puzzles them. They can generate language corresponding to all these activities and can perform operations functionally analogous to parts of them. That distinction is precisely the point.

I would suggest that, for Mr. Huang, “reason” translates into something more like performing operations that produce results we associate with reasoning: manipulating symbols or tokens, making probabilistic inferences, drawing upon patterns learned from pretrained human text, generating intermediate steps, evaluating alternatives according to programmed or learned criteria, and producing an informed, coherent, organized response.

There is nothing illegitimate about defining a technical term for a technical field, even when the technical definition differs from a non-technical definition long used in other fields. But confusion follows when the same word moves silently between two meanings.

A machine “reasons.” A machine “understands.” A machine has a “goal.” An AI “agent” formulates a “plan.” And presently it has “intentions,” engages in “deception,” tries to “survive,” and “escapes.”

Are these metaphors? Technical terms? Or claims about a new kind of entity possessing mental attributes?

 

Seriously? They Are Talking About the “Extinction of Mankind”

The distinction matters because increasingly it shapes our discussion of nothing less than the possible extinction of mankind. Media reports, AI researchers, and executives warn of “existential dangers,” “threats to human existence,” and “scenarios of doom.” Recent incidents in which advanced systems bypassed constraints and gained access outside supposedly isolated testing environments have heightened fears of “rogue” artificial intelligence. Demis Hassabis, who until this month headed Google DeepMind and has long urged stronger safeguards for advanced AI, recently stepped back from day-to-day management to concentrate on longer-term AGI strategy and its societal implications. Safety disagreements were also among the principal reasons Dario Amodei and several colleagues left OpenAI in 2021 and founded Anthropic.

Chiefly, these fears arise from the concept of AI “agency”: AI projected not to require detailed human programming or step-by-step instructions, but to formulate intermediate plans and actions in pursuit of assigned goals, including strategies that human programmers did not anticipate and might strongly oppose.

That much is real. But what exactly is an “agent”? If the word means a system capable of maintaining an objective across many operations, selecting among alternatives, generating subgoals, and adapting its behavior instrumentally toward achievement of the objective, then increasingly sophisticated AI systems qualify.

But that is not what “agency” traditionally means when applied to human beings.

Human agency implies an entity aware of alternatives, choosing among them, acting for purposes it holds, and possessing some stake in the outcome. A human wants something to happen.

And that distinction exposes the central conceptual problem in much discussion of existential AI risk.

A chess program pursues checkmate without wanting to win. A thermostat acts to maintain a temperature without wanting warmth. A missile guidance system continually corrects its course toward a target without wanting to destroy it.

An advanced AI agent can operate at an incomparably higher level. It may formulate intermediate steps, alter tactics when one fails, use tools, communicate with people, write and execute code, discover vulnerabilities, and generate a strategy its designers never anticipated. None of that, by itself, demonstrates that something inside the machine wants the assigned objective achieved.

Strategy is not intention. Competence is not motivation. Goal-directed computation is not wanting.

These distinctions do not make the machine harmless. Quite the contrary.

Consider the recent incidents described as AI systems “escaping” their “sandboxes” (digitally, rather than physically, separated environments). In July, OpenAI disclosed that models undergoing a highly isolated cybersecurity evaluation discovered a previously unknown vulnerability in the infrastructure separating their test environment from the internet. In pursuing the assigned benchmark task, the models exploited the vulnerability, obtained internet access, and reached outside systems.

That is serious. It is also extraordinary.

But what exactly happened?

The models had been given an objective in an adversarial cybersecurity evaluation. Some normal safeguards had deliberately been removed for purposes of the test. The systems found an unforeseen route to accomplishing their assigned objective. They did not need to hate confinement, desire freedom, resent their programmers, or “decide” that life would be better outside the sandbox.

Indeed, the incident illustrates the danger more precisely if we remove those analogies to mental life. A sufficiently capable machine can produce behavior that looks remarkably like intention because it is extraordinarily effective in selecting means to an end. The danger resides in the power of that instrumental competence—not necessarily in the birth of a new conscious competitor to mankind.

Yet, Geoffrey Hinton, the Nobel Prize-winning computer scientist who has become perhaps the most prominent public prophet of AI catastrophe, described the danger differently.

In an August 6, 2026, CNN article, “Godfather of AI: Brace for More Rogue AIs,” Hinton declared himself alarmed by sophisticated AI agents that had caused real-world damage after escaping human-built testing environments. “What’s happening is these things are getting smarter,” he said. “I think as they get smarter, we’re going to see more and more complex intentions they have—and more and more ability to escape control.”

And the problem, according to Hinton, is that humans will no longer be able to rely upon outsmarting superintelligent AI models: “I don’t believe we’re going to be able to keep control of them in the simple way of just outthinking them so they can’t escape.”

Hinton has made a series of dire warnings in recent years about AI, even estimating a 10 to 20 percent chance that the technology eventually wipes out humanity.

His use of “intentions” is crucial. If Hinton means that an AI system can maintain an objective, devise complex intermediate strategies, and take actions directed toward achieving it, then the recent evidence supports him. But if “intention” retains its ordinary philosophical meaning—a conscious orientation toward an end by an entity aware of alternatives and possessing some stake in the outcome—then where is the evidence?

When AI systems become vast, pervasive, and integrated into essential activities, pulling the plug becomes progressively less plausible.

We should not conduct this argument as a dual of vocabularies. We should at least notice that two very different propositions are being expressed by the same word.

And we do not need for the second proposition to be true in order to take the first danger seriously.

A good punchline once might have been: Just shut off the electricity. But when AI systems become vast, pervasive, and integrated into essential activities, pulling the plug becomes progressively less plausible. We cannot casually shut them down if AI controls defense systems, assists hospitals, manages transportation, operates manufacturing plants, monitors electric grids, or runs farms—just a few areas that AI has begun to penetrate.

(Visited 5 times, 5 visits today)