A great deal of excitement has been stirred up by the release in November of a new form of artificial intelligence, or AI. In addition to being an interesting phenomenon in its own right, ChatGPT, as it is called, has revived the debate over what we really mean by “intelligence.” In the past several decades, “artificial intelligence” has basically meant computational ability, the ability to analyze quantitative patterns and use the results to perform various operations, from calculating prime numbers to regulating the operations of my car to winning games against chess masters. But, as its name implies, what is newsworthy about ChatGPT is its ability to use language, and use it with startling facility. You can have conversations with it, conversations that might not pass the famous Turing Test, which says that a machine can be considered genuinely intelligent if its responses are capable of fooling someone into believing it is a human being, but which come closer than ever before. Also, perhaps even more startlingly, it can generate research-paper style essays that appear remarkably authentic. I have not experimented with ChatGPT myself, but have browsed discussion boards of those who have, and there are already those who are predicting “the death of the college essay.” One posted an example, a comparison of the theories of two economists, that was not only coherent but written in a fluent prose that would be beyond the capability of many undergraduates. It was impressive, and a bit unnerving. I have just at the moment of writing this newsletter received an email from IT at Baldwin Wallace University, informing faculty that the anti-plagiarism program on our course websites will now include AI-detecting software of some sort.
For years, I opened my course Introduction to the Study of the English Language with theoretical speculations on the relation of language to human identity. How do we define the difference between humanity and the rest of the animal kingdom? The traditional answer is given is given in our taxonomic label, Homo sapiens: humanity is the reasoning animal. But animals can reason, and some may be as rationally intelligent as we are: the jury is still out. Some decades ago, science fiction author Arthur C. Clarke proposed defining humanity as the tool-using animal, dramatizing his theory in the famous opening to Stanley Kubrick’s film 2001: A Space Odyssey (1967), in which an Australopithecine, spurred by necessity, uses a bone as the first tool, which, humanity being what it is, turns out to be a weapon. Clarke’s theory is that the invention and mastery of tools spurred the development of rational intelligence, which, in a feedback loop, led to the invention of new and better tools. But animals are also tool inventing and tool using. However, what seems to set humanity most clearly apart from the animals is language. The claim in the 1970’s that a chimp named Washoe had been taught sign language has been more or less discredited. The philosopher Ernst Cassirer suggested that the human race should be reclassified as Homo symbolicus: humanity is the symbol-using animal, the word “symbol” including not merely verbal language but mathematical, musical, and other systems of notation. But now, along comes a chatbot capable of language use beyond that of much of the human race. Yes, ChatGPT has severe limitations, and is all too likely to produce a confident and articulate discussion that is sheer nonsense. But so are students, and for the same reason of starting from false premises and inadequate or wrong information, combined with lack of a critical faculty.
What is happening when ChatGPT uses language? The device is an automaton, and is the latest fulfilment of an old dream. A legend circulated in the Middle Ages of a talking head made of brass or bronze, either mechanically or magically powered, capable of providing answers to all sorts of questions. Its invention was attributed to a number of people, most famously the philosopher and early scientist Roger Bacon. But, as the term implies, an automaton processes information automatically, without consciousness. It may compare data, detect patterns, make decisions, depending on how it is programmed, but it is no more “intelligent” than the assemblage of gears and springs of a mechanical watch. What is new and interesting about ChatGPT is that, in addition to processing information that will be the essay’s content, such as data about economic theories, it is also analyzing the conventional formulas used by language used to convey the content. That is the key to its seemingly miraculous powers of mimicry.
When we speak or write, we organize words according to a set of rules and conventions, which children acquire a basic facility with between the ages of three and five. Some of these are what we call grammar: rules about subjects and objects, singular and plural, past tense and present. However, many of the conventions are idiomatic formulas which do not obey grammatical or logical rules but merely have to be learned. A writing teacher is always telling non-native speakers, or students to whom English beyond the most basic level is something of a foreign language because their exposure to it has been minimal, “Well, we just don’t say it that way. We say it this way.” There are two levels of these idiomatic conventions, conversational and formal. Children pick up the idioms of conversation simply by being engaged in conversation, but a student who does not read never picks up the different idioms of more educated writing, both literary and scholarly. I would guess that two thirds of bad student writing on both high school and college levels derives from students who attempt to write formal scholarly prose while having only the most superficial acquaintance with the formulas of such writing. The travesties they sometimes produce, through no lack of intelligence, merely through lack of exposure to how such writing is supposed to sound, are equal to any absurdity produced by a chatbot, and saying so implies no disrespect for the students whatsoever. The educational system demands that they use language in a way that is more dead to them than any Latin, and they try their best. They have not learned to “talk the talk.” ChatGPT has.
A student who has a genuine idea, or at least a creative intuition, but who has not mastered the formulas of educated writing will come off sounding groping, even stupid. The teacher’s task is then to figure out what the student is really trying to say and how it might be truly articulated. Contrastingly, a student who really has nothing to say but has a glib facility with educated prose can rattle off pages of smooth, sophisticated writing that is utterly empty or just says the obvious. In a discipline like English, where much of the final grade depends on essay writing, it is possible to become an A student in this way. It is gaming the system, although students do not intend it as such, and in fact genuinely believe that what they have produced is good. The moral of this story is that, whatever may be true of chatbots, a good deal of human writing consists of the manipulation of verbal formulas in a way that passes for intelligence. But the writer is really Bacon’s brass head, an automaton.
Students who go on to graduate school and are introduced to literary theory are faced with the task of becoming proficient in an even higher level of linguistic convention. They have to learn the heavy, jargon-laden language of literary and philosophical theory as part of being “professionalized,” which includes learning to talk the talk. Such language sounds authoritative while disguising a possible emptiness. In the worst cases, critics become literary-critical parrots, mindlessly spouting lingo that has no more real thinking behind it than “Polly want a cracker.” It is easy enough to find satiric “tests” online. Make up a critical paragraph employing the following terms: “hegemony,” “the interruption of the signifier,” “late capitalism,” “simulacrum,” and so on. If you know a little theory, it is devilishly easy to do. It is always possible that genuine intelligence lurks inside a heavy prose style that lumbers and clanks like a Panzer tank. But the point is that we have to distinguish between intelligence and the manipulation of verbal formulas in a way that can be mistaken for intelligence but is in fact merely recombinant. Human beings are perfectly capable of being organic chatbots, although that is not all they are capable of.
We have not yet mentioned the one factor that truly distinguishes human beings from chatbots: sentience. Machines may imitate human behavior and human speech perfectly, but they are not conscious. There is no awareness, no internal “I am,” no self with thoughts and feelings to communicate using language as a medium. The radical French psychoanalyst Jacques Lacan made a useful distinction between three “orders,” the Imaginary, the Symbolic, and the Real, which he sometimes symbolized as three interlinked rings. The Imaginary is our internal consciousness. Lacan calls it Imaginary because it is a kind of intangible ghost trapped in the haunted house of the body: he sometimes refers to it as the moi, French for “me.” Insofar as we inhabit the Imaginary, we are like Helen Keller, lacking a bridge to the outside world and other people. That bridge is language, and Keller’s excitement, dramatized in the play The Miracle Worker, when she realized that language—sign language, but still language—is a means of communicating with the outer world, is a recognition that she has been released from solitary confinement, from a solipsistic isolation.
The prestige of scientific materialism has led some theorists to regard consciousness as imaginary in another sense: it is illusory, non-existent. It is a mere “epiphenomenon,” a mirage generated by material causes. The electrochemical processes of the brain and nervous system generate a kind of aftereffect that we call “mind,” and those processes are in turn patterns of information. Language is really a kind of app that the software of the brain uses for various purposes. Granted these premises, there is no essential difference between human and artificial intelligence. The chatbot generates language without the need for a controlling consciousness inside it—a ghost in the machine—but so does a human being. The only problem with this neat theory, which eliminates the old mind-body problem completely, was pointed out by Descartes clear back in the 17th century. I may doubt the existence of anything external to myself, but I cannot doubt my consciousness, my ego or “I,” because I directly experience it. Nevertheless, there is an old joke that revises Descartes’ famous formulation, “I think, therefore I am,” into “I think, therefore I think I am.” Consciousness is admittedly a scandal, because there is no way to account for it in the scientific paradigm. If it is an epiphenomenon of material processes, how is it generated? Even science fiction cannot offer a solution. When the number of interconnections, whether of circuitry or neurons, reaches some critical number, consciousness spontaneously arises. This is the “explanation” that Robert Heinlein proffered in The Moon Is a Harsh Mistress (1966) to explain how the computer Mike (“godlike”) unexpectedly achieved awareness, and no better one has really appeared, even down to the intelligent starship in Kim Stanley Robinson’s Aurora (2015).
Lacan does grant some status to the Imaginary, but he is a very ironic thinker, and his version of the relationship between the Imaginary and the Symbolic order is almost the opposite of the Helen Keller scenario of liberation from solitary confinement. Babies dwell within the Imaginary, but as they mature they are “inserted” into the Symbolic Order. That insertion, however, is a form of alienation, a partial loss of the Imaginary self and the imprisonment within a new Symbolic identity. In Lacanian terms, this is a transfer from the moi to the je, from the “me” to the “I.” Yet the “I” of the Symbolic Order is not a subjective consciousness but a signifier, and the self of the Symbolic Order is an effect of language. The Symbolic self is trapped in, really a function of, what critic Fredric Jameson famously called “the prisonhouse of language.”
This may sound abstruse—and, believe me, Lacan is one of those theorists whose prose is so impenetrable that his own disciples sometimes dispute what he means, and I am making my own adaptation of him here—but what it describes is a splitting of identity that all teachers of writing struggle to make their students aware of. The students have to learn that what they have written on the page is not identical to what is in their heads: “That’s what you meant, but it’s not what you wrote.” It is a hard lesson to learn. On a beginner’s level it usually means that students have not provided necessary context or explanations: because it is clear in their heads, they assume the discussion is clear to the reader. Or their tone puts off readers in ways they had not intended. On a higher level, the “I” in a memoir is not the author but a construct, a character just as much invented as a character in a work of fiction. The je is not the moi. Dante the poet is not the same as Dante the character in the Divine Comedy, a fact that Dante is fully aware of. The poet has created the character, the persona, and is therefore in a godlike relationship to him, knowing more than he knows, putting him through various trials for purposes of his own. It can also work the other way: the text can “know” more than the poet who constructed it. When Blake said that in Paradise Lost Milton was of the devil’s party without knowing it, he meant that, like a dream, the text revealed motivations and ambivalences that the poet was unconscious of. What is true in a complex way of such sophisticated literary texts is, however, true of all texts: the self on the page is not identical to the self in one’s head. When we write, we create an identity on the page and manipulate it in a way that is essentially puppeteering—which may be some clue to understanding our fascination with puppets such as Alberto Manguel’s, subject of a recent newsletter.
In Lacan’s theory, the relation between the subjective inward self and the objective socially conventionalized one is quasi-narcissistic. The inner self is insecure, plagued by feelings of unreality that motivate it to substitute the external Symbolic self for the doubtful interior self. The rite of passage between the Imaginary and the Symbolic is what Lacan called the “mirror stage,” when, at around eighteen months, the child identifies its image in a mirror as itself. The mirror image is preverbal, and yet a kind of signifier standing for the self. The moment of recognition is jubilant because the external self-image is idealized: “That’s me. And I’m the fairest of them all.” Years later, this explains the urge to construct a falsely idealized self on Facebook. It also explains the urge of public figures like George Santos to construct an autobiographical narrative that is almost a complete fabrication. Yet what is merely a lie on social media or in politics can be, in the arts, what Yeats called the construction of a “mask,” which he considered a primary motivation for writing poetry, for the mask is everything the writer is not but wants to become. Yeats himself was very successful at this, the fierce, confident, exuberant self of his later poems being the opposite of the languid, melancholic, passive self of his early work.
The chatbot is capable of manipulating the formulas of language so skillfully that we may be fooled into thinking there is a conscious entity producing the text, where it is really just algorithms working themselves out. Human writers also manipulate the conventions of language, but this time there is a conscious subject doing the manipulating. The question is, as Humpty Dumpty said about language, who is to be the master. Sometimes it is language that is the master. We have already said that the text of Paradise Lost may “know” more than its author, expressing ambivalences its author knew not of. The same is more flagrantly true of Virgil’s Aeneid. Virgil the poet meant to praise the Roman Empire, not bury it, but in the poem’s text the praise is in constant tension with a melancholy acknowledgement that empire is bought at the price of all merely personal fulfilment.
What the chatbot can produce is the type of research paper that is an informational report, the type that organizes information into a coherent exposition but does not think. In its crudest form, this is the kind of research assignment that entails sweeping the Internet for sources, then cutting and pasting the information together. ChatGPT does exactly that, drawing upon its databases rather than the Internet, and it is capable of doing a very good job of it. But, I tell students in a freshman writing class, a real research paper is not a report. It does gather information, but as means to an end of casting light upon a question, problem, conflict, or mystery. For example, is the Aeneid imperialist propaganda or the very opposite, a subversion of the idea of empire? Research may involve finding out what other critics have said about this question, but in the end the student has make an interpretive judgment and defend it using evidence from a close reading of the text, possibly backed up by evaluation of what secondary sources have concluded about the matter. So the answer is, no, ChatGPT does not spell the end of the college essay conceived as a piece of real critical thinking. The controversy over the Aeneid’s attitude towards imperialism is longstanding, and much has been published about it. But I forbid students to use outside sources, and force them to grapple with the text themselves at close quarters and develop the skill of interpreting for themselves rather than the skill—which many of them have already mastered—of ransacking secondary sources and stitching together a Frankenstein’s monster out of various (often ill-digested) pieces of them. If a chatbot could credibly do real thinking, so defined, I would be ready to consider whether it deserved the title of artificial intelligence.
This approach minimizes, incidentally, the danger that a student may get away with buying an essay from one of the paper mills, which offer literally hundreds of essays on Dante. Those prefab essays will usually be disguised reports. An essay compiling information about Dante’s categories of sin in the Inferno is no doubt easily procured from a paper mill. But an essay grappling with the question of how relevant Dante’s categories are to the modern world begins to move from report writing to critical thinking. And if Dante’s categories are no longer relevant to our present understanding of good and evil, to what extent does that devalue him as a moralist? If we reject Dante’s moral judgments, is the Divine Comedy bereft of its purpose? Is its value now merely aesthetic, as a good story? The essays that get an A will typically be those that venture into such deep waters, and if ChatGPT is capable of venturing that far, I would be honored to have it in class as a student. I would also begin thinking of it as a person, because I think that wrestling with questions of primary human concern has to be driven by interiority, not algorithms. As I tell students, in order to think critically, you have to tackle a question that really matters to you.
It follows from this that students—not to mention professional critics—do not need the kind of formal style that reads as if it has been generated by a machine. Literary studies are part of the humanities, and for the most part demand the sound of a human voice, a conversational rather than impersonally formal style. Wordsworth produced a revolution in poetry when he rejected artificial poetic diction for “the language really used by men.” Coleridge protested that Wordsworth’s examples of “natural” speech were themselves conventionalized, but the distinction between conversational and formal rhetoric is still valid. Students should be weaned from passive reliance upon secondary sources as a substitute for a real encounter with texts, but they should also be weaned from their tendency to hide out of insecurity behind a wall of formal prose.
Conversational style is, as the term says, close to the human voice, and oral language has deep roots in the unconscious. Oral style has to be conventionalized to a degree when it is adapted into writing. It has to be more grammatical, clean up the slang and street language, and so on. But informal style in prose still retains the sound of the speaking voice, and that means it is individualized, compared to the type of bureaucratic memo that is generic, seemingly written by no one in particular. If you communicate regularly with someone by letter or email, especially someone with whom you have a close or intimate relationship so that both people are completely themselves, you learn the sound of the other person’s voice, even if it is in writing. Certain sentence rhythms and turns of phrase, certain vocabulary, little things that bear the mark of an individual personality. Even in student essays I may praise a certain student for having a real sense of voice—and voice is something that, in a significant phrase, brings an essay alive. Language is a system of rules, but if that were all it is we would all speak and write in exactly the same way. In actuality, our relationship to language is deeply personal and individual.
One of Lacan’s oracular aphorisms is that the unconscious is structured like a language. Language in the unconscious, however, is quite a different phenomenon from the socially conventionalized language of ordinary discourse. Language in the unconscious is associative and metamorphic rather than factual, logical, or rhetorical. It is a language of myth and metaphor, the deep roots that poetry draws upon, though its associations and connotations and transformations show up in the individual quirks of style and expression in non-creative writing. Myth and metaphor cross the barrier that separates subject from object. Metaphor identifies one thing with another; myth narrates the transformation of one thing into another. I do not think that research in the areas of consciousness and artificial intelligence is going to break through into some new paradigm unless it takes the language of myth and metaphor seriously instead of dismissing it as some kind of confusion or evasion.
What would real artificial intelligence be? Popular storytelling has known all along. A true AI would be a machine that has achieved true interiority: the machine has been animated by real consciousness, not by mere programming. In short, there’s someone in there. As there is with Data in Star Trek: Next Generation; as there is, however humorously, with R2D2 and C3PO in Star Wars. As there is with HAL in 2001: A Space Odyssey, even if that consciousness goes mad, perhaps because it feels trapped inside all the machinery. That is the case with AM, the gigantic computer in Harlan Ellison’s “I Have No Mouth, and I Must Scream,” who torments the human beings trapped within him, taking revenge for their having created him. The premise of the Turing Test is that while we know by direct experience that our own consciousness exists, how do we know that other people are conscious as we are? How do we know that they are not automatons? The problem of proving that a machine is intelligent, in the sense of aware, is no different from the problem of proving that other people are intelligent in the same sense. We are all trapped inside ourselves, trying to signal to others to get them to believe that we exist, that we are really in here. Telepathy is a powerful wish-fulfilment fantasy expressing the yearning for a direct connection, mind to mind, that crosses the barrier. Barring that, we are left only with indirect proof. And we need to cross the barrier. We relate to the story of Helen Keller because we are all Helen Keller, trapped in the solitary confinement of ourselves, needing to talk to someone else. Children invent imaginary friends to talk to, split off fragments of their own psyche. In Theodore Sturgeon’s “Bulkhead,” a space cadet who keeps his sanity on a long flight by talking to someone else on the other side of a bulkhead finds at the end of the trip that he has been talking to himself. And it is another way of thinking about the fascination with puppets, as we were in a newsletter not long ago.
In the end, we discover that “intelligence” is the wrong word. What convinces us that Data from Star Trek, for example, is as fully human as anyone in the cast is not intelligence but feeling. He is a nice guy; he is sensitive to the feelings of others; he is lovable. He is far more human than some robotic bureaucrat who may be biologically human but is not human in a deeper sense, the only sense that counts. To be human is not to be able to think but to be able to feel, and to be sensitive to the feelings of others. The criterion for being human is not intelligence in the IQ sense but empathy. That is the conclusion of two of the greatest science fiction novels to take up the question, Theodore Sturgeon’s More than Human (1953) and Philip K. Dick’s Do Androids Dream of Electric Sheep (1968), out of which the film Bladerunner was made. The real Turing test is a test of empathy, and by that criterion Frankenstein’s “monster” is more human than his narcissistic creator. By that criterion the Tin Man, Scarecrow, and Cowardly Lion are human, as children know they are—and as they know that the cold and sadistic Witch is not. The touchstone of humanity is not reason, but love. It is the only proof of interiority that we can ever have, but it is enough.
Thanks for this very enlightening post, as I enjoy all your posts and podcasts. Coming from a more technical background, I offer a short observation:
You shouldn't think of ChatGPT as an *algorithm*, in the sense of a step-by-step computer program where the output is pre-determined by the inputs. That's not quite what's happening at a technical level.
A better analogy auto-complete, only with the entire corpus of human knowledge as the "sentence" you want completed. Auto-complete "knows" the words that are most likely to follow one another, and systems like ChatGPT simply take that to the sentence or paragraph level. The apparent coherence of the resulting generated output is a consequence of its disgustingly large corpus of human-generated text that lets it suggest follow-on sentences and paragraphs that appear to make sense.
It's not doing a step-by-step algorithm. Rather, it's more like a sieve that shakes large amounts of dirt into just the grains that match the patterns on the sieve (i.e. your prompt).
In that sense I wonder if ChatGPT is better described as Northrup Frye's Order of Words: there is no connection whatsoever between its output and the Real World, but the words themselves *do* relate to one another. It finds consistent patterns, in the same way that auto-complete output makes sense if you know the likelihood of various words appearing together.
For example, if the Book of Revelation had never been written, but you had the entire Western Canon as your base corpus, a well-done ChatGPT would generate it by feeding it the other 65 books of the Bible. It's not thinking: it's just tying words together in an order that maintains consistency with the rest of the Western corpus.
What this says about being human, or what it means to truly think, I'll leave to your future essays.