Much in philosophy is controversial. But I doubt that any philosopher who has studied [Hegel and Wittgenstein] would dissent from the view that self-consciousness and language emerged together...
Thus wrote the philosopher Sir Roger Scruton, in 1996.1 What would he make of present-day AIs like ChatGPT who display astounding facility with language? Alongside language, in their billions and billions of artifical neurons, has there emerged self-consciousness?
Hardly anyone believes it has. We see AIs as a new kind of computer; not a new kind of person. This then begets the question: how does language use by AI differ from language use by humans? Sadly, Sir Roger passed away in January 2020. So the task at an answer falls to us.
Make It So, Number One
We use language not just to describe reality. Sometimes, we use it to make reality. In Start Trek TNG, when Picard says Make it so, Number One, he is kicking off a chain of events that have tangible real-world effects. Now you might argue that it is the crew's subsequent physical actions that effect actual change, not Picard's spoken words. Perhaps. So let me cite another example where no physical actions are involved:
COURT CLERK: Will the foreperson please rise. As to Count 1 charging the defendant, [REDACTED], Murder in the Second Degree, how do you find the defendant, guilty or not guilty?
JURY FOREPERSON: Guilty.
In the philosopher John Searle's categorization of speech acts2, the above example would be a "declaratory point", an utterance that changes reality when spoken in the right context. The defendant's life, his status in society, changed upon the utterance Guilty.
Picard's Make it so utterance would fall under Searle’s "directive point" category. Here, the speaker attempts to get the listener to do something. An utterance indicating that the speaker is going to do something would on the other hand be a "commissory point": I promise that I'll keep this essay interesting to the end.
Interesting as speech act theory is, does it separate AI language-use from ours'? Turn left in 300 ft says my car's GPS computer, plainly directing me. Your download will complete in 3 minutes promises my browser, sounding like a "commissory" speech act. AI can seemingly participate even in "declaratory" acts: Your loan application is approved, emails a company's AI to a customer.
So it is not clear that we are making progress in distinguishing AI speech from human speech. Perhaps we can shed more light by considering the computer's point of view.
Machine Language

The first "language" used to instruct a computer was assembly language, and it appeared in 1947. Each line corresponds to executable machine code that performs basic operations within the computer's chips and circuits. Here's an example instruction:
Add the number stored in Register A to that stored in Register B, and store the result in Register B.
Registers are locations in a computer's CPU that function as memory. The above instruction increases the value residing in Register B by the amount in Register A. We have come a long way since such basic computer languages. In today's Python programming language, one can instruct a computer in just a few lines to load and sharpen a blurry 42 megapixel photograph:
from PIL import Image, ImageFilter
img = Image.open("bike-race.jpg")
sharp_img = img.filter(ImageFilter.SHARPEN)
sharp_img.save("bike-race-sharpened.jpg")
The above code runs a sophisticated formula across ~126 million numbers that describe the photograph. Not only are modern programming languages much more powerful, they are also much more intelligible—compare the Python code above with Figure 1's assembly. This improved readability is by design, to make computer programs easier to troubleshoot, modify and add to. Now, we are ready to bring AI into the discussion:
Is chatting with an AI merely an evolution of using programming languages to instruct computers? Are we simply continuing to program computers, except this time using plain English?:
Me: Can you please sharpen the attached image for me?
ChatGPT: Sure! Give me a few seconds...
As programming languages evolved from low-level assembly to high-level Python, they required more and more sophisticated software called compilers. Compilers translate code written in the programming language into instructions a computer can execute. If English is the "programming language", the "compiler" needs a whole new level of sophistication. Unlike regular programming languages, human languages do not obey strict syntactic and semantic rules, and are strongly context dependent.
Furthermore, when chatting with AI, it is not just the human half of the conversation that is in loosy-goosey English. The AI responds back in English too. So in a sense the AI has to do double the work, running its internal "compiler" in both directions. These difficulties were why the ChatGPT moment was a genuine breakthrough, and took so long to arrive.
If our present discussion has merit, it indicates that the examples I gave in the preceding section of AI performing speech acts were not true examples. When an AI gives us GPS directions, or makes loan application decisions, it is merely executing instructions from some underlying program. The AI's utterances are not speech acts. They are just computer output.
But then, who's to say that our utterances are speech acts and not computer output—aren't we machines too, running programs encoded in our DNA?
Speaker Meaning
Perhaps we are deterministic golems fueled by food. Even so, our speech differs from AI's in one crucial respect—it has meaning. When on the phone with you I say “It's raining here”, I want you to believe it is raining in New York. Assuming you trusted me, you would indeed form that belief, by recognizing that such is my intention. Conversely, if after dinner at a Chinese restaurant, you open the fortune cookie that arrives with the check, and it says “It's raining here”, you would look around, and shrug.
The philosopher Paul Grice calls our belief-inducing speech second order intention: having the intention to get the other party to accept one's intention.3 More recent academic discourse uses the term speaker meaning.4 Does AI speech have speaker meaning? Let's see.
My car's GPS doesn't seem to care if I choose to follow its directions, happily rerouting, showing no annoyance. How about AI chatbots? The latest ones do seem to have intentions. They often end an answer with "Was that useful? Would you like more detail?" Clearly they intend to be helpful. But these do not come across as second-order intentions, intentions that in some sense have designs on me, to get me to change my beliefs to match the AI's.
Epilogue
So here we have a candidate answer to this essay's opening question. Our speech, unlike AI's, is laced with intentionality. It is situated in a community of subjects, rather than a system of objects.5 Human speech emerges from and is the ingredient of human dialogue. In this social context, reasons for actions are given, intentions for plans shared, consent is sought, concessions made, rights insisted upon, and obligations honored.6
We need not reject the 1990s consensus view among philosophers that language and self-consciousness emerged together. After all, without that historical precedent there would be no examples of language to train AIs on. Perhaps what AIs do show is that language content outside of a social context can nevertheless imbue machines with—if not self-consciousness—intelligence.
Roger Scruton, 1996. An Intelligent Person’s Guide to Philosophy, p 65.
John Searle, 1969. ‘A taxonomy of illocutionary acts,’ in K. Gunderson (ed.), Language, Mind and Knowledge, Minneapolis, MN: University of Minnesota Press, 344–369.
Herbert Paul Grice, 1947. Meaning. Philosophical Review 66 (3):377-388
Stanford Encyclopedia of Philosophy. Speech Acts, Note 13. https://plato.stanford.edu/entries/speech-acts/notes.html#note-13
I am borrowing the language Sir Roger uses in his last chapter of An Intelligent Person’s Guide to Philosophy, where he discusses the philosophy of history. See p. 158.
ibid, p. 67. Sir Roger gives a more exhaustive treatment on the features of human dialogue.