Main site

Drawings

Writings

Netherworldly
Horrors!

Ateisma í
Føroyum

The Chinese Room

‘(...) from syntax alone you don't get semantics.’ [1]

In 1980 John R. Searle put forth his now so famous argument against strong AI in an article called ‘Minds, Brains, and Programs’[2] wherein he asks us to consider the following gedankenexperiment based on the work of Roger Schank to program computers to answer questions about a story not explicitly stated in it. According to Searle strong AI’s claim, that the computer’s ability to answer such questions implies genuine understanding of the story, can be likened to his following example.

Suppose that a man is locked inside a room and this man does not understand Chinese at all. In fact his understanding of Chinese is so poor indeed that he could not for the life of him recognise Chinese symbols for what they are were he presented to them. When he sees Chinese symbols he sees nothing but a bunch of meaningless squiggles. Granted he is able to distinguish one squiggle from another “squoggle” [sic] but still he has no idea whatsoever what any of it means or even that it has an inherent meaning at all. Although the man is a completely illiterate in Chinese, the man nevertheless understands English perfectly. Unbeknownst to him there is a number of scientists outside the room who are intent on teaching him to read and write Chinese.

Therefore they devise a large compilation of symbol match-up rules for him in English and give him two batches of Chinese symbols. After the man has familiarised himself with the rules they start to regularly give him various symbols through a slot in the wall whereby he is expected to provide them with the correct Chinese symbols in response by comparing them with the English rules and the first two batches of Chinese. What the man is unaware of is that the scientists call the first two batches “a script” and “a story”, the English rules “a program”, and their input and his output “questions” and “answers” respectively.

Now suppose that the scientists became increasingly good at writing rules and that our man became increasingly apt at following them so that in the end his responses were indistinguishable from those of a native Chinese speaker. Just by looking at the answers provided by the man it is impossible to deduce that he does not even speak a word of Chinese, he does not actually understand any of it. He is merely faking it by manipulating uninterpreted formal symbols. In short the man has syntax but no semantics whatsoever.

According to Searle this discredits the validity of The Turing Test because it has been shown that there could exist two systems both of which pass The Turing Test, but only one of which can truly be said to understand.

It is important to make clear that Searle does not outright deny that machines could think, since according to him we are just that kind of machines and we can think. What Searle denies is that something could think solely in virtue of being a computer with an instantiated right kind of program. This would not be a sufficient condition of understanding because there is no intentionality in merely manipulating formal symbols, since the symbols do not actually symbolise anything and therefore, from the computer’s point of view, cannot be said to be symbols at all. The man inside The Chinese Room does have intentionality, and is programmed with a formal program, which does not contribute to him understanding Chinese and therefore it carries no additional intentionality than that which the man had in advance.

In an interview published in the Free Inquiry Searle was asked the question:

‘Do you think there ever could be an artificial intelligence that has consciousness?’

Whereby he answered:

‘I don't see why not. If we knew how the brain produced consciousness, we could eventually produce it artificially. Let's suppose that we actually discover that consciousness is produced by a certain electrical sequence in the brain, and we build a machine that would have that electrical sequence. Such an achievement might be medically useful because if my brain was starting to decay they might be able to replace it with these electrical gimmicks. So I don't have any difficulty in principle with the idea of building artificial intelligence. But if you're talking about actual conscious thought processes, you can't do it with a computer.’[3]