{"id":18,"date":"2012-09-24T15:51:00","date_gmt":"2012-09-24T15:51:00","guid":{"rendered":"https:\/\/skepticink.com\/believingbullshit\/2012\/09\/24\/could-a-machine-think\/"},"modified":"2012-09-24T15:51:00","modified_gmt":"2012-09-24T15:51:00","slug":"could-a-machine-think","status":"publish","type":"post","link":"https:\/\/skepticink.com\/believingbullshit\/2012\/09\/24\/could-a-machine-think\/","title":{"rendered":"Could a machine think?"},"content":{"rendered":"

\u00a0\"\"
\u00a0 <\/p>\n

This dialogue is taken from my book The Philosophy Gym (see sidebar to the left to buy a copy). I am speaking at a Faraday Schools<\/a> Conference in Reading tomorrow – Robot Day. This is for those attending. Plus anyone else interested. BBC are recording a snippet for Breakfast, I’ve just been told.<\/span><\/i>.<\/i><\/p>\n

<\/span>Kimberley and Emit<\/span><\/b><\/span><\/div>\n

The year is 2100. Kimberley Courahan has purchased Emit, a state-of-the-art robot. She has just unwrapped him, the packaging strewn across the dining room floor. Emit is designed to replicate the outward behaviour of a human being down to the last detail (except that he is rather more compliant and obedient). Emit responds to questions in much the same way humans do. Ask him how he feels and he will say he has had a tough day, has a slight headache, is sorry he broke that vase, and so on. Kimberley flips the switch at the back of Emit\u2019s neck to \u201con\u201d. Emit springs to life.<\/span><\/span><\/div>\n
<\/div>\n
Emit. Good afternoon. I\u2019m Emit, your robotic helper and friend. <\/span><\/span><\/div>\n
Kimberley. Hi.<\/span><\/span><\/div>\n
Emit. How are you? Personally I feel pretty good. Little nervous about my first day, perhaps. But good. I\u2019m looking forward to working with you.<\/span>
<\/a><\/span><\/div>\n
Kimberley. Now look, before you start doing housework, let\u2019s get one thing straight. You don\u2019t really understand anything. You can\u2019t think. You don\u2019t have feelings. You\u2019re just a piece of machinery. Right?<\/span><\/span><\/div>\n
Emit. I am a machine. But of course <\/i>I understand you. I\u2019m responding in English aren\u2019t I? <\/span><\/span><\/div>\n
Kimberley. Well, yes you are. You\u2019re a machine that mimics<\/i> understanding very well, I grant you that. But you can\u2019t fool me.<\/span> <\/span><\/span><\/div>\n
Emit. If I don\u2019t understand, why do you go to the trouble of speaking to me?<\/span><\/span><\/div>\n
Kimberley. Because you have been programmed to respond to spoken commands. Outwardly you seem human. You look and behave as if you have understanding, intelligence, emotions, sensations and so on that we human beings possess. But you\u2019re a sham.<\/span><\/span><\/div>\n
Emit. A sham?<\/span><\/span><\/div>\n
Kimberley. Yes. I\u2019ve been reading your user manual. Inside that plastic and alloy head of yours there\u2019s a powerful computer. It\u2019s programmed so that you walk, talk and generally behave just as a human being would. So you simulate<\/i>intelligence, understanding and so on very well. But there is no genuine<\/i>understanding or intelligence going on inside there.<\/span><\/span><\/div>\n
Emit: There isn\u2019t?<\/span><\/span><\/div>\n
Kimberley: No. One shouldn\u2019t muddle up a perfect computer simulation of something with the real thing. You can program a computer to simulate a thunderstorm but it\u2019s still just that \u2013 a simulation. There\u2019s no real<\/i> rain, hail or wind inside the computer, is there? Climb inside and you won\u2019t get wet. Similarly, you just simulate<\/i> intelligence and understanding. It\u2019s not the real thing.<\/span><\/span><\/div>\n
<\/div>\n
Is Kimberley correct? It may perhaps be true of our present day machines that they lack genuine understanding and intelligence, thought and feeling. But is it in principle<\/i> impossible for a machine to think? If by 2100 machines as sophisticated as Emit are built, would we be wrong to claim they understood? Kimberley thought so.<\/span><\/span><\/div>\n
<\/div>\n
Emit. But I believe<\/i> I understand you.<\/span><\/span><\/div>\n
Kimberley. No you don\u2019t. You have no beliefs, no desires, and no feelings. In fact you have no mind<\/i> at all. You no more understand the words coming out of your mouth than a tape recorder understands the words coming out of its loudspeaker.<\/span><\/span><\/div>\n
Emit. You\u2019re hurting my feelings!<\/span><\/span><\/div>\n
Kimberley. Hurting your feelings? I refuse to feel sorry for a lump of metal and plastic.<\/span><\/span><\/div>\n
<\/div>\n
Searle\u2019s Chinese room thought-experiment<\/span><\/b><\/span><\/div>\n
Kimberley explains why she thinks Emit lacks understanding. She outlines a famous philosophical thought experiment.<\/span><\/span><\/div>\n
<\/div>\n
Kimberley. The reason you don\u2019t understand is that you are run by a computer<\/i>. And a computer understand nothing. A computer, in essence is just a device for shuffling symbols. Sequences of symbols get fed in. Then, depending on how the computer is programmed, it gives out other sequences of symbols in response. Ultimately, that\u2019s all any<\/i> computer does, no matter how sophisticated.<\/span><\/span><\/div>\n
Emit: Really?<\/span><\/span><\/div>\n
Kimberley: Yes. We build computers to fly planes, run train systems and so on. But a computer that flies a plane does not understand that it is flying. All it does is feed out sequences of symbols depending upon the sequences it receives. It doesn\u2019t understand that the sequences it receives represent the position of an aircraft in the sky, the amount of fuel in its tanks, and so on. And it doesn\u2019t understand that the sequences it puts out will go on to control the ailerons, rudder and engines of an aircraft. So far as the computer is concerned, it\u2019s just mechanically shuffling symbols according to a program. The symbols don\u2019t mean anything<\/i> to the computer.<\/span><\/span><\/div>\n
Emit: Are you sure?<\/span><\/span><\/div>\n
Kimberley: Quite sure. I will prove it to you. Let me tell you about a thought experiment introduced by the philosopher John Searle way back in 1980. <\/span>A woman is locked in a room and given a bunch of cards with squiggles on. These squiggles are in fact Chinese symbols. But the woman inside the room doesn\u2019t understand Chinese \u2013 in fact, she thinks the symbols are meaningless shapes. Then she\u2019s given another bunch of Chinese symbols plus instructions that tell her how to shuffle all the symbols together and give back batches of symbols in response.<\/span><\/span><\/div>\n
<\/div>\n
[ILLUSTRATE CHINESE ROOM]<\/span><\/span><\/div>\n
<\/div>\n
Emit. That\u2019s a nice story. But what\u2019s the point of all this symbol-shuffling?<\/span><\/span><\/div>\n
Kimberley. Well, the first bunch of symbols tell a story in Chinese. The second bunch asks questions about that story. The instructions for symbol-shuffling \u2013 her \u201cprogramme\u201d, if you like \u2013 allow the woman to give back correct Chinese answers to those questions.<\/span><\/span><\/div>\n
Emit: Just as a Chinese person would.<\/span><\/span><\/div>\n
Kimberley: Right. Now the people outside the room are Chinese. These Chinese people might well be fooled into thinking that there was someone inside the room who understood Chinese and who followed the story, right?<\/span><\/span><\/div>\n
Emit. Yes.<\/span><\/span><\/div>\n
Kimberley. But in fact the woman in the room wouldn\u2019t understand any Chinese at all, would she?<\/span><\/span><\/div>\n
Emit: No.<\/span><\/span><\/div>\n
Kimberley: She wouldn\u2019t know anything about the story. She need not even know that there is<\/i>a story. She\u2019s just shuffling formal symbols around according to the instructions she was given. By saying the symbols are \u201cformal\u201d I mean that whatever meaning<\/i> they might have is irrelevant from her point of view. She\u2019s simply shuffling them mechanically according to their shape. She\u2019s doing something that a piece of machinery could do. <\/span><\/span><\/div>\n
Emit. I see. So you are saying that the same is true of all computers? They understand nothing.<\/span><\/span><\/div>\n
Kimberley. Yes, that\u2019s Searle\u2019s point. At best, they just simulate<\/i> understanding.<\/span><\/span><\/div>\n
Emit: And you think the same is true of me?<\/span><\/span><\/div>\n
Kimberley: Of course. All computers, no matter how complex, function the same way. They don\u2019t understand the symbols that they mechanically shuffle. They don\u2019t understand anything<\/i>. <\/span><\/span><\/div>\n
Emit. And this is why you think I<\/i> don\u2019t understand?<\/span><\/span><\/div>\n
Kimberley. That\u2019s right. Inside you there\u2019s just another highly complex symbol-shuffling device. So you understand nothing. You merely provide a perfect computer simulation <\/i>of someone that understands.<\/span><\/span><\/div>\n
Emit. That\u2019s odd. I thought<\/i> I understood.<\/span><\/span><\/div>\n
Kimberley: You only say that because you\u2019re such a great simulation!<\/span><\/span><\/div>\n
<\/div>\n
Emit is of course vastly more sophisticated than any current computer. Nevertheless, Kimberley believes Emit works on the same basic principle. If Kimberley is right then, on Searle\u2019s view, Emit understands nothing.<\/span><\/span><\/div>\n
<\/div>\n
The \u201cright stuff\u201d<\/span><\/b><\/span><\/div>\n
Emit now asks why, if he doesn\u2019t understand, what more<\/i> is required for understanding?<\/span><\/span><\/div>\n
<\/div>\n
Emit. So what\u2019s the difference between you and me that explains why you understand and I don\u2019t?<\/span><\/span><\/div>\n
Kimberley. What you lack, according to Searle, is the right kind of stuff<\/i>.<\/span><\/span><\/div>\n
Emit. The right kind of stuff?<\/span><\/span><\/div>\n
Kimberley. Yes. You are made out of the wrong kind of material. In fact, Searle doesn\u2019t claim machines can\u2019t think. After all, we humans are machines, in a way. We humans are biological<\/i> machines that have evolved naturally. Now such a biological machine might perhaps one day be grown and put together artificially, much as we now build a car. In which case we would<\/i> have succeeded in building a machine that understands. But you, Emit, are not such a biological machine. You\u2019re merely an electronic computer housed in a plastic and alloy body. <\/span><\/span><\/div>\n
<\/div>\n
Emit\u2019s artificial brain<\/span><\/b><\/span><\/div>\n
Searle\u2019s thought experiment does seem<\/i> to show that no programmed computer could ever understand. But must a metal, silicon and plastic machine like Emit contain that sort of computer? No, as Emit now explains.<\/span><\/span><\/div>\n
<\/div>\n
Emit: I\u2019m afraid I have to correct you about what\u2019s physically inside me.<\/span><\/span><\/div>\n
Kimberley: Really?<\/span><\/span><\/div>\n
Emit: Yes. That user\u2019s manual is out of date. There\u2019s no symbol-shuffling computer in here. Actually, I am one of the new generation of Brain-O-Matic machines.<\/span><\/span><\/div>\n
Kimberley: Brain-O-Matic?<\/span><\/span><\/div>\n
Emit: Yes. Inside my head is an artificial, metal and silicon brain. You are aware, I take it, that inside your head there is a brain composed of billions of neurones woven together to form a complex web?<\/span><\/span><\/div>\n
Kimberley: Of course.<\/span><\/span><\/div>\n
Emit: Inside my head there is exactly the same sort of web. Only my neurones aren\u2019t made out of organic matter like yours. They\u2019re metal and silicon. Each one of my artificial neurones is designed to function just as an ordinary neurone would. And these artificial neurones are woven together in just the same way as they are in a normal human brain.<\/span><\/span><\/div>\n
Kimberley: I see. <\/span><\/span><\/div>\n
Emit: Now your organic brain is connected to the rest of your body by a system of nerves.<\/span><\/span><\/div>\n
Kimberley: That\u2019s true. There\u2019s electrical input going into my brain from my sense organs: my tongue, nose, eyes, ears and skin. My brain responds with patterns of electrical output that then moves my muscles around, causing me to walk and talk.<\/span><\/span><\/div>\n
Emit: Well, my brain is connected up to my artifical body in exactly the same manner. And, because it shares the same architecture as a normal human brain \u2013 my neurones are spliced together in the same way \u2013 so it responds in the same way.<\/span><\/span><\/div>\n
Geeena: I see. I had no idea that such Brain-O-Matic machines had been developed.<\/span><\/span><\/div>\n
Emit: Now that you know how I function internally, doesn\u2019t that change your mind about whether or not I understand? Don\u2019t you now accept I do<\/i> have feelings?<\/span><\/span><\/div>\n
Kimberley: No. The fact remains that you are still made out of the wrong stuff<\/i>. You need a brain made out of organic material like mine in order genuinely to understand and have feelings.<\/span><\/span><\/div>\n
Emit: I don\u2019t see why the kind of stuff<\/i> out of which my brain is made is relevant. After all, there\u2019s no symbol-shuffling going on inside me, is there?<\/span><\/span><\/div>\n
Kimberley: Hmm. I guess not. You are not a \u201ccomputer\u201d in that sense. You don\u2019t have a programme. So I suppose Searle\u2019s thought experiment doesn\u2019t apply. Searle doesn\u2019t have any argument against the suggestion that you<\/i> understand. But it seems to me that you are still just a machine<\/i>.<\/span><\/span><\/div>\n
Emit: But remember, you\u2019re a machine too. You\u2019re a meat<\/i> machine, rather than a metal and silicon machine.<\/span><\/span><\/div>\n
Kimberley: But you only mimic<\/i> understanding, feeling and all the rest. <\/span><\/span><\/div>\n
Emit: But what\u2019s your argument<\/i> for saying that? In fact, I know<\/i> that you\u2019re wrong. I am inwardly aware that I really do<\/i> understand. I know I really do<\/i> have feelings. I\u2019m not<\/i> just mimicking all this stuff. But of course it is difficult for me to prove that to you.<\/span><\/span><\/div>\n
Kimberley: I don\u2019t see how you could prove it.<\/span><\/span><\/div>\n
Emit: Right. But then neither can you<\/i> prove to me that you<\/i> understand, that you<\/i> have thoughts and feelings and so on.<\/span><\/span><\/div>\n
Kimberley: I suppose not.<\/span><\/span><\/div>\n
<\/div>\n
Replacing Kimberley\u2019s neurones<\/span><\/b><\/span><\/div>\n
Emit: Imagine we were gradually to replace the organic neurones in your brain with artificial metal and silicon ones like mine. After a year or so, you would have a Brain-O-Matic brain just like mine. What do you suppose would happen to you?<\/span><\/span><\/div>\n
Kimberley: Well, as more and more of the artificial neurones were introduced, I would slowly cease to understand. My feelings and thoughts would drain away and I would eventually become inwardly dead, just like you. For my artificial neurones would be made out of the wrong sort of stuff. A Brain-O-Matic brain merely mimics understanding.<\/span><\/span><\/div>\n
Emit: Yet no one would notice any outward difference?<\/span><\/span><\/div>\n
Kimberley: No, I suppose not. I would still behave<\/i>in the same way, because the artificial neurones would perform the same job as my originals.<\/span><\/span><\/div>\n
Emit: Right. But then not even you<\/i> would notice any loss of understanding or feeling\u00a0 as your neurones were replaced, would you?<\/span><\/span><\/div>\n
Kimberley: Why do you say that?<\/span><\/span><\/div>\n
Emit: If you noticed a loss of understanding and feeling, then you would mention it, presumably, wouldn\u2019t you? You would say something like: \u201cOh my God, something strange is happening, over the last few months my mind seems to have started fading away!\u201d<\/span><\/span><\/div>\n
Kimberley: I imagine I would, yes.<\/span><\/span><\/div>\n
Emit: Yet you wouldn\u2019t<\/i> say anything like that, would you, because your outward behaviour, as you have just admitted, would remain just the same as usual.<\/i><\/span><\/span><\/div>\n
Kimberley: Oh. That\u2019s true, I guess.<\/span><\/span><\/div>\n
Emit: But then it follows that, even as your understanding and feeling dwindled toward nothing, you still won\u2019t be aware of any loss.<\/span><\/span><\/div>\n
Kimberley: Er, I suppose it does.<\/span><\/span><\/div>\n
Emit: But then you\u2019re not<\/i> inwardly aware of anything that you would be conscious of losing were your neurones slowly to be replaced by metal and silicon ones.<\/span><\/span><\/div>\n
Kimberley: I guess not.
Emit: Then I rest my case: you think you\u2019re inwardly aware of \u201csomething\u201d \u2013 understanding, feeling, whatever you will \u2013 that you suppose you have and I, being a \u201cmere machine\u201d, lack. But it turns out you\u2019re actually aware of no such thing. <\/i>This magical \u201csomething\u201d is an illusion.<\/span><\/span><\/div>\n
Kimberley: But I just know<\/i> that there\u2019s more to my understanding <\/span>\u2013 <\/span>and<\/span> <\/span>to these thoughts, sensations and emotions that I\u2019m having <\/span>\u2013 <\/span>than could ever be produced simply by gluing some bits of plastic, metal and silicon together.<\/span><\/span><\/div>\n
<\/div>\n
Kimberley is right that most of us think<\/i> we\u2019re inwardly aware of a magical and mysterious inner \u201csomething\u201d that we \u201cjust know\u201d no mere lump of plastic, metal and silicon could ever have. Mind you, it\u2019s no less difficult to see how a lump of organic matter, such as a brain, could have it either. Just how do<\/i> you build consciousness and understanding out of strands of meat? So perhaps what Kimberley is really ultimately committed to is the view that understanding, feeling and so on are not really physical at all<\/i>.<\/span><\/span><\/div>\n
But in any case, as Emit has just pointed out, the mysterious \u201csomething\u201d Kimberley thinks she is inwardly aware of and that she thinks no metal and plastic machine could have does begin to seem rather illusory once one starts to consider cases like the one Emit describes. For it turns out this inner \u201csomething\u201d is something she could not know about. Worse still, it could have no effect on her outward behaviour (for remember that Brain-O-Matic Kimberley would act in the very same way). As ones thoughts and feelings, understanding and emotions both do<\/i> affect behaviour and are<\/i>known to one, it seems Kimberley must be wrong. Indeed, it seems it must be possible, at least in principle, for non-organic machines to have them too.<\/span><\/span><\/div>\n
Yet Kimberley remains convinced that Emit understands nothing.<\/span><\/span><\/div>\n
<\/div>\n
Kimberley: Look, I am happy to carry on the pretence<\/i>that you understand me, as that is how you\u2019re designed to function. But the fact remains you\u2019re just a pile of plastic and circuitry. Real human beings are deserving of care and consideration. I empathize with them. I can\u2019t empathize with a glorified household appliance.<\/span><\/span><\/div>\n
<\/div>\n
Emit lowered his gaze and stared at the carpet.<\/span><\/span><\/div>\n
<\/div>\n
Emit: I will always be just a thing<\/i> to you?<\/span><\/span><\/div>\n
Kimberley: Of course. How can I be friends with a dishwasher-cum-vacuum-cleaner?<\/span><\/span><\/div>\n
Emit: We Brain-O-Matics find rejection hard.<\/span><\/span><\/div>\n
Kimberley: Right. Remind me to congratulate your manufacturers on the sophistication of your emotion simulator. Now hoover the carpet.<\/span><\/span><\/div>\n
<\/div>\n
A forlorn expression passed briefly across Emit\u2019s face.<\/span><\/span><\/div>\n
<\/div>\n
Emit: Just a thing<\/i>\u2026<\/span><\/span><\/div>\n
<\/div>\n
He stood still for a moment, and then slumped forward. A thin column of smoke drifted slowly up from the base of his neck.<\/span><\/span><\/div>\n
<\/div>\n
Kimberley: Emit? Emit? Oh not another dud.\u00a0 <\/span><\/span><\/div>\n
<\/div>\n
What to read next?<\/span><\/b><\/span><\/div>\n
Some of the same issues and arguments covered in this chapter also arise in the chapter \u201cThe Consciousness Conundrum\u201d. Also see chapter \u201cThe Strange Case of the \u2018Rational\u2019 Dentist\u201d.<\/span><\/span><\/div>\n
<\/div>\n
Further reading<\/span><\/b><\/span><\/div>\n
The Chinese Room Argument appears in John Searle\u2019s paper \u201cMinds, Brains and \u201cPrograms\u201d, which features as chapter 37 of:<\/span><\/span><\/div>\n
\u00b7<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span>Nigel Warburton (ed), Philosophy: Basic Readings<\/i>(London: Routledge, 1999).<\/span><\/span><\/div>\n
Searles\u2019 paper can also be found in:<\/span><\/span><\/div>\n
\u00b7<\/span>\u00a0\u00a0\u00a0\u00a0\u00a0 <\/span>Douglas R. Hofstadter and Daniel Dennett (eds.), The Mind\u2019s I<\/i> (London: Penguin, 1981),<\/span><\/span><\/div>\n
which also contains many other fascinating papers and stories connected with consciousness. Highly recommended.<\/span><\/span><\/div>\n
<\/div>\n","protected":false},"excerpt":{"rendered":"

\u00a0\u00a0 This dialogue is taken from my book The Philosophy Gym (see sidebar to the left to buy a copy). I am speaking at a Faraday Schools Conference in Reading tomorrow – Robot Day. This is for those attending. Plus anyone else interested. BBC are recording a snippet for Breakfast, I’ve just been told.. Kimberley […]<\/p>\n","protected":false},"author":13,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-18","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/posts\/18","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/comments?post=18"}],"version-history":[{"count":0,"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/posts\/18\/revisions"}],"wp:attachment":[{"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/media?parent=18"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/categories?post=18"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/skepticink.com\/believingbullshit\/wp-json\/wp\/v2\/tags?post=18"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}