The English Turing Test

For more info on the Turing Test see here, specifically the "immitation game" portion.

     After recently reading an article about a Korean middle school girl who achieved a perfect score on a standardized multiple choice English test (TOEFL) I had to say to myself, wow, this girl studied very hard, and did a good job. Good for her. But my next thought was, is a multiple choice test a fair measure of language ability? There's no multiple choice merger negotiation, or essay. I have yet to see a multiple choice public interview. [perhaps I don't understand the test properly?]

     One of the things we have yet to enable computers to do is to beat the Turing test. How is this relevant you might ask. The Turing test involves sitting a human at a computer terminal and trying to figure out if the messages on the terminal are coming from a computer or a human. The means of responding to the person includes a language element. Language ability is one defining characteristic of humans that distinguishes us from a machine. The second point of my argument is that since the machine can evaluate the answers to the TOEFL test, because it is a multiple choice test and not an open ended test, I propose the machine could theoretically learn/study for, and also pass the test. This machine, mind you, not is not yet able to pass the Turing test. So you have a non-human like machine able to pass what supposedly is designed to test what only humans are extensively proficient in. This means you're not really testing the critical, in my opinion, or what I might call the 'organic' or 'human' part of language.


     I propose a different kind of test: an "English" Turing test. Here the test taker would attempt to respond in English to a native English speaker mediated through some means. The native speaker or speakers, or perhaps English scholars or some other authorities on English would then have to decide whether or not the "person behind the curtain" was a native speaker or not. If the expert deemed them a native speaker then they pass. If not then they fail. Obviously there are problems with this test that would have to be addressed first, like discovering the probability of false negatives on the control groups of natives, and some way of determining an acceptable way of mediating the subjectivity of what constitutes a positive or negative, however it would be interesting to see if this is an effective testing method. Maybe this has already been proposed, but I think if it hasn't it ought to be attempted.