Turing Test, perception, communication

[Peter Cariani, 960213, 1100]

In response to Martin Taylor, [Martin Taylor 960212 18:40].

Thanks for your useful thought experiments. I agree that the
the issues surrounding the Turing Test and imitation games
are complex, and not necessarily clear-cut. I was
pondering some of the ambiguities yesterday as I was writing --
I think I cut some of them out because they made the train
of thought less coherent (even than it was),
and some because they led into even murkier and more involved questions.

I agree with Martin when he said:

It is a _hypothesis_ (one to which I subscribe) that sensory-motor abilities
are required for the development of humna-like intelligence. The fact that
you and I both believe it to be true does not make it true. If it did,
there would be no need for it to be tested.

I think some of my cognitive dissonance re: the development of intelligence
and/or ordered thought centers around the different roles that perception
and action play in their formation and maintenance. I agree that if
one put a human being in a sensory deprivation tank and asked questions,
the human being would be able to respond appropriately to any questions that
did not depend on (new) perceptions or actions. However, this ability to
respond appropriately was gained through past perception and action, and
if the human were in the tank long enough without external stimulation,
I believe his/her thought processes would eventually deteriorate. So if one
takes the "snaphot" view of the situation and disallows empirical questions
which require (new) perceptions, then it does appear that there need not be
an essential difference between the behavior of the human and that of the
Turing machine. This is basically reiterating the point (with which I agree)
that any behavior, if well-specified, can in principle be simulated
using a finite state automaton.

The issue of ongoing perception and action and its relation to intelligence
in the Turing Test context cuts both ways. The problem is that the criteria
of the test in part determine the result of the test and its interpretation.
If one doesn't think that intelligence requires (independent) ongoing percepts,
then one eliminates them from the test by fiat. If one does think that
intelligence involves ongoing perception and action (aside from the
communications channel with the testor), then one includes the possibility
of asking those kinds of questions that require it. I am not really
interested in definitions of "intelligence" per se, but more in how different
kinds of devices have different capacities and limitations. Human beings
are a kind of device (or depending upon how one looks at them, and what
their task is, their behavior can resemble that of different kinds of devices).
So the issue for me is not carbon flesh-and-blood vs. silicon chips, it's
what can be done with pure computation vs. what can be done when sensors
and effectors are added. I agree with Martin that the communications channel
through the keyboard can be a significant source of perceptions
for the human or machine behind the curtain, but what information passes
to the human/machine testee is under the control of the testor (which makes
this a different situation from one where the human/machine can gather
information independently of the testor).

The thought experiments involving the series of graded Turing Tests, I think,
move in the direction of the "imitation game" aspects of the test and
away from the original questions that Turing was posing. There are a
number of basic definitional problems with these behavioristic
tests (e.g. Precisely how does one define who is "Japanese" and who isn't?).
Once the predicate is clearly defined, then there is also a clear test.
If the predicate is a behavioral one, you're home-free; if it's not, then
a unique behavioral correlate must be found. If one exists, one has a
test decision criterion; if not, the testor can be fooled. In the Japanese
example, an effective test might be conducted using English or some other
language (there are probably some English words and meanings that every person
who grew up in Japan knows, that are peculiar to their use in the Japanese
context, e.g. well-known "Japlish" terms).

We probably <are> thinking of two different test situations. I see the
participants in the test as being in two very different situations. The
testor is attempting to guess the identity of the human/device behind the
curtain, while the human/device is attempting to fool the testor into making
an incorrect guess. It's a very unbalanced and contrived set-up, because it
encourages the testor to give the testee as little information as possible,
and it encourages the testee to be as evasive and vague as possible. A better
situation is one where the computer is in some Internet "chat room" amongst
other humans and computers, and one tries to determine whether each
participant is a computer or a human. (Sometimes you wonder.)
I would not disallow any kind of question, including ones like
"How did the Iowa caucuses turn out?" or "How cold is it where you are?",
and I would change the structure of the game such that truthful rather
than evasive responses are encouraged (make it less adversarial).

Martin said:
* If you mean by "empirical enquiry" that you will make the discrimination
* according to which sensors the entity is provided with, then yes, I disallow
* it. I go along with Turing in saying that a blind person can be intelligent.
* If the entity can't look out of the window to see whether it is now snowing,
* would "I can't see" determine for you that the entity was non-human? How about
* "I'll have to ask my friend," which is an answer perfectly suited to current
* machine intelligence?

I do agree with Martin that a blind person can be as "intelligent"
as a sighted one, but this isn't really the distinction that's being
made, it's whether the two people can answer the same sets of questions
in the same way (and if it requires vision, the answer seems to be obviously
"no"). "I'll have to ask my friend" is a somewhat evasive response, to
which I could reply, "go ask your friend and come back with an answer."
The Turing machine is not allowed any other inputs save the communications
link with me, the testor, so it would have to evade again. The (blind)
human being is not constrained in this way, and can communicate with others
in the room. But in either case, it would be much preferable to have an
interactive structure that encourages both participants to tell the truth.

Communications. Ah, well.
I stand by what I said, that is

"A shared material environment is not necessarily an extra kind
of communication channel."

I mean this in that if both parties are observing a distant
thunderstorm that neither party has any influence over, neither can
use the thunderstorm to send a signal (which causes a reduction of
uncertainty in the other).I say "not necessarily" because there are
situations where the common material environment can be used to
send signals.

So,
"Blah, bugli unpershum inkpot" is a communication to me (I have received
a definite signal, although it's not very meaningful to me,
I would only have some vague semantic linkages for 'inkpot').
"Gully claimed a bump was a catch" is more meaningful, since I have semantic
links for more of the words and phrases. It becomes much more meaningful
when you tell me the terms involve the game of cricket, since I can then
add new semantic linkages to 'bump' and 'catch' that allow me to develop
a more coherent meaning for the sentence.

Martin says:

The point is that communication is _always_ relative to a shared context,
whether that context be linguistic, cultural, or factual. The existence of
the shared factual basis is as much a definition of the communication
channel as are the symbols that can be used in the channel.

I disagree that communication "is _always_ relative to a shared context"
because we always have different interpretive contexts that we have each
built up over time. Those interpretive contexts become calibrated and
brought into some kind of rough agreement over time, as we talk to and
cooperate (or conflict) with other human beings, animals, and devices,
but they are always different. [This is a "control" process
par excellence.] If the interpretive contexts do not
produce good coordination of actions, then the communication does not
advance our (mutual) goals very well, but good coordination of actions
does not depend on having the same context (in plenty of human
relationships the actors can have widely divergent ideas about what is
going on, yet the result of their interactions can nevertheless be
positive for both parties).

In fact, [shared material context]
utterly changes the nature of the communication channel. The
Turing Test is not supposed to be a test of the nature of the communication
channel. It's supposed to be a test of the humanity of the intellectual
processes that use the channel.

I think it's rather artificial to parse things this way. On one hand we
insist on using 'language' in the test, on the other we truncate the
nature of the communicative process that is bound up with language use.
While it's clear to me that finite-state automata can emulate many aspects
of human thought processes, there are better, more precise ways than the
Turing Test (e.g. in cognitive psychology) to frame questions
concerning which ones are amenable to such a description
(I'm sure Martin and I would agree on this).

Despite our different interpretive contexts, I think the discussion has
been fruitful (for me, at least) and it's caused me to rethink some
core issues.

Peter Cariani
peter@epl.meei.harvard.edu

(Please cc: replies to my address also. We're still working out bugs in
my CSG connection. Thanks.)

[Martin Taylor 960215]

Peter Cariani, 960213, 1100

We probably <are> thinking of two different test situations. I see the
participants in the test as being in two very different situations. The
testor is attempting to guess the identity of the human/device behind the
curtain, while the human/device is attempting to fool the testor into making
an incorrect guess.

Yes, we are thinking of two different situations. I'm thinking of one in
which the tested entity (human or machine) is trying to get the tester to
think he/she/it is human.

It's a very unbalanced and contrived set-up, because it
encourages the testor to give the testee as little information as possible,

I don't see that, even in your situation.

and it encourages the testee to be as evasive and vague as possible.

In your situation, yes. In mine, no. I thought mine was what Turing intended,
but I could be wrong. It's uninteresting as to whether a human can sustain
a pretence of being a machine. A "Mechanical Turk" was exhibited around
Europe playing chess a couple of hundred years ago, and it was not unmasked
until someone saw the dwarf emerging from under the table. There might
be more of a problem in a language interaction, although "Wizard of Oz"
experiments seem to be pretty convincing sometimes (a person pretends to be
a computer, in order to test some kind of a proposed interface that doesn't
yet exist, particularly one that uses speech recognition). The problem is
for the machine designer, to make a machine that seems to be a human.
And if a human would normally try _not_ to be seen as a machine, by being
discursive and emotional, the machine would be readily unmasked if it were
taciturn and evasive.

A better
situation is one where the computer is in some Internet "chat room" amongst
other humans and computers, and one tries to determine whether each
participant is a computer or a human. (Sometimes you wonder.)
I would not disallow any kind of question, including ones like
"How did the Iowa caucuses turn out?" or "How cold is it where you are?",
and I would change the structure of the game such that truthful rather
than evasive responses are encouraged (make it less adversarial).

That is pretty much the situation I envisaged. But "truth" doesn't come
into it, so much as "natural humanness."

"I'll have to ask my friend" is a somewhat evasive response, to
which I could reply, "go ask your friend and come back with an answer."
The Turing machine is not allowed any other inputs save the communications
link with me, the testor, so it would have to evade again.

The Turing machine is a sort of computational benchmark. Given infinite
time, it can compute whatever any other computer (finite-state machine)
can compute. But the Turing Test is applicable to any machine, and if the
machine answers more slowly than a human would, the test is likely to
be failed. Other machines are allowed other inputs; what isn't allowed
is a machine that serves as a pass-through communication channel to a
human who is actually answering the questions. Playing simultaneous
chess I can either draw twice or win one and lose one against two
grandmasters, provided I play white against one and black against the other.
My success says no more about my chess-playing ability than that I can
manipulate the pieces on the board.

"Go and ask your friend and come back with an answer" is appropriate once
in a while, but it's kind of marginal. If the tested entity were actually
a blind person, then this kind of problem would arise every time the
interaction required the tested entity to see something. The tester should
be able to note that and discount it when considering the entity's humanity.

I disagree that communication "is _always_ relative to a shared context"
because we always have different interpretive contexts that we have each
built up over time. Those interpretive contexts become calibrated and
brought into some kind of rough agreement over time, as we talk to and
cooperate (or conflict) with other human beings, animals, and devices,
but they are always different.

Of course. That's why human communication (whether with another human or
with a computer) can _never_ be treated as encoding a message at one end
and decoding it at the other. It _always_ has to be seen as control of
one's perception of the state of one's partner, a perception that (in the
Turing Test) can be developed only by observing the output of the typewriter.
The perception of that state involves the determination of what aspects
of the context are shared (and I illustrated linguistic, cultural, and
factual contexts in a previous posting).

The difference between the partners is why communication is _always_ relative
to a shared context, and cannot be relative to a coding algorithm. Each
partner has to model what context the other is working with--it's part of
the perception that comes through the typewriter. In our Layered Protocol
writings we have used a somewhat Chinese protocol: "Three Independences"
(1) Independence of design
(2) Independence of data sources
(3) Independence of action
to define a need for "intelligent" communication (as opposed to encoding-
decoding).

In fact, [shared material context]
utterly changes the nature of the communication channel. The
Turing Test is not supposed to be a test of the nature of the communication
channel. It's supposed to be a test of the humanity of the intellectual
processes that use the channel.

I think it's rather artificial to parse things this way. On one hand we
insist on using 'language' in the test, on the other we truncate the
nature of the communicative process that is bound up with language use.

From my viewpoint there's no one-hand-and-the-other here. The only truncation

I see is in the restriction to language.

While it's clear to me that finite-state automata can emulate many aspects
of human thought processes, ...

Is this an assertion that humans are infinite-state machines, or that
something non-physical is essential to the everyday behaviour of people,
or what? Any continuous system can be emulated to any desired degree of
precision by a sufficiently powerful finite-state automaton, so far as
I am aware, so I'm not clear how you intend this comment.

... there are better, more precise ways than the
Turing Test (e.g. in cognitive psychology) to frame questions
concerning which ones are amenable to such a description
(I'm sure Martin and I would agree on this).

I didn't think the Turing test was aimed in that direction, so I guess I
have to agree with you. I thought it was aimed at asking whether a machine
could be built and programmed so that its language behaviour would allow
it to pass as human in a free (language-only) interaction with another
human who was aware of the possibility that it might be a machine on the
other end of the link.

Martin