Turing Tests and perception

[Peter Cariani 960208 2PM] (I've been away for a few days).

Martin Taylor [960202 15:00] had written:

The Turing Test restricts your perceptions and actions in only
one way: you can affect the test object only through language, and you
can see only its language actions. Otherwise, do what you will.

I, (Cariani) replied:

Turing's Test is actually even worse than this. It systematically excludes
both perception and action from the evaluative process.

Martin replied:

As the terms are used in this discussion group, if the enetity could not
use perception, it would have no access to what the tester typed, and
if it could not use action, the tester would have no access to what it
typed. You mean that the Turing test excludes non-language perception
and action, exactly as I said. The human is restricted to whatever
inputs and outputs are available to the tested entity. If that were
not so, the test would be inherently unfair.

When one looks closely at finite-state automata and Turing machines, the
specification of the machine and its operation does not include a description
of the process by which it gets its inputs (i.e. the description is concerned
with input-output mappings that are internal to the workings of the machine,
not about how (or why) the machine got the particular input that it did at
some particular time.

Turing was concerned with which aspects of human thought and intelligence
could be modelled/mimicked by deterministic automata, and therefore constrained
his test in such a way that only those aspects that could be performed with
a Turing machine were allowed in the test.

Martin and I both agree that "the Turing test excludes
non-language perception and action" (although I maintain that
the symbolic exchange going on between tester and machine may not
be "language" in a strict sense -- any more than typing in numbers on
your calculator keypad and reading off the result involves the
calculator's use of "language." It amazes me that many AI people
are so loathe to accept the possibility of
"language" in other primates and animals, but are so quick
to allow it in machines....but these are secondary issues).

The human is restricted to whatever
inputs and outputs are available to the tested entity. If that were
not so, the test would be inherently unfair.

I don't follow this logic at all. What does "fairness" have to do with it?
If you truncate the problem in this way, the resulting test can become
tautological. We could limit input and output to LISP statements, and it
would be hard if not impossible to distinguish the human from the Turing
machine, but the test itself would be rendered totally meaningless.
If perception and action are crucial for "thought" or "intelligence", or even
simply to mimic human capabilities, such as carrying on a conversation,
then these capabilities need to be included in the test.
We would then discover very rapidly that one needs to
add sensors and effectors to the Turing automaton in order to
take into account these capabilities. There is nothing wrong with this
process, except that the initial hypothesis, that Turing machines
by themselves, can do these things, would be immediately invalidated.

I, Cariani had said:

So, I'm not even sure that
the computer manifests what could be called "language-actions" when it
spits out a string of symbols in response to an input string. If language
isn't pure syntax, but requires something more, then the Turing Test is
not even dealing with "language" per se (Searle's argument).

Martin:

You are asserting that the Turing test can never be passed. I'd say that
this is a question that can not now be answered in the negative.

I didn't say that the Turing test can never be passed, because sometimes
it can, if the questions asked are not appropriate or if the answers
given by the human participant behind the curtain are not cooperative
(answering "I don't know" to every question does not distinguish a
human from a computer, but neither does it cause one to impute "intelligence"
or "thought" or conversational ability to the unknown respondent.) I
think that there are lines of questioning (e.g. empirical questions that
could be verifiable by a (cooperating) human but not a Turing machine)
that would invariably expose the difference between a machine without
inputs (save those from the tester) and a human being. In the public
demonstrations of the Turing test that I know about, the <kinds>
of questions that were permitted to be posed were sharply delimited.

Martin:

Your idea of language is vastly different from mine.

Perhaps. It may be also be just a different usage of the term.

In my view, language
is a way of affecting one's prception of the state of a communictive
partner. By using language you can change the partner's state, and
by the partner's using language you can perceive something of the
partner's state. In control/retrofaction/perfaction theory terms,
language serves both as output medium and as sensory input in a feedback
loop. In a cooperative dialogue, one partner has a perception that can
be brought to a reference state by the actions of the other (which may
but need not be completely verbal). The other partner has a reference
that the originator should come to be satisfied with the recipient's
state--which implies that the recipient adequately interpret the
originator's message.

This is fine for a theory of communicative cooperation.
I generally don't like to argue about the semantics of terms like
"language" (let alone "spontifaction" or "retrofaction"),
but would you call an interaction with a calculator a
"language-based" interaction? There is some signalling, yes, but
no rewards (pragmatics) or linkages to the rest of the world
(external semantics) on the calulator side.
There is just syntax. As you (Martin) say,
"'Pure syntax' has very little to do with language in use." but
this is what is going on on the Turing machine side of the dialog.
How does the Turing machine, "the other partner", "have a reference
state that the originator should come to be satisfied with the recipient's
state"? I don't see any perception of the state of the testor
happening nor is there any reward interaction in the Test, either, so
why call what is going on here "language"?
I agree that the discussion of these issues is very slippery and
very difficult, which is why I generally don't use the term 'language'
very often (too much baggage). My mistake.

I, Cariani had said:

Had he stuck with automata with finite tapes,
we would have a theory of computability that is coextensive with what
devices can physically be built (i.e. what computations can be actually carried
out by physical machines), instead of the present confusions
about whether Turing-computability per se matters at all
for real world computation or for the workings of the brain.

Martin:

It's worse than that. Turing computability relates to algorithms, processes
whereby an input data set is transformed into an output data set. A given
algorithm will give the same output every time it is given the same input.
Algorithms don't work in a disturbed environment. People do. People
interact with their environment, whether you believe in retrofaction
theory or not, and the data on which they are working changes constantly.
It is immaterial whether there are algorithms to compute this or that
function, provided that what a person does keeps her alive and well.

I'm very ambivalent about using terms like "algorithm" and "computation"
in the context of neural systems -- on one hand neural systems are very
much unlike digital computers, but on the other, these terms have come to
mean, in general parlance, "informational operations".

The translation of "algorithms" into "neural processes" is therefore
anything but straightforward, but one can definitely say that
there are reliable transformations that are carried out in all neural
systems (we could call them "perceptual computations", like recognizing
the call of frog's of one's own species, or distinguishing the kind of
flying insect by its echo-pattern).

I believe that feedback control is
very, very important in adapting to a constantly changing environment, but
stability of perceptual processing and motor actions is also important.
There are some parts of the system that you don't want to be constantly
changing. If we had a very different percept every time
a given stimulus was presented, we would not be able
to reliably make an appropriate response. To the extent
that these processes are stable and reliable, they can be described as
rule-governed (or if discrete, even "algorithmic") processes.
I believe that these processes can be either analog or discrete.
(analog: perception of color or pitch, steering a car)
(discrete: sequencing of discrete percepts/actions/thoughts,
e.g. in speech perception/production/cognition). I believe that there are both
analog and discrete aspects to perception, action, and cognition.

To relate computability to intelligence is an enormous mistake, far more
damaging to cognitive science than the damage you impute Turing as
having done to AI.

I'm not sure if I understand this. (I believe) Turing cast intelligence purely
in terms of discrete computation. Turing did some very seminal
work in using differential equations to model morphogensis, (he understood
the complexity of the behaviors open to analog systems) so this could be
an oversimplification of his position.

I agree that the adoption of a discrete computational metaphor
for cognitive science has been very, very damaging in and of itself,
without even compounding the problem by introducing infinite machines
into the brew.

What matters is not whether a particular computation might require an
infinite tape, but whether the person can jump out of the way before
the leaping tiger arrives. It's time that's the big problem,in real life.
In the case of the Turing Test, a computer that took 24 hours to answer
any question, however complex, would not be readily confused with a human.
Time comes into even the test using typed communication.

I couldn't agree more. It's clear you need fast, reliable sensors
and "perceptual computations" to detect and recognize the tiger,
and fast, reliable reflexes to get out of the way.

Time is of the essence.

Peter Cariani
peter@epl.meei.harvard.edu

[Martin Taylor 960209 13:40]

Peter Cariani 960208 2PM

I get a distinct flavour of cross-purpose discussion. Normally that's
unfruitful. Let's see if we can identify some of the difference and go from
there.

I, (Cariani) replied:

Turing's Test is actually even worse than this. It systematically excludes
both perception and action from the evaluative process.

Martin replied:

As the terms are used in this discussion group, if the enetity could not
use perception, it would have no access to what the tester typed, and
if it could not use action, the tester would have no access to what it
typed.

When one looks closely at finite-state automata and Turing machines, the
specification of the machine and its operation does not include a description
of the process by which it gets its inputs (i.e. the description is concerned
with input-output mappings that are internal to the workings of the machine,
not about how (or why) the machine got the particular input that it did at
some particular time.

The Turing _machine_ has no specification of its input mechanism, but the
Turing _Test_ requires that there be one. Similarly, the Turing _machine_
has no specified output, but the Turing _Test_ requires one. In fact the
Turing _Test_ specified what the input and output mechanism were to be
--typewriters.

The human who might be the tested entity has access to the tester _only_
through the typewriter. So does the machine (Turing or not) who/that
might be the tested entity. Whatever else either can do is not specified.
I can ask you "Is it raining where you are?" and if you are either a human
or a machine, you can answer "Yes, but only gently." If I can look out of
the window and have good reason to believe that I am seeing what you are
seeing, then I have an extra kind of communication channel with you other
than the typewriter. That's not allowed.

To see this more clearly, imagine a question "How hard does a feather fall?"
Answer "About like the snowflakes we are seeing out of the window." If the
parties aren't seeing the same kind of snow, that's no answer, but if they
each believe the other to be seeing the same thing, it is a good answer.
That's using a communication channel outside the typewriter.

The point I see in the Turing _Test_ is the interaction between the entity
and the tester. To put it in PCT terms, the tester tries to disturb something
that the entity might be controlling. If no such thing can be found, either
you have a non-cooperative human (as I said before) or you have an S-R machine
with preprogrammed algorithms for generating responses. Without the
interactive probing, the Turing tester is unable to tell the difference.

The human is restricted to whatever
inputs and outputs are available to the tested entity. If that were
not so, the test would be inherently unfair.

I don't follow this logic at all. What does "fairness" have to do with it?
If you truncate the problem in this way, the resulting test can become
tautological. We could limit input and output to LISP statements, and it
would be hard if not impossible to distinguish the human from the Turing
machine, but the test itself would be rendered totally meaningless.

Why would it be hard? Because humans have a difficult time reading LISP
statements? The feedback loop remains the same. One would look for evidence
of control on the other side, and the higher the level of perception at
which one probed, the more stringent the test. What does it matter what
kind of symbol strings are allowed in the test, if their syntax allows
an indefinite number of them?

Fairness has to do with not requiring the machine to be a blue-eyed
blond, with squishy pink-skinned flesh. The Turing test refers to its
thought processes. If these attributes are _in fact_ required before an
entity's thought processes can be humanlike, then _in fact_ no machine
without them will pass the test. But the test does not, and should not,
require them _a priori_.

Language is a pretty fair way to interact with what we vaguely call
"intelligence," but the test is readily extended to cover any other ways
that someone might dream up, provided that there is agreement that those
new ways relate to "intelligence" and not some unrelated attribute. I think Hawking is intelligent, but he doesn't
much manipulate his physical environment. Why do I think him intelligent?
Because of the results of his use of language.

If perception and action are crucial for "thought" or "intelligence", or even
simply to mimic human capabilities, such as carrying on a conversation,
then these capabilities need to be included in the test.

This is where I think there is a deep difference in understanding. I argue
that the Test presupposes that the tested entity is at least able to
perceive and act through the typewriter. You seem to require the word
"perception" to include anything _except_ the typewriter. I don't mind
what other sensory capabilities the tested entity might have, provided
they don't open up a potential channel to the tester (as illustrated above).

Or else you are asserting _a priori_ what the Turing test is intended to
discover. Can an entity that interacts with the world only through the
typewriter during the test do so in a way indistinguishable from a human?

We would then discover very rapidly that one needs to
add sensors and effectors to the Turing automaton in order to
take into account these capabilities.

Perhaps so. But we don't know so, yet.

You are moving into the range of Stevan Harnad's "Total Turing Test" here.
But think of what you are saying--that a human, placed in a soundproof
room dark except for the screen with the tester's writing, would no longer
be human and would be indistinguishable from a machine.

There is nothing wrong with this
process, except that the initial hypothesis, that Turing machines
by themselves, can do these things, would be immediately invalidated.

And, as before, you are asserting _a priori_ that the Turing test is
unpassable by a computer, past, present, or future. I think you miss the
point of the test.

I didn't say that the Turing test can never be passed, because sometimes
it can, if the questions asked are not appropriate or if the answers
given by the human participant behind the curtain are not cooperative
(answering "I don't know" to every question does not distinguish a
human from a computer, but neither does it cause one to impute "intelligence"
or "thought" or conversational ability to the unknown respondent.)

Yes, there are two ways to pass the test--by the human being taken for a
machine, and by the machine being taken for a human. When we talk about
passing the test, we usually by default mean the latter. Humans have
been successfully imitating machines for centuries, so it's no news when
the test gets passed that way.

I think that there are lines of questioning (e.g. empirical questions that
could be verifiable by a (cooperating) human but not a Turing machine)
that would invariably expose the difference between a machine without
inputs (save those from the tester) and a human being.

Now again you are asserting an answer to the question Turing posed. You
say that such questions exist, and I think that you should archive them
so that if and when a plausible candidate machine passes other attempts
to test it, your lines of questioning should then be used to unmask the
pretender.

But I'm not at all sure how you would do this, since the essence of the
feedback loop is that the interactions are hard to predict in advance. If
they were not, an S-R machine could execute them.

As you (Martin) say,
"'Pure syntax' has very little to do with language in use." but
this is what is going on on the Turing machine side of the dialog.

The difference between the Turing machine and the human in this respect is
that you know how the Turing machine works and can make that assertion (using
a particular notion of "syntax" to mean "logical operations"). You don't
know how the human works, and therefore you assert that the human doesn't
use logical operations (syntax). Since the point of the test is to determine
whether such a distinction is generic, it is improper of you to assert
that it is before running the test. Your comment is like the following
"syllogism": A is an X. I don't know whether B is an X. Therefore B is not
of the same kind as A.

How does the Turing machine, "the other partner", "have a reference
state that the originator should come to be satisfied with the recipient's
state"?

In most computers, that's built in by the designer. If one were to design
a program with the intention of passing the Turing test, one would surely
have to build in the possibility that it had such a reference perception
and an analysis of the state of the tester to compare it with.

I don't see any perception of the state of the testor
happening

In which machine? Have you in fact programmed one to which you want to
apply the Turing test, and you left out this essential component? Or
if not, how do you know that the programmer of the machine in question
did so? In the interface development projects that we try to do, we try to
make this perception of the state of the user (tester, if you will) quite
explicit, at least at the higher levels of abstraction. It's part, only a
small part, but a most important part, of the system's model.

nor is there any reward interaction in the Test, either, so
why call what is going on here "language"?

What is "reward?" In PCT it is only the reduction of error in some system,
presumably more rewarding the more closely the controlled perception relates
to an intrinsic variable. In a computer, the "intrinsic variables" are
whatever the designer decided on, and I am supposing here that one of them
is the perception of the user's state of satisfaction. But there could be
others, this perception being only a way to bring the others to their
reference values. What does it matter that the control system is made of
silicon rather than carbon compounds?

If you are concerned about the difference between analogue and logical
interactions, remember that any finite-bandwidth analogue process can be
emulated to any predetermined precision by a sufficiently fast logical
process.

The translation of "algorithms" into "neural processes" is therefore
anything but straightforward, but one can definitely say that
there are reliable transformations that are carried out in all neural
systems (we could call them "perceptual computations", like recognizing
the call of frog's of one's own species, or distinguishing the kind of
flying insect by its echo-pattern).

Any input-output transform can be considered an algorithm, and if its
input data keep changing, it can be considered a filter. When its output
affects its input within the time frame of a "complete computation cycle"
(however you want to interpret that for a particular structure), things
become a little awkward. The results may not be indeterminate, but they
are highly likely to become chaotic, to explode to infinity, or to become
independent of the data input. One of the problems of designing a complex
control structure is avoiding such regimes.

To relate computability to intelligence is an enormous mistake, far more
damaging to cognitive science than the damage you impute Turing as
having done to AI.

I'm not sure if I understand this.

The arguments about computability deal with problems like the stopping
problem, or whether something is N-P complete,..., or Godel's theorem.
All of these things don't matter to an intelligent entity whose job is
to "survive and be happy." A travelling salesman who sells enough
and gets home often enough doesn't much care if his trip could have
been shortened by 1%. If a statement is neither provably true nor provably
false, it doesn't matter provided that the wrong choice doesn't hurt too
much. There's a lot else happening in the world that might make the "right"
choice wrong in practice. "Intelligence" is the ability to perform safely
and accurately in a wide range of disturbing environments. Most logical
analyses take the environment as given and look for the consequences of
certain prior states.

If you remember, at one time the definitive test of intelligence was
whether the entity could do arithmetic. When it was found that an easily
constructed machine could do it better than any human, arithmetic ability
was no longer deemed to have anything to do with intelligence.

Turing had a better idea, related it more to what humans do--can the machine
behave like a human in real time, given the input-output mechanisms known
to him. Furthermore, when the machine is tested _by_ a human who is allowed
to try to fake it out, will it still seem like a human. Will it pass the
PCT Test?

Time is of the essence.

And computability is not.

Martin