Persistent objects

CSGnet Archive CSG2009
1

[From Richard Kennaway (2009.09.20.2235 BST)]

This is a thought I've been thinking for the last few days, but haven't got very far with. Maybe someone else can take it further. Or maybe this is all in B:CP already.

Suppose I have some object in front of me -- say, a book. I can pick it up, turn it around, riffle the pages. Its image on my retina and the feelings in my hands are changing, yet throughout all that, I still have the perception of that particular book. That perception remains constant despite the disturbances.

Therefore there is a control system at work.

The same applies to every experience of a persistent object. The raw input is always changing but the perception of "that object" persists unchanged. Whenever we look around at things, or pick out a sound from the background noise, the objects of our experience are not merely perceptions, but controlled perceptions. Or perhaps the objects of our experience are not the perceptions, but the references.

But these control systems don't seem to fit into the PCT hierarchy. Where do those references come from? If the book just happened to be in front of me, no higher-level control system decided to perceive that book. I can look around at the thousands (at least) of objects around me, all of them retaining their subjective identity, yet I didn't specifically intend to perceive these particular objects.

Objects aren't just things that can be named. When I look at a textured surface, the details seem to have the same sort of persistency, as long as I'm looking at them.

If these perceived objects are controlled perceptions or references, what makes those control systems start up and shut down? What gives them their references? What were those neurons doing before the object was perceived, and what will they be doing after?

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, http://www.cmp.uea.ac.uk/~jrk/
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

2

Iirc, the 'interactive AI' work by Phil Agre and David Chapman that was once upon a time discussed on CSGNET (I don't think there are any available archives from back then (late 90s)) used a concept of a 'register' to track objects; the register would contain information about the object such as its current location and other relevant attributes, and it struck me once upon a time that various control systems might be involved in keeping the contents of a register synchronized with the attributes of whatever it is that the register is tracking.

These systems would have to control perceptions such that, if those perceptions stay close enough to their references, the contents of the register will update properly with the properties of what it's supposed to be tracking (third person scientists' view not available to the organism itself, but rather part of a putative explanation of why the organism doesn't quickly become extinct).

One possible such perception is that the visual properties represented by whatever is in the location specified by the register stay constant, so that for example, if the visual property is 'deer' at location 30 deg (relative some axis), and the deer moves, then the visual property will change to 'vegetation', and the control system can try to fix this by changing the location a bit so that the visual property is 'deer' again. A technique that will fail if there are too many deer in the vicinity that are moving too fast, but that's when we lose track of which deer we're looking at.

So the perception of 'sameness' would then be lo/no error signal from the tracking systems associated with the registers

···

-----Original Message-----
From: Control Systems Group Network (CSGnet) on behalf of Richard Kennaway
Sent: Mon 21/09/2009 8:11 AM
To: CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Persistent objects

[From Richard Kennaway (2009.09.20.2235 BST)]

This is a thought I've been thinking for the last few days, but
haven't got very far with. Maybe someone else can take it further.
Or maybe this is all in B:CP already.

Suppose I have some object in front of me -- say, a book. I can pick
it up, turn it around, riffle the pages. Its image on my retina and
the feelings in my hands are changing, yet throughout all that, I
still have the perception of that particular book. That perception
remains constant despite the disturbances.

Therefore there is a control system at work.

The same applies to every experience of a persistent object. The raw
input is always changing but the perception of "that object" persists
unchanged. Whenever we look around at things, or pick out a sound
from the background noise, the objects of our experience are not
merely perceptions, but controlled perceptions. Or perhaps the
objects of our experience are not the perceptions, but the references.

But these control systems don't seem to fit into the PCT hierarchy.
Where do those references come from? If the book just happened to be
in front of me, no higher-level control system decided to perceive
that book. I can look around at the thousands (at least) of objects
around me, all of them retaining their subjective identity, yet I
didn't specifically intend to perceive these particular objects.

Objects aren't just things that can be named. When I look at a
textured surface, the details seem to have the same sort of
persistency, as long as I'm looking at them.

If these perceived objects are controlled perceptions or references,
what makes those control systems start up and shut down? What gives
them their references? What were those neurons doing before the
object was perceived, and what will they be doing after?

--
Richard Kennaway, jrk@cmp.uea.ac.uk, http://www.cmp.uea.ac.uk/~jrk/
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

3

[Martin Taylor 2009.09.21.0049]

[From Richard Kennaway (2009.09.20.2235 BST)]

Suppose I have some object in front of me -- say, a book. I can pick it up, turn it around, riffle the pages. Its image on my retina and the feelings in my hands are changing, yet throughout all that, I still have the perception of that particular book. That perception remains constant despite the disturbances.

Therefore there is a control system at work.

That seems a very long leap of logic to me. It could be true, but I'd like to see smaller steps so that I could follow the reasoning.

Martin

4

[From Richard Kennaway (2009.09.21.0850 BST)]

[Martin Taylor 2009.09.21.0049]
> [From Richard Kennaway (2009.09.20.2235 BST)]
> Suppose I have some object in front of me -- say, a book. I can pick

it up, turn it around, riffle the pages. Its image on my retina and
the feelings in my hands are changing, yet throughout all that, I
still have the perception of that particular book. That perception
remains constant despite the disturbances.

> Therefore there is a control system at work.

That seems a very long leap of logic to me. It could be true, but I'd
like to see smaller steps so that I could follow the reasoning.

It seems a short one to me. There is a perception, and disturbances that one would expect to change the perception, but the perception remains constant. Therefore etc. This is just the tect for the controlled variable.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

5

[From Fred Nickols (2009.09.21.0818 EST)]

My reaction is different from Martin's. When you turn the book in your hands, riffle its pages, etc, I would guess you are not controlling for a fixed image or perception of the book in your imagination; you are instead controlling for something else (perhaps the feel or look or heft of the book as you turn it, or perhaps to reacquaint yourself with its contents, or perhaps to simply occupy your hands). Only the test would tell. But the very act of moving the book around is totally inconsistent with maintaining that "fixed image" or perception in your imagination.

Can you in fact have and maintain such an image? I would assume so and I would also imagine you can position the book to more or less align with that image if you choose to do so. But that's a different action and a different outcome.

···

--
Regards,

Fred Nickols
Managing Partner
Distance Consulting, LLC
nickols@att.net
www.nickols.us

"Assistance at A Distance"
  
-------------- Original message ----------------------
From: Richard Kennaway <jrk@CMP.UEA.AC.UK>

[From Richard Kennaway (2009.09.21.0850 BST)]

>[Martin Taylor 2009.09.21.0049]
> > [From Richard Kennaway (2009.09.20.2235 BST)]
> > Suppose I have some object in front of me -- say, a book. I can pick
>> it up, turn it around, riffle the pages. Its image on my retina and
>> the feelings in my hands are changing, yet throughout all that, I
>> still have the perception of that particular book. That perception
>> remains constant despite the disturbances.
>>
> > Therefore there is a control system at work.
>>
>
>That seems a very long leap of logic to me. It could be true, but I'd
>like to see smaller steps so that I could follow the reasoning.

It seems a short one to me. There is a perception, and disturbances
that one would expect to change the perception, but the perception
remains constant. Therefore etc. This is just the tect for the
controlled variable.

6

[From Richard Kennaway (2009.09.21.1329 BST)]

[From Fred Nickols (2009.09.21.0818 EST)]

My reaction is different from Martin's. When you turn the book in your hands, riffle its pages, etc, I would guess you are not controlling for a fixed image or perception of the book in your imagination; you are instead controlling for something else (perhaps the feel or look or heft of the book as you turn it, or perhaps to reacquaint yourself with its contents, or perhaps to simply occupy your hands). Only the test would tell. But the very act of moving the book around is totally inconsistent with maintaining that "fixed image" or perception in your imagination.

Can you in fact have and maintain such an image? I would assume so and I would also imagine you can position the book to more or less align with that image if you choose to do so. But that's a different action and a different outcome.

I'm not talking about an image, but the perception that this is "the same" book, even while the physical sensations radically change. That perception remains constant, and no positioning of the book in the visual field is required. How does this happen?

Here's a non-verbal example: ceiling | There is actually a reason I uploaded this picture… | Flickr
(A photograph of the ceiling above my chair.) You can move your head around while looking at that picture, and yet the details appear to have a certain persistence. How does that happen?

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

7

Okay, I see what you're talking about now. My honest answer is that I don't
know how it happens. I would guess that your perception of it being "the same
book" is also a conclusion based on comparing perceived properties of the book
against your reference signal for "that" book.

···

--
Regards,

Fred Nickols
Managing Partner
Distance Consulting, LLC
nickols@att.net
www.nickols.us

"Assistance at A Distance"
  
-------------- Original message ----------------------
From: Richard Kennaway <jrk@cmp.uea.ac.uk>

[From Richard Kennaway (2009.09.21.1329 BST)]

>[From Fred Nickols (2009.09.21.0818 EST)]
>
>My reaction is different from Martin's. When you turn the book in
>your hands, riffle its pages, etc, I would guess you are not
>controlling for a fixed image or perception of the book in your
>imagination; you are instead controlling for something else (perhaps
>the feel or look or heft of the book as you turn it, or perhaps to
>reacquaint yourself with its contents, or perhaps to simply occupy
>your hands). Only the test would tell. But the very act of moving
>the book around is totally inconsistent with maintaining that "fixed
>image" or perception in your imagination.
>
>Can you in fact have and maintain such an image? I would assume so
>and I would also imagine you can position the book to more or less
>align with that image if you choose to do so. But that's a
>different action and a different outcome.

I'm not talking about an image, but the perception that this is "the
same" book, even while the physical sensations radically change.
That perception remains constant, and no positioning of the book in
the visual field is required. How does this happen?

Here's a non-verbal example: ceiling | There is actually a reason I uploaded this picture… | Flickr
(A photograph of the ceiling above my chair.) You can move your head
around while looking at that picture, and yet the details appear to
have a certain persistence. How does that happen?

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

8

[Martin Taylor 2009.09.21.10.45]

[From Richard Kennaway (2009.09.21.0850 BST)]

[Martin Taylor 2009.09.21.0049]
> [From Richard Kennaway (2009.09.20.2235 BST)]
> Suppose I have some object in front of me -- say, a book. I can pick

it up, turn it around, riffle the pages. Its image on my retina and
the feelings in my hands are changing, yet throughout all that, I
still have the perception of that particular book. That perception
remains constant despite the disturbances.

> Therefore there is a control system at work.

That seems a very long leap of logic to me. It could be true, but I'd
like to see smaller steps so that I could follow the reasoning.

It seems a short one to me. There is a perception, and disturbances that one would expect to change the perception, but the perception remains constant. Therefore etc. This is just the tect for the controlled variable.

I guess that what I mean by a long leap is that the leap excludes other possibilities, such as those that the computational vision community (and "conventional psychologists") have been pursuing. There are various threads in those approaches, but they amount largely to the same thing as is implied by the hierarchic levels of perception in classic HPCT. At various stages of processing, different kinds of properties are abstracted, leading in the case of a book, to a category-level perception. At a very low level, for example, T-junctions of brightness edges are often assumed to be a passively abstracted feature. At a higher level, object continuity over time occurs in the absence of unusually abrupt alterations in the motions of coherent edge patterns.

I'm not saying that the approaches used by these communities are correct, only that your leap of logic fails to exclude them. It is a "deemed solution", not a demonstration.

To my mind, your designation of these changes as "disturbances" leads me to question the usage of the word. You can call me on this one, but I think the word "disturbance" should be applied after, not before, one has reason to suspect that a particular variable is being controlled. Your whole question hinges on the assumption that _every_ stable perception necessarily must represent a controlled variable, and that therefore any changes in the input data pattern must necessarily be disturbances to that control, and that therefore it is necessary to discover a control loop that stabilizes the perception. The communities I mention start with the opposite assumption, which is that there is inherent in the incoming data stream something that allows for the perception of stable, coherent objects.

My own assumption, no more legitimate than theirs or yours, is that stability of perception of objects in the world depends in part upon computable coherences in the data stream, and in part upon the coherences that do or do not exist when one deliberately (controlling perception) manipulates the world. COnsider your book example. If you did not move your observation point, pick up the book, riffle the pages, and so forth, you would not be able to determine whether it was an "Ames Room" construction or a chocolate sculpture. So my assumption accepts that control is involved in the stability of perception, but it is a control whose action output directly alters the external environment. Interaction with the world seems to be very important in developing perceptions at even moderately high levels, whereas it seems less important at the lower levels.

All of the above is not to say you are wrong in your assumption, but to suggest that the logical leap is a longer one than you imagine.

Martin

9

[From Bill Powers (2009.09.21.0852 MDT)]

Richard Kennaway (2009.09.21.1329 BST) --

I'm not talking about an image, but the perception that this is "the same" book, even while the physical sensations radically change. That perception remains constant, and no positioning of the book in the visual field is required. How does this happen?

I think this is simply a phenomenon of perception, not of control. A perceptual signal is a function of many lower-order perceptual signals.
This means that there are certain ways in which the lower-order perceptual signals can change without changing the higher-order perceptual signal, and orthogonal to those ways are the changes that do change the higher perceptual signal. See "orthogonal trajectories".

  When people talk about perceptions being "invariants," they have to mean invariant under certain kinds of transformations, but of course not all possible transformations. Disturbances which change the inputs to a perceptual function along trajectories that don't alter the value of the function are not disturbances to a control system which is controlling the value of the function.

Unfortunately, I can't tell you what the functions are that derive invariants such as the configuration we call (at a higher level) "a book," but obviously there are transformations (such as rotation in 3-space) that do not disturb the perception of bookness, while others (such as ripping the cover off) do disturb it. When we understand how the third level of perception is brought about, this will probably become obvious. I don't think anyone is close to doing that yet. I think we understand, a little, the first two of eleven levels.

There's another phenomenon in your examples that brings in the method of levels, particular the way awareness can move from level to level in the brain. When your awareness narrows down to perceptions of classes of configurations, and even further to books, suddenly the whole universe consists of configurations that are classified either as books or non-books. When the sun rises in the morning, we can see that it is clearly not a book. And so on. This works at every level of perception, and is a source of confusion when you lose sight of the fact that other levels of perception exist, or when you view a system from a level of your own that is higher than any that exist in that system being examined (a definition, perhaps, of anthopomorphism).

Best,

Bill P.

10

[From Rick Marken (2009.09.21.0830)]

Martin Taylor (2009.09.21.10.45)–

I guess that what I mean by a long leap is that the leap excludes other possibilities, such as those that the computational vision community (and “conventional psychologists”) have been pursuing. There are various threads in those approaches, but they amount largely to the same thing as is implied by the hierarchic levels of perception in classic HPCT.

I agree with Martin here. I think the PCT solution to the “persistance” of the perception of something like a book despite variation in other perceptual dimensions such as orientation, distance, etc. is in the perceptual function that produces the constant (persistant) perceptual signal. Of course, how one builds such perceptual functions is quite another matter. But it’s possible that the neural networks that implement the perceptual functions that produce this kind of constant perceptual signal may require some closed loop arrangements in the neural network itself. So Avery and Richard are probably right, too.

By the way, this is no longer a completely philosophical problem. My ATM can now read the checks I deposit. I would imagine that the software that does the reading process does some active adjustments (controlling) of the binary representation of the image (the intensity level perception) to compensate for variations how the checks comes through the slot in order to do some of the “passive” optical character recognition" (configuration and categgory level percpetoin?).

Best

Rick

···

–
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

11

[Martin Taylor 2009.09.21.10.45]
I'm not saying that the approaches used by these communities are
correct, only that your leap of logic fails to exclude them. It is a
"deemed solution", not a demonstration.

It's just a hypothesis at the moment.

To my mind, your designation of these changes as "disturbances" leads me
to question the usage of the word. You can call me on this one, but I
think the word "disturbance" should be applied after, not before, one
has reason to suspect that a particular variable is being controlled.

"Influences", then. The word doesn't matter, only the phenomenon of stability of high-level perceptions in spite of variation in the lower-level signals, and without any external actions undertaken to maintain them.

So my assumption
accepts that control is involved in the stability of perception, but it
is a control whose action output directly alters the external
environment. Interaction with the world seems to be very important in
developing perceptions at even moderately high levels, whereas it seems
less important at the lower levels.

That seems to be converging on what I suggested originally. If there's a difference, it's that I'm allowing for control processes going on entirely within the nervous system, instead of the actions always proceeding out through the muscles and the environment before their effects on perceptions come back in.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

12

can you please remove my email from your list,

Thanks

** Cristian
Mazilu**

···

— On Mon, 9/21/09, Bill Powers powers_w@FRONTIER.NET wrote:

From: Bill Powers
powers_w@FRONTIER.NET
Subject: Re: Persistent objects
To: CSGNET@LISTSERV.ILLINOIS.EDU
Date: Monday, September 21, 2009, 6:14 PM

[From Bill Powers (2009.09.21.0852 MDT)]

Richard Kennaway (2009.09.21.1329 BST) –

I’m not talking about an image, but the perception that this is “the same” book, even while the physical sensations radically change. That perception remains constant, and no positioning of the book in the visual field is required. How does this happen?

I think this is simply a phenomenon of perception, not of control. A perceptual signal is a function of many lower-order perceptual signals.
This means that there are certain ways in which the lower-order perceptual signals can change without changing the higher-order perceptual signal, and orthogonal to those ways are the changes that do change the higher perceptual signal. See “orthogonal trajectories”.

http://en.wikipedia.org/wiki/Orthogonal_trajectory

When people talk about perceptions being “invariants,” they have to mean invariant under certain kinds of transformations, but of course not all possible transformations. Disturbances which change the inputs to a perceptual function along trajectories that don’t alter the value of the function are not disturbances to a control system which is controlling the value of the function.

Unfortunately, I can’t tell you what the functions are that derive invariants such as the configuration we call (at a higher level) “a book,” but obviously there are transformations (such as rotation in 3-space) that do not disturb the perception of bookness, while others (such as ripping the cover off) do disturb it. When we understand how the third level of perception is brought about, this will probably become obvious. I don’t think
anyone is close to doing that yet. I think we understand, a little, the first two of eleven levels.

There’s another phenomenon in your examples that brings in the method of levels, particular the way awareness can move from level to level in the brain. When your awareness narrows down to perceptions of classes of configurations, and even further to books, suddenly the whole universe consists of configurations that are classified either as books or non-books. When the sun rises in the morning, we can see that it is clearly not a book. And so on. This works at every level of perception, and is a source of confusion when you lose sight of the fact that other levels of perception exist, or when you view a system from a level of your own that is higher than any that exist in that system being examined (a definition, perhaps, of anthopomorphism).

Best,

Bill P.

13

It is accomplished.

···

On Mon, Sep 21, 2009 at 10:31 AM, Cristian Mazilu themazilu@yahoo.com wrote:

can you please remove my email from your list,

Thanks

** Cristian
Mazilu**

— On Mon, 9/21/09, Bill Powers powers_w@FRONTIER.NET wrote:

From: Bill Powers
powers_w@FRONTIER.NET
Subject: Re: Persistent objects
To: CSGNET@LISTSERV.ILLINOIS.EDU

Date: Monday, September 21, 2009, 6:14 PM

[From Bill Powers (2009.09.21.0852 MDT)]

Richard Kennaway (2009.09.21.1329 BST) –

I’m not talking about an image, but the perception that this is “the same” book, even while the physical sensations radically change. That perception remains constant, and no positioning of the book in the visual field is required. How does this happen?

I think this is simply a phenomenon of perception, not of control. A perceptual signal is a function of many lower-order perceptual signals.
This means that there are certain ways in which the lower-order perceptual signals can change without changing the higher-order perceptual signal, and orthogonal to those ways are the changes that do change the higher perceptual signal. See “orthogonal trajectories”.

http://en.wikipedia.org/wiki/Orthogonal_trajectory

When people talk about perceptions being “invariants,” they have to mean invariant under certain kinds of transformations, but of course not all possible transformations. Disturbances which change the inputs to a perceptual function along trajectories that don’t alter the value of the function are not disturbances to a control system which is controlling the value of the function.

Unfortunately, I can’t tell you what the functions are that derive invariants such as the configuration we call (at a higher level) “a book,” but obviously there are transformations (such as rotation in 3-space) that do not disturb the perception of bookness, while others (such as ripping the cover off) do disturb it. When we understand how the third level of perception is brought about, this will probably become obvious. I don’t think
anyone is close to doing that yet. I think we understand, a little, the first two of eleven levels.

There’s another phenomenon in your examples that brings in the method of levels, particular the way awareness can move from level to level in the brain. When your awareness narrows down to perceptions of classes of configurations, and even further to books, suddenly the whole universe consists of configurations that are classified either as books or non-books. When the sun rises in the morning, we can see that it is clearly not a book. And so on. This works at every level of perception, and is a source of confusion when you lose sight of the fact that other levels of perception exist, or when you view a system from a level of your own that is higher than any that exist in that system being examined (a definition, perhaps, of anthopomorphism).

Best,

Bill P.

–
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

14

[From Bill Powers (2009.09.22.0756 MDTE)]

BP: Hi, Ave! Good to hear from you. I miss your dad.

AA: Iirc, the 'interactive AI' work by Phil Agre and David Chapman that was once upon a time discussed on CSGNET (I don't think there are any available archives from back then (late 90s)) used a concept of a 'register' to track objects; the register would contain information about the object such as its current location and other relevant attributes, and it struck me once upon a time that various control systems might be involved in keeping the contents of a register synchronized with the attributes of whatever it is that the register is tracking.

BP: Sure, if there is a register. But surely that is just a literal transplant from computer models, referring to registers in the CPU chip! Which image goes back to the registers as in the old hand-operated Marchant calculating machines.

What are the registers for? That is, what is keeping track of where objects are, and why? Is the idea that we have internal maps of the environment that we have to keep updated for coordination among the senses? What is it that looks at these maps, if that is what the registers are?

AA: One possible such perception is that the visual properties represented by whatever is in the location specified by the register stay constant, so that for example, if the visual property is 'deer' at location 30 deg (relative some axis), and the deer moves, then the visual property will change to 'vegetation', and the control system can try to fix this by changing the location a bit so that the visual property is 'deer' again. A technique that will fail if there are too many deer in the vicinity that are moving too fast, but that's when we lose track of which deer we're looking at.

BP: Does this mean that every time a new object appears, something assigns a register to it and fills in the data? What sort of something is that? Or are we talking category level, where only the names of perceptions would go into the register, rather than the perceptual signals themselves?

I could make more sense of this if I knew what the overall model is in which these registers are embedded.

Love,

Uncle Bill

···

At 10:41 AM 9/21/2009 +1000, Avery Andrews wrote:

15

[From Avery Andrews (2009.09.23.0807 AEST)]

I would guess that the idea came from computer registers, but the real reason is presumably that they thought they needed it for the game-player simulations they were constructing. Part of the idea is that there are only a few registers, the maximum number would be the number of distinct objects that you can keep track of at the same time, one for each, so when you stop tracking something specifically, its register would be reassigned or left unused for a while. So if I'm making coffee, one might get allocated to the grinder, one to the coffee itself, and one to the plunger, so that once I've noticed where all these things are, I don't have to look for them again each time I turn my back on one of them (but if an unhelpful person moves them around behind my back, I might get upset). If there are any old CSG archives around somewhere, there might be more discussion of this under the rubric of 'Jerry' (the name of a contemplated simulation of Jerry Fodor doing things in the kitchen, can't remember why I (and maybe Penny Sibun) were picking on Foder).

I can't find my copy of the Chapman book to refresh my memory of how their model works (things can hide in plain site on my bookshelves for a very long time), but my version of the idea is that it's something that might help to support control of certain kinds of perceptions, especially ones that involve following procedures. Also moving around in dynamic environments, for example if somebody or something is moving, and you can assign to it a register that contains its velocity information, then you can generate rough predictions of where it is now for a while without having to look at it continuously. One of Oliver Sack's books describes a patient who doesn't have this capacity, for whom being out on the street is very dangerous, and being at parties very stressful, since people and things keep popping up in completely unexpected places.

That picture of you and Mary from the old days on fb was a bit of a tearjerker too!

- Avery

···

-----Original Message-----
From: Control Systems Group Network (CSGnet) on behalf of Bill Powers
Sent: Wed 23/09/2009 12:10 AM
To: CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Re: Persistent objects

[From Bill Powers (2009.09.22.0756 MDTE)]

At 10:41 AM 9/21/2009 +1000, Avery Andrews wrote:

BP: Hi, Ave! Good to hear from you. I miss your dad.

AA: Iirc, the 'interactive AI' work by Phil Agre and David Chapman
that was once upon a time discussed on CSGNET (I don't think there
are any available archives from back then (late 90s)) used a concept
of a 'register' to track objects; the register would contain
information about the object such as its current location and other
relevant attributes, and it struck me once upon a time that various
control systems might be involved in keeping the contents of a
register synchronized with the attributes of whatever it is that the
register is tracking.

BP: Sure, if there is a register. But surely that is just a literal
transplant from computer models, referring to registers in the CPU
chip! Which image goes back to the registers as in the old
hand-operated Marchant calculating machines.

What are the registers for? That is, what is keeping track of where
objects are, and why? Is the idea that we have internal maps of the
environment that we have to keep updated for coordination among the
senses? What is it that looks at these maps, if that is what the
registers are?

AA: One possible such perception is that the visual properties
represented by whatever is in the location specified by the register
stay constant, so that for example, if the visual property is 'deer'
at location 30 deg (relative some axis), and the deer moves, then
the visual property will change to 'vegetation', and the control
system can try to fix this by changing the location a bit so that
the visual property is 'deer' again. A technique that will fail if
there are too many deer in the vicinity that are moving too fast,
but that's when we lose track of which deer we're looking at.

BP: Does this mean that every time a new object appears, something
assigns a register to it and fills in the data? What sort of
something is that? Or are we talking category level, where only the
names of perceptions would go into the register, rather than the
perceptual signals themselves?

I could make more sense of this if I knew what the overall model is
in which these registers are embedded.

Love,

Uncle Bill

16

[From Bill Powers (w2009.089.23.0642 MDT)]

[From Avery Andrews (2009.09.23.0807 AEST)]

I would guess that the idea came from computer registers ... so that once I've noticed where all these things are, I don't have to look for them again each time I turn my back on one of them.

...

Also moving around in dynamic environments, for example if somebody or something is moving, and you can assign to it a register that contains its velocity information, then you can generate rough predictions of where it is now for a while without having to look at it continuously.

OK, it makes more sense now. Not the idea of "registers" attached to an "it", but the observation that we have to account somehow for the ability to "know where something is" in relation to us.

I'm sitting with my back to a door, but I know it's there. Part of the knowing is a very sketchy image of the door as it would look if I turned around. Another part of it is a sort of awareness of the back of my head, the part of me that is toward the door right now. I can imagine now reaching backward to point toward the door. Imagination mode, of course.

The idea that seems to fit for me is that of a "map", or a 3-D mental model like a diorama. The "register" concept would suggest just storing numerical coordinates and verbal descriptions (as in your example of the "deer" at a "30-degree" angle), but I think that's probably pretty rare. I don't see "door" or "180 degrees", I see the image of the door (sort of) and feel a relationship to it, in analog form.

The 3-D map idea also comes into play in accounting for the way vision and the other senses like kinesthesia get calibrated in terms of each other. It's as if there is some common framework that gets built up, and something translates directional and location information into that framework with a suitable set of rotations, translations, and magnifications to make the different representations agree with each other. When, with my eyes closed, I reach my arm out so it feels "straight ahead," it also looks "straight ahead" when I open my eyes. Putting funny glasses on messes up the mutual calibration and it takes a while for agreement to be re-established, and taking them off afterward requires re-calibration. Knowing where things are would then entail looking in imagination at this mental model. You'd still have to search for them as you do when they're in the real field of view, of course, but you'd be looking about in the right direction.

Tim Carey put me onto a book called "The brain that changes itself," by Norman Doidge [sic], MD (which my local public library had). It's about the extreme plasticity of the brain being observed in modern times, completely contrary to some strong beliefs about the brain's ability to reorganize held in the past (Doidge uses the term "reorganize"). The examples are very illuminating though unfortunately the theoretical framework in which they're described is pretty primitive SR stuff. I can't say I read every chapter with equal attention, but one series of experiments I did read about is relevant to this "intercalibration map" idea. For example, the experimenter has you hold your hand palm up under a table so you can't see it, and starts tapping the table top above your hand. At the same time he reaches under the table and taps the palm of your hand in unison with the taps on top. After a short time, you allegedly feel the tapping as if it's being felt on the table top you can see instead of down where your real hand is that you can't see. I tried it on my knee under the table but could't get the illusion, probably because I could feel myself tapping the table top with one hand and my knee with the other. I haven't had a chance to try this with someone else.

There's another you can do alone: cross your middle finger over your index finger far enough so you can slide a pencil longitudinally against the V between the fingertips, like operating a miniature saw or violin bow. Close your eyes while doing this, and you'll feel after a while that there are two pencils. I've never been able to recalibrate that one to where it feels like one pencil again, though just at first the experience wobbles back and forth between one and two. Oops, there, I just made it go back to one by looking at it.

[Strange phrases that suddenly make sense. "I just made it go back to one" is total nonsense outside the context above. Theodore Sturgeon wrote a short story with the baffling title, "To Here and the Easel." It was about an artist who, in the middle of painting a picture, is transported into a different world of fantasy in some distant place (I forget the details), and finally describes finding himself returned to here and the easel. Then there was "An adventure in the yolla bolly middle eel wilderness" which I leave as an exercise for readers not familiar with California (as I wasn't when I encountered the story in F&SF).] Once you know the context these phrases all make sense, but not until then.

It's pretty obvious that our PCT model of the brain is only scratching the surface. It could be that perceptual functions in the brain are really ALL reorganization with NO preorganization into levels. That would imply that the levels which do show up in similar form in different people are a result of the interaction of the brain with Real Reality, so there is really some external organization with similar degrees of freedom. A real comfort to naive realists.

And this would help explain how it is that blind people given a matrix of vibrations on their backs, mapped from a video camera, start seeing very much as sighted people do, though with lower resolution. Newborn ferrets that had their optic nerves surgically relocated to their auditory cortexes started acting as if they could see, and the auditory cortex, if I'm recalling this right, began to take on the anatomical appearance of the visual cortex in normal animals.

And when you use a cane as a blind person does, with your eyes closed, you start feeling objects with the tip of the cane instead of your hand. Use a ball-point pen's tip to feel the shape of your coffee cup, eyes closed. I just tried it, and it works fine. Wierd!

Best,

Bill P.

17

[From Avery Andrews (2009.09.24.0935 AEST)]

There clearly need to be maps, and, furthermore, some kind of facility
for long-term persistent objects (our current television, my computer at
university), different from the short-term ones. The registers would
best be seen as a rather mere speculation about what might be inolved in
control of program-level perceptions such as [the one that one might
express verbally as] 'I am going through the steps of making coffee'.
One of the reasons I stopped thinking about Jerry (a hypothetical
simulation of Jerry Fodor making breakfast in a kitchen) was lack of
foundational skills to implement anything like that to a reasonable
degree of depth, another was the difficulty of substantiating the
details, whatever they might turn out to be.

In linguistics, there turns out to be an interesting difference between
'stages' and 'individuals', whereby 'stage level' attributes such as
'drunk' and 'naked' apply to what one might think of as manifestations
of individuals in particular situations (John at Susan's party'), so
that if you encounter an individual with that attribute in one
situation, you don't really expect them to have it in a later encounter,
while 'individual level' attributes such as 'tall' and 'intelligent' are
expected to carry over between encounters. So I'd conjecture that the
stage-level atributes go into the registers (short term, local
management of particular situations), while the individual-level ones
are associated with long term identity of individuals (which has its own
set of paradoxes, such as the computer in which every part gets replaced
in succession (the realistic modern example of the ship where every
plank gets replaced)).

  - Avery

···

-----Original Message-----
From: Control Systems Group Network (CSGnet)
[mailto:CSGNET@LISTSERV.ILLINOIS.EDU] On Behalf Of Bill Powers
Sent: Thursday, 24 September 2009 12:18 AM
To: CSGNET@LISTSERV.ILLINOIS.EDU
Subject: Re: Persistent objects

[From Bill Powers (w2009.089.23.0642 MDT)]

[From Avery Andrews (2009.09.23.0807 AEST)]

I would guess that the idea came from computer registers ... so that
once I've noticed where all these things are, I don't have to look for
them again each time I turn my back on one of them.

...

Also moving around in dynamic environments, for example if somebody or
something is moving, and you can assign to it a register that contains
its velocity information, then you can generate rough predictions of
where it is now for a while without having to look at it continuously.

OK, it makes more sense now. Not the idea of "registers" attached to an
"it", but the observation that we have to account somehow for the
ability to "know where something is" in relation to us.

I'm sitting with my back to a door, but I know it's there. Part of the
knowing is a very sketchy image of the door as it would look if I turned
around. Another part of it is a sort of awareness of the back of my
head, the part of me that is toward the door right now. I can imagine
now reaching backward to point toward the door. Imagination mode, of
course.

The idea that seems to fit for me is that of a "map", or a 3-D mental
model like a diorama. The "register" concept would suggest just storing
numerical coordinates and verbal descriptions (as in your example of the
"deer" at a "30-degree" angle), but I think that's probably pretty rare.
I don't see "door" or "180 degrees", I see the image of the door (sort
of) and feel a relationship to it, in analog form.

The 3-D map idea also comes into play in accounting for the way vision
and the other senses like kinesthesia get calibrated in terms of each
other. It's as if there is some common framework that gets built up, and
something translates directional and location information into that
framework with a suitable set of rotations, translations, and
magnifications to make the different representations agree with each
other. When, with my eyes closed, I reach my arm out so it feels
"straight ahead," it also looks "straight ahead" when I open my eyes.
Putting funny glasses on messes up the mutual calibration and it takes a
while for agreement to be re-established, and taking them off afterward
requires re-calibration. Knowing where things are would then entail
looking in imagination at this mental model. You'd still have to search
for them as you do when they're in the real field of view, of course,
but you'd be looking about in the right direction.

Tim Carey put me onto a book called "The brain that changes itself,"
by Norman Doidge [sic], MD (which my local public library had). It's
about the extreme plasticity of the brain being observed in modern
times, completely contrary to some strong beliefs about the brain's
ability to reorganize held in the past (Doidge uses the term
"reorganize"). The examples are very illuminating though unfortunately
the theoretical framework in which they're described is pretty primitive
SR stuff. I can't say I read every chapter with equal attention, but one
series of experiments I did read about is relevant to this
"intercalibration map" idea. For example, the experimenter has you hold
your hand palm up under a table so you can't see it, and starts tapping
the table top above your hand. At the same time he reaches under the
table and taps the palm of your hand in unison with the taps on top.
After a short time, you allegedly feel the tapping as if it's being felt
on the table top you can see instead of down where your real hand is
that you can't see. I tried it on my knee under the table but could't
get the illusion, probably because I could feel myself tapping the table
top with one hand and my knee with the other. I haven't had a chance to
try this with someone else.

There's another you can do alone: cross your middle finger over your
index finger far enough so you can slide a pencil longitudinally against
the V between the fingertips, like operating a miniature saw or violin
bow. Close your eyes while doing this, and you'll feel after a while
that there are two pencils. I've never been able to recalibrate that one
to where it feels like one pencil again, though just at first the
experience wobbles back and forth between one and two. Oops, there, I
just made it go back to one by looking at it.

[Strange phrases that suddenly make sense. "I just made it go back to
one" is total nonsense outside the context above. Theodore Sturgeon
wrote a short story with the baffling title, "To Here and the Easel."
It was about an artist who, in the middle of painting a picture, is
transported into a different world of fantasy in some distant place (I
forget the details), and finally describes finding himself returned to
here and the easel. Then there was "An adventure in the yolla bolly
middle eel wilderness" which I leave as an exercise for readers not
familiar with California (as I wasn't when I encountered the story in
F&SF).] Once you know the context these phrases all make sense, but not
until then.

It's pretty obvious that our PCT model of the brain is only scratching
the surface. It could be that perceptual functions in the brain are
really ALL reorganization with NO preorganization into levels. That
would imply that the levels which do show up in similar form in
different people are a result of the interaction of the brain with Real
Reality, so there is really some external organization with similar
degrees of freedom. A real comfort to naive realists.

And this would help explain how it is that blind people given a matrix
of vibrations on their backs, mapped from a video camera, start seeing
very much as sighted people do, though with lower resolution. Newborn
ferrets that had their optic nerves surgically relocated to their
auditory cortexes started acting as if they could see, and the auditory
cortex, if I'm recalling this right, began to take on the anatomical
appearance of the visual cortex in normal animals.

And when you use a cane as a blind person does, with your eyes closed,
you start feeling objects with the tip of the cane instead of your hand.
Use a ball-point pen's tip to feel the shape of your coffee cup, eyes
closed. I just tried it, and it works fine. Wierd!

Best,

Bill P.