TC:I do think we need to understand better how a control system (CS) learns
how to control its output to obtain/maintain the input demanded by its
reference signal, and I've got a suggestion for that, below. But being in a
hierarchy, a CS also needs to learn how to help the CS which addresses it
obtain/maintain the input demanded by that CS's reference signal. Otherwise
you don't have a hierarchy, actually, you just have a stack of CSes each
doing its own thing when addressed by the CS above it.
>
>RM:I have used a very simply version of the basic reorganization model to
>account to the behavior in a control task where the consequences of
>action are random. You can see the data and behavior of the model at:
>
> Selection of Consequences
>
>The "Subject" button collects data from you; the "Control" button runs
>an E. coli reorganization model and the "Reinforcement" button runs a
>reinforcement model. The "Control" model generally does better than
>the "Reinforcement" model, in the sense that it, like the "Subject" is
>able to get the dot to the target consistently.
TC:That's an excellent demo of the superiority of PCT over "learning theory"
(as we used to call it). I've admired it before. But it doesn't show how a
CS works, nor need it. And it doesn't include learning, unless you count the
learning of the person working the space bar. E. coli, apparently, doesn't
learn.
>
>RM:I'd like to know specifically what kind of behavior is it that your
>model is designed to handle. When I know that I'll be more interested
>in trying to understand how your model works.
>
TC: I hope I've piqued your interest.
=================================================
>
>Erling Jorgensen (2000.08.27 1035 EDT)
>
>>Ted Cloak (2010.08.26.1700 MTD)
>
>>TC: I wish other CSGNeters would chime in, too.
>
>EJ: Allow me to briefly chime, to clarify a background issue as I, too, try
>to understand your model better.
>
>>TC: The mechanism proposed in my diagram is not about how CSB learns to
>control the perceptions coming into its Input Function. It is, rather,
about
>how CSB learns how to control the perceptions coming into CSA's Input
>Function so that CSA will stop addressing CSB.
>
>EJ: If CSB is guided by the CSA address to providing a perception that
>the CSA Input Function needs, I'm not sure that CSA addressing stops once
>the CSA perception is brought to sufficient control.
TC:Sure it stops, because the error signal from CSA's Comparator to CSA's
Output Function stops, because CSA's perception approximates CSA's reference
signal, which is implied by your expression "brought to sufficient control".
Or do I misunderstand your question?
>To stop addressing
>CSB would be to lose the contribution of the CSB perception, which would
>seem to make control worse for CSA.
TC:Excellent point. I started with a simple binary, on-off, idea of signals.
With that, once CSA's perception is brought to sufficient control, it stays
that way. That's not realistic, of course; indeed it's not PCT, and it's not
how Sidi's nervous system works, in all probability.
>
>EJ: Once CSA control is adequately achieved, you would want to stop
>_changing_ the address signal to CSB, essentially saying "That's the one;
>you got it!" I think that's what might be achieved by having the CSA
>_error signal_ decide how much drift in the address signaling is desired,
>as Bill was proposing.
TC:Yes, having the error signal work through the address signal to manage
the RVG, perhaps by varying the frequency of changes in the reference
signal, might do the trick.
>
>EJ: I appreciate your efforts to try to bring memory & references-as-
>address-signals more explicitly into the PCT model.
>
TC:You're welcome. But references-as-address-signals? That's not my
intention.
=====================================================
>Bill Powers (2010.08.27.0835 MDT)
>
>Ted Cloak (2010.08.26.1700 MTD) --
>
>>TC: No, the address signal from CSA is, indeed, the trigger and sustainer
>>for the RVG's action, by closing the NO relay and keeping it closed.
>
>BP: OK, so we have an address signal that not only can specify an
>address, but can cause a relay to close. What if it specifies an
>address of 00000000? In binary, that's as good an address as any
>other. Does that make the relay close?
TC:Probably not. I now realize that the NO relay has to be a lot more
sophisticated, modifying the frequency RVG's actions according to the
intensity, or some other feature, of the incoming address signal. See above
discussion with Erling.
>
>BP:There's something very odd going on in your modeling process that's
>going to take me a while to figure out, if I can do it at all.
>
>>TC: In the first place, why should CSB be privy to what's going on
>>in CSA's inner workings?
>
>BP: But it's not. It has no perception of signals coming into it from
>above, or of anything not represented in its own perceptual signal,
>which originates in lower systems. What it "knows" is only what it
>perceives, and that is represented by the perceptual signal.
TC:Okay, you interpret "be privy to" as "have perception of". Let's try
this: I think it's better to have all communication between CSes in a
hierarchy be through as few interfaces as possible. Having a direct line
from the error signal of one CS to the Memory of another violates that
principle. Of course, who knows how evolution will have rigged such a thing?
>
>>TC: In the second place, for all we know, CSA's Output Function may
>>have elected not to address CSB despite receiving the error signal from
>>CSA's Comparator; it might address some other CS at CSB's level instead,
in
>>which case we wouldn't want the reference signal currently in Register r
of
>>CSB's Memory to be changed.
>
>BP: Well, either that reference signal would be zero, or some other
>system or set of systems at level A would be determining it.
>Reference signals don't just sit where you put them like writing on a
>blackboard. They're continuous variables reflecting the current
>outputs of higher systems that are contributing changing amounts to them.
TC:I agree, providing you mean "address signal" rather than "reference
signal". You refer next to "the recorded perceptual signal that is stored in
that location". In my usage, a recorded perceptual signal that is stored in
Memory is a reference signal stored in a register.
>
>BP:I think you're confusing the address signal, which selects a storage
>location, with the value of the recorded perceptual signal that is
>stored in that location.
TC: No, I'm not. Please look at my diagram.
>BP:The address signal just says "play back
>whatever is in this location." It can't also specify what particular
>waveforms are to be stored in that location.
TC:No, but it can regulate random change in [one of] those waveforms via
regulating the RVG.
>
>>TC: The mechanism proposed in my diagram is not about how CSB learns to
>>control the perceptions coming into its Input Function. It is, rather,
about
>>how CSB learns how to control the perceptions coming into CSA's Input
>>Function so that CSA will stop addressing CSB.
>
>BP:But how does CSB even know that CSA exists?
TC:It doesn't. It only "knows" (my usage, not yours, hence quotes) that its
Register r and its RVG are receiving a signal of a certain value.
>BP:And why should a lower
>system be able to tell a higher one to stop addressing it? In PCT, it
>doesn't. It has control only over its own perceptions as they relate
>to the reference signal it receives (which it also does not perceive
>since it comes into the comparator, a simple subtractor, and not into
>its input function). CSB can't decide what to send on up to CSA and
>what not to send, or even know what effects it is having on CSA.
TC:I defy you to find in my diagram any indication that CSB directly
influences CSA, let alone controls CSA. The only way CSB influences CSA's
input perception (or anything else to do with CSA) is through the long, long
loop down the hierarchy, through the environment, and back up through a
series of Input Functions. My model is intended to show how CSB learns how
to do that better and faster.
>
>Let me put that another way. Of course it CAN do such things, but in
>order to do them it will need abilities that have to be put into the
>model. What will you have to put into CSB to enable it to do the
>things you describe? What you're describing goes far beyond anything
>in a PCT model, and I have no idea how we would go about simulating
>it to show that it would actually behave in the way you describe.
TC:I hope I answered that above.
>
>>TC: More technically, it is about
>>how CSB continually adjusts the reference standard in its memory until, as
>>it attempts to control its perception to that standard, the perceptual
>>signals coming into _CSA's_ Input Function approximate CSA's reference
>>standard, so CSA ceases to send address signals to CSB.
>
>BP: Now you have CSB adjusting its own reference standards. What part
>of CSB does that, and how does it act so the adjustment is what is
>needed to accomplish the result you describe?
TC:That is done by the RVG.
>BP:How does CSB know when
>CSA's perception is approaching its reference standards?
TC:Now using "know" in my loose sense, it "knows" because the address signal
eases or stops. That completes the long, long loop.
>BP:How can CSA
>act if CSB is adjusting its own reference standards instead of CSA
>doing so? You've taken away CSA's only means of controlling its own
>perceptions, which is to adjust the reference standards for CSA.
TC:CSA adjusts CSB's reference standards by the address signal it sends,
which varies the behavior of CSB's RVG.
>
>>TC: The NC relay is superfluous only if the frequency with which the RVG
>>changes the contents of Register r is slower than the time it takes for
the
>>entire hierarchy from CSB on down to operate and return a perceptual
signal
>>to CSA. I assume that the RVG would go a lot faster than that, since much
of
>>the time the new reference signal would approximate the perceptual signal
>>from the Input Function closely enough that no error signal would be going
>>out.
>
>BP:You're making claims about how this new organization would behave, so
>now the burden is on you to show that it would actually behave this
>way. I'm kind of relieved to turn that over to you, since I have no
>idea how such things could be done, or whether they should be done.
>How do you know that the organization you describe would, if
>simulated as accurately as possible, generate the results you
>describe just above?
TC:I'd appreciate it if you'd stick with this until you understand what
claims I am actually making. I admit, however, that I don't know that this
organizational scheme would really work, when instantiated in the head of a
kitten with millions of neurons in hundreds of thousands of control system
hierarchies all going at once. Would it cause a gradual convergence on more
and more expert behavior? I don't know, but I do know that there is such a
convergence, and I'm as certain as I can be that it is based on a process of
blind variation and selective retention.
>
>> >BP: In Manchester, Yu Li is developing a program to explore multi-level,
>>multi-system reorganization, for a graduate student working on his
doctoral
>>degree. I think he will end up with something close to what you are
>>describing, if you accept my small modifications.
>>
>>TC: Fascinating. Recursion will no doubt be required. I hope Yu Li is
>>reading this and will chime in.
>
>BP:Recursion? Do you mean a function that calls itself as a subroutine,
such as
>sin(sin(sin ...(x))? Feedback loops are not recursions. There are no
>recursions in my PCT models.
>
TC:OK, I was being a trifle facetious there. I didn't mean that hPCT works
by recursion, that would be absurd. I meant that a computer program
simulating hPCT might utilize recursion to avoid writing the same code over
and over for each level simulated. Perhaps iteration would do the trick.
TC:Next question: How does a CS learn to get the perceptions it needs to
approximate its reference signals in its Comparator? Does it matter, in a
hierarchy, whether it does get that approximation, unless it's the top CS in
the hierarchy? Would an RPG in the Output Function which variates the
outgoing address signals do the job?
Best to all
Ted