[From Bill Powers (960804.0700 MDT)]
Hans Blom, 960803 --
Assume that the controller lives in a world where perception and
action are related according to
p := x + y * a
where p is the perception, a is the action, and x and y are UNKNOWN
constants. The goals of the MCT controller are to control, that is
to maintain p at a certain reference value r, AND TO GET TO KNOW
THE VALUES OF THE UNKNOWNS x and y.
....
Now, is it necessary to get to know the values of x and y? No. PCT
shows that one can bring p to r through a method that could be
called "climbing the gradient". For example, if r = 20, x = 10, and
y = 2, all one has to do is to somehow arrive at the value a = 5 to
bring p to the value of r.
I think that rather than saying "one" has to arrive at the value a = 5
you should say that _the system being modeled_ has to arrive at the
value a = 5. The whole question is _how the system arrives at this
value._
Your proposal for how the system arrives at this value is as follows:
Would it be nice to know the values of x and y? Yes. If we know
them and are presented with a new task (a different value of r), we
will immediately find the correct solution rather than having to
grope around. The above formula gives the solution:
So what is the advantage of "knowledge"? That we immediately arrive
at the correct solution rather than having to rely on a slow
gradient climb.
It is not "we" who must immediately arrive at the correct solution; it
is the system being modeled that must do so. The system being modeled
must first contain an internal representation of the formula p := x + y
* a. Then it must substitute r for p to get r = x + a*y. Then it must
subtract y from both sides of the equation to get r - x = a*y, and
finally it must divide both sides of the equation by y to get
a = (r-x)/y.
Then, given a numerical value of r and values for x and y, it must
subtract x from r, divide the result by y, and produce an action on the
world equal to the final result.
How do you suggest it does all these things?
···
a := (r - x) / y
-------------------------------------------
Your description of how the PCT model would arrive at p = r is
incorrect. The PCT model would be organized so that
1. a = k*integral(r - p)*dt
This would bring p to any value of r from any starting value p0,
approximately along the curve
p = p0 + (r - p0)*(1 - exp(-k*x*t))
In the given universe, all that has to be determined for the PCT model
is a workable value of k. The system can be organized to find such a
value by sensing the error (r - p) and minimizing it by varying k
(through some hill-climbing process, perhaps). It never has to know the
values of x and y (neither constant appears in equation 1). Once a value
of k has been found, the same organization will bring p to any given
value of r without any further changes in k.
I suggest that the comparison process could be accomplished in a single
neuron by making the signal frequency representing p inhibitory, and
that representing r excitatory, and feeding both signals to inputs of a
neuron. The output of the neuron, the error signal, would be
proportional in frequency to the excess of excitation over inhibition.
The error signal could then enter a neuron with a long internal time
constant, such that given a steady input frequency, the output frequency
would increase with time to some large asymptote frequency. The action a
of the system would be proportional to this output signal. The rate of
integration is determined by the sensitivity of the synaptic connection
to the output neuron, and that is what is reorganized during the hill-
climbing "learning" process to determine k.
Neither the subtraction nor the integration is exact, but this system is
feasible neurologically and it will produce the desired result within
some small percentage of error over a range of values of x and y, and
over a range of changes of k, the sensitivity of the output actuator.
So clearly, if r is changed, the PCT system does not have to "grope
around" to produce the corresponding new value of p. As soon as r
changes, the system alters p in the right direction so it again
approaches r. In the linear world you defined, the time required to do
this can be made as small as desired by making k as large as necessary.
So how does your model accomplish all the operations it needs to carry
out?
---------------------------------------------------------------------
Best,
Bill P.