[From Bruce Abbott (2017.11.20.0955 EST)]
I'm interested in joining the LWG; however, I may not be in a position to
contribute much at present.
Rupert Young (2017.11.20 10.15)]
(Martin Taylor 2017.11.19.14.14]
Therefore, I would like to set up a learning working group for those
interested in taking part by building models, contributing expertise
or just following. If anyone is interested let me know and we'll
either set up a separate email group or tag subjects on csgnet with
"LWG".
I'd certainly be interested, but I'm not in much of a position to
contribute anything other than ideas.
I think the first question to ask would be what the intrinsic
variables might be, because the robot won't learn if it doesn't have
any indication that its learning is of some benefit. I don't think
simple quality of control in itself is sufficient, but it's probably
necessary.
RY: Sure. But what is intrinsic error; is it not error from another (higher)
system?
BA: In his book, Design for a Brain, W. Ross Ashby (1954) suggested that a
system could organize itself based on error in what he called its "essential
variables" -- those that must be kept within certain limits if the organism
is to survive. Bill Powers borrowed this idea for PCT, although he renamed
essential variables as "intrinsic" variables. In PCT, the entire perceptual
hierarchy is supposed to develop under the supervision of these intrinsic
variables, each succeeding level providing better control to help prevent
those intrinsic variables from moving beyond survivable limits.
BA: So, no, intrinsic error is not error from another, higher system, it is
error in the regulation of intrinsic variables. To the extent that
higher-level systems fail to keep the error in intrinsic variables low,
those higher-level systems will be targeted for reorganization.
BA: From this perspective, how does the reorganizing system "know" which
system to reorganize? All the demos of reorganization of which I am aware
simply target whatever system is afflicted by persistent error, with the
rate of reorganization being made proportional to the size of that
persistent error. I suppose it was assumed that persistent error in any
control system within the hierarchy would threaten to increase error in one
or more intrinsic variables.
One interesting problem would be how you would set up an environment
with enough complexity to justify learning more than simply
controlling sensory variables. Either you use a real world with real
robots, or you put most of the effort into building the world in which
the simulation must work.
RY: Yep, one thing at a time. Simply learning sensory variables would be a
good start, as that is further than we have got at the moment. For
environments, I think, there are plenty of existing simulated environments
that would be sufficient for our purposes for quite some time. I am also
thinking of real world images as suitable environments for learning
perceptual functions.
Maybe the answer is not to start with something like the mountain car
problem, which needs the learner to have learned to distinguish
objects, perceive properties such as "impenetrable" or "can
slide/roll" "weight" and perform actions such as applying force to one
object but not another. What intrinsic variables would doing those
things (and solving the mountain car problem) help to control? I think
the baby needs to start with something simpler, and with a definition
of intrinsic variables.
RY: I am open to suggestions, but I think the mountain car problem can be
resolved with continuous variables without the knowledge you state. For
example, it is not necessary to distinguish objects as long as the system is
provided with a position signal. The "intrinsic" variable to be controlled
would be the position related to the target.
BA: Position in space relative to target is not likely to be an intrinsic
variable in a biological system.
But then again, maybe you aren't talking about a "baby" but a
well-formed hierarchy that has to learn something new in order to keep
its intrinsic variables in good condition.
RY: Initially I'd want to start with right at the beginning with
understanding the weight adjustments in Bill's arm model and see how that
could be applied to multi-variate output functions (leaky
integrator) and to perceptual functions with multiple inputs.
BA: Demo 8-2 (Coordination) might be a better place to start. It includes
three systems whose output weights are adjusted by reorganization, and
displays both the output weights and the variables by which those weights
are multiplied during reorganization. The latter determine the direction in
which the output weights change as well as the sizes of those changes, which
vary as reorganization proceeds. In addition, the demo illustrates both
individual and global reorganization strategies. Bill provides an excellent
description of the algorithm in LCS III, Chapter 8.
Bruce