Use of classical conditioning to study mental development

[From Rick Marken (2011.02.20.1740 PST)]

Bill Powers (2011.02.20.1153 MST)–

BA: … PCT offers a different
kind of theory, one based on physical
mechanisms. It is supposed to explain human and animal behavior, and
that

would include the types of behavior that emerge during classical

conditioning. If the present framework of PCT is sufficient to explain
all

behavior (at least in principle), then one should be able to develop
models

that exhibit the phenomena of classical conditioning that fit within
that

framework. If not, then this will show that there is something missing
in

the PCT framework and we (all those working to develop PCT) will have
to

search for the missing pieces of the puzzle. That’s how science
progresses.

BP: I agree. We should not just abandon classical conditioning, because
there is some sort of phenomenon there and we owe its discoverers the
courtesy of dealing with it. However, I think we are going to end up
abandoning the term conditioning because the phenomenon can be shown to
be simply a case of learning a control system, so there is no mysterious
special process involved.

Weren’t you developing a model of classical conditioning, Bill? Did anything ever come of that? If so, could you post a description of the model.

Best

Rick

···


Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

There was a little mistake in initials. This text was mine although Bruce
marked it as his. I believe it was accidentally.

BH instead BA: I think we should ask why should organism care so much about
external requirements if the problem is inside organism (keeping intrinsic
variables in limits). Keeping outside requirements (variables) in certain
limits is probably necessary only then when some control systems can't keep
intrinsic variables within survivable limits.

BP: Keeping intrinsic variables within survivable limits requires
controlling physical variables in the environment. If the internal variables
would stay in their optimum states all by themselves, no behavior would be
required until the organism decided to try changing the internal states. But
entropy always increases and dynamic processes need power and materials that
can be obtained only from the environment. And the environment keeps
generating disturbances.

BH : My assumption was that not all intrinsic variables require controlling
physical variables in the external environment all the time.

While sleeping some behavioral control systems in hierarchy are probably
"shoot down" (reduced or lacking consciousness, relatively suspended sensory
activity, and inactivity of nearly all voluntary muscles).
Because of entropy, metabolism during sleeping actually decreases only about
5�15%. Data differ. Maybe because they were taken during different phases of
sleep. Anyway, body temperature must be maintained. That's why I think that
some internal control systems are "running" all the time, and behavioral
only sometime if we neglect "tonus".

If the temperature of the external environment is high enough, decreasing
entropy, who would move :)). "Just joking".

Best,

Boris

···

On Sun, 20 Feb 2011 15:12:41 -0700, Bill Powers <powers_w@FRONTIER.NET> wrote:

[From Bill Powers (2011.02.20.1153 MST)]

Bruce Abbott (2011.02.17.2250 EST)

[From Bill Powers (2011.02.20.0705 MST)]

Rick Marken (2011.02.20.1740 PST) --

Weren't you developing a model of classical conditioning, Bill? Did anything ever come of that? If so, could you post a description of the model.

It's on hold. I'm overcommitted on error signals and have to cut back on projects. Anyone who wishes can take it over.

The basic idea is fairly simple -- it is to show that classical conditioning, just like operant conditioning, is simply one rather distorted view of the same process of reorganization by which we learn every skill we have. Hungry animals learn what they have to do to feed themselves, cold ones what they have to do to keep warm, and so on. What we learn through reorganization does not have to make sense. If pressing a lever five times causes a pellet of food to drop into a dish, and we're hungry for a pellet, we'll press the lever five times once we discover that we can get food that way. We don't need to know why the local universe is organized that way. If shivering when the skin feels cool keeps us warm, we will shiver. If pulling a hand away from a thorn sticking into our paw keeps the pain down, we will learn to pull the paw back when the tip of the thorn punctures the skin. If pulling the paw back when a whistle toots reduces the pain even more and faster, we learn to pull the paw back when the whistle toots.

Note that the pulling back is what we learn to do when we feel the sensation of something sharp against the skin -- a little point of sensation. The reason we learn that is that this sensation soon causes another one that we really don't like; if we have language we call it pain. The sensation and the pain are two different things, with the pain being the real reason for acting at all. The sensation is just information.

We have probably known about pulling the paw back from the sensation of a sharp object for a long time -- millions of years. A sketch of that control system is passed on along with our eye color. We have probably known about pulling the paw back when we hear a strange sound also for a long time, but that one is pretty unreliable and we keep forgetting that one -- it doesn't really work even if it seems to work for a while. It's just a useless superstition. Once in a while, some wise guy sets up the local environment so the toot (or the rumbling sound) is always (for a while) followed by pain or loss unless we do something or other when we hear it, so when we stumble across the secret, we dutifully do the something or other and the pain goes away or the loss is replaced until the wise guy gets bored with controlling us and goes away. Then we can stop, once we're sure the rule is no longer in effect. If we can, we get rid of the wise guy, but that can be hard to do if he's too big, strong, or smart. Or supernatural, omnipotent, and invisible.

When I say "we", I'm just referring to us living things. When I say "learn" I mean the way we living things can reorganize to get control over the local environment -- some a lot better than others. And by "reorganize" I mean altering what matters to us least in order to control what matters to us most. What matters to us least are items like size, eye or skin color, shape, number of limbs, diet, and species. What matter most are the things those of us who can talk call intrinsic or essential variables experienced as hunger, thirst, suffocation, nausea, confusion, and pain among other nasty feelings that tell us something is wrong.

SO:

If you can set up a model environment with a number of variables that can be sensed and acted upon, and make certain states of some of them important by giving them reference conditions, and supply a random reoganizing output and a few other things, you can model how behavior comes to control perceptions of all kinds. You can mess with the model environment yourself, creating arbitrary relationships among the variables, and demonstrate operant, classical, and any other kinds of conditioning you find it pleasing to invent.

That's the general idea. Now all you have to do is implement it in a model. This will put the entire subject into the proper perspective, after which we can let conditioning go the way of phlogiston. Good luck.

Best,

Bill P.

[From Bill Powers (2011.02.20.0815 MST)]

BH: There was a little mistake in initials. This text was mine although Bruce
marked it as his. I believe it was accidentally.

BP: I love that "I believe it was accidentally." I hope you will extend that somewhat doubtful pardon to me, as it was I who -- unintentionally -- mislabeled that paragraph (and at least one other). But thanks for the nice illustration of the difference between "believe" and "know."

BH : My assumption was that not all intrinsic variables require controlling
physical variables in the external environment all the time.

BP:I don't know how literally you mean "all the time." Clearly, you have to control the intake of oxygen in certain ways to keep breathing, but you can do without oxygen intake for 30 seconds or so before your body begins to complain. Even under normal conditions, you do without oxygen intake while you are exhaling, which happens quite often. And you control the environment to obtain and consume food only at much longer intervals -- a few hours or, if you're unlucky, days.

BH: While sleeping some behavioral control systems in hierarchy are probably
"shut down" (reduced or lacking consciousness, relatively suspended sensory
activity, and inactivity of nearly all voluntary muscles).
Because of entropy, metabolism during sleeping actually decreases only about
5�15%. Data differ. Maybe because they were taken during different phases of
sleep. Anyway, body temperature must be maintained. That's why I think that
some internal control systems are "running" all the time, and behavioral
only sometime if we neglect "tonus".

BP: For any control system, everything else is in its environment. But the important factor here is the time scale. Reorganization is very slow, and the more complex the level of control being reorganized, the slower it has to be. You do not learn a new skill in one day; for some skills, like learning the calculus, in one year. Babies kick off their blankets even in a cool room. Adults have learned to waken and pull the blankets up.

Reorganization has to be very slow because it has to deal only with average values of perceptions, not every momentary change. Reorganizing too fast breaks down organization instead of building it up. Think of the E. coli model. Most of the time, E. coli is swimming steadily in a constant direction, changing its position in X, Y, and Z. When that gradual change of position starts to make the concentration of an attractant start falling insgtead of rising, it tumbles for a few seconds and then starts swimming in a new direction so X, Y, and Z are now changing in different proportions. If the new direction makes the rate of change positive again, E. coli keep on swimming. But if E. coli just keeps tumbling, it will not make any progress.

X, Y, and Z for E. coli translate into slow changes in three different parameters of a control system (or one parameter in three different control systems). As long as the total error (absolute value or square) is decreasing, the slow change in the parameters continues, all dimensions of change slowing down as some error signal gets smaller. When a minimum of error is reached and the absolute error starts to increase again, a tumble occurs: the rates of change of all the parameters are altered at random, and the slow changes in values recommence, but now in different proportions. A new direction in parameter space.

So you can see that while many tumbles in a row might occur, once most of the parameters are changing so the total error is decreasing, the gradual change might continue for a relatively long time before another change of direction or tumble occurs. During that time, many parameters are slowly changing, and their rate of change is decreasing as the overall error decreases. When all the different error signals have become very small, the changes become very slow and tiny reorganizations keep the system close to the optimum state, the lowest possible sum of absolute errors.

All during this process, which can take anywhere from minutes to years, life goes on. The control systems continue to control, although the way they are controlling gradually drifts. In LCS3 you can see this happening in demo 8-1, where 14 control systems are being simultaneously reorganized to gain control over 14 joints in an arm and hand, with the reorganization process being based on the mean square of the error signals in all 14 system. "Tumbles" alter the rates of change of output weights in all connections from each system to all 14 of the joint angles, a total of 196 weights that are slowly changing, or suddenly tumbling so all 196 rates of change of the parameters are altered at random somewhere between -1 and 1. A chart shows the total squared error fluctuating but gradually decreasing -- in this demonstration, it takes about 20 minutes for this total intrinsic error to reach a minimum, with movements about the joints occurring continuously all during the approach to the minimum. Eventually, each control system ends up with its outputs connected through a large weight only to the joint that it senses (which in this demo is always the same).

Use this demo to see the relationship between ongoing behavior and reorganization.

If the temperature of the external environment is high enough, decreasing
entropy, who would move :)). "Just joking".

Unfortunately, the temperature of the body must be higher than the temperature of the surroundings to avoid a fatal accumulation of heat in the body. I once built a calorimeter containing a heater which kept the environment inside a small outer container at the same temperature as that of a smaller inner container, so the inner container could lose or gain very little heat from outside it. This was part of a device for measuring the intensity of radiation inside a Cobalt-60 source of gamma radiation. It worked by measuring the power dissipated in the heater for the outer container (the room temperature was held constant). Removed from the radiation, the temperature of the water in the calorimeter would remain constant (within the limits of measurement) for half an hour. Nice to remember that!

Best,

Bill P.

···

At 05:33 AM 2/21/2011 -0600, Boris Hartman wrote:

[From Rick Marken (2011.02.13.0920)]

  Adam Matic --

I would like an opinion about a research method involving classical
conditioning.

That's what I got plenty of.. opinions;-)

I'll get right to the point. If child can perceive a certain 'stimulus' (say
a sound of a bell) then it could be 'conditioned' to it. For example,
if some air is puffed to the retina, the eye closes. If the puff of air
gets paired a dozen times with the sound of a bell, the child will close the
eye after hearing the bell. That's how we can know that the child can
perceive the sound of a bell.

If the hierarchy of levels is developed in leaps, then in the first period
of development, the child�can not perceive more complex stimuli and
it can not be conditioned to them,

Actually, it's not "stimuli" that are perceived; perceptions are
constructions (in PCT anyway); functions of physical variables. More
complex perceptions are more complex functions of these physical
variables. So if the physical variables are x1, x2...xn, then the
simplest perceptions are like p = k*x1; more complex perceptions are
like p = k1x1+k2x2; still more complex are p = kdx1/dx2. etc. It's
the complexity of the _function_ of physical variables that defines
the complexity of a perception, in PCT anyway.

Conditioning itself probably involves the ability to perceive
relationships between physical variables (and other perceptions of
physical variables), such as the relationship between the the bell and
the air puff (and the reaction to that disturbance). So conditioning
itself probably involved the ability to compute fairly complex
perceptions.

but in every next level
the child can be conditioned to more and more complex stimuli (sensations,
transitions and so on).

Yes, the child must be able to perceive the CS in order to make that
part of the control action that is what I think a "conditioned
response" is. Of course, you would have to design the conditioning
situation so that you were sure it was the more complex aspect of the
stimulus that was the CS and not just the occurrence of the CS (a low
complexity perception) that was doing it. So, for example, if you
wanted to see if the child could perceive a decreasing tonal sequence
(transition) then you might play different tonal sequences at
different times but only the decreasing sequence prior to the US and
see if that becomes a CS.

The child could also discriminate between complex stimuli only if the needed
level of hierarchy is developed.

I agree.

So, that way, conditioning could be used to prove the existence of hierarchy
in early development of a brain
(perhaps best to start with dogs or cats).
Should I work more on the idea or is it flawed in some fundamental way?

I think your idea is definitely on the right track but my inclination
is to think that the use of conditioning as a method of detecting
levels of control is way too complex. I agree that you need a
non-verbal approach method for testing what kids can perceive at
different points in their development, but I think there are far less
time consuming ways to achieve this using non-verbal version of the
Test for the Controlled Variable. What you want to see is what _kinds_
of variables a child can perceive (and, thus, control) at different
points in their development. The trick here would be figuring out how
to give infants (with very limited output capabilities) the ability to
control certain perceptions (if they can perceive them). I think some
of Piaget's tasks could be adapted to this goal more readily than
Pavlov's. There is also work by the Plooijs (if Dag Forrsell is
listening in he can point you to it) that is directly relevant here. I
think your research goal (determining how perceptual -- and thus
control -- ability develops) is a great one. But I think classical
conditioning is probably a too cumbersome approach. My suggestion is
to look to Plooij and Piaget rather than Pavlov for inspiration. But
keep up the good work!!

Best

Rick

···

--
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com