[From Rick Marken (951205.1600)]
Bruce Abbott (951205.1320 EST) --
I think I do get the picture
Well...er....
I am maintaining that if the reinforcer did not tend to reduce error, it
would not serve as a reinforcer.
A reinforcer is not an agent; it can't "tend to reduce error". However, error
will be reduced if the variable called the "reinforcer" is a controlled
variable and is part of a dynamically stable, high gain negative feedback
loop. Error is the difference between reference and controlled variable (r-
p). What we call a reinforcement is the environmental correlate of the
controlled variable (the controlled quantity); so the word "reinforcer" really
refers to a controlled quantity.
the system will reach an equilibrium in which the reduction in error
supplied by the reinforcer will just balance the increase in error supplied
by the disturbance
Again, an external variable (reinforcer) isn't an agent; it doesn't "supply"
error reduction; error is reduced when the controlled quantity (actually the
perception thereof) is brought to its reference state.
the reinforcer continues to supply an error-reducing effect on the
controlled variable each time it occurs as a result of control-system
action. If it ceased to provide this service, it would cease to function as
a reinforcer.
It looks like you are saying:
(1) re = k1o (2) cv = k2re (3) e = r -cv
where re = reinforcer, cv = controlled variable, e is error and r is the
reference for the cv, which is here viewed as the perceptual representation
of re. So
e = (r - k2re)
Error, e, is the difference between the reference and reinforcer; the "error
reducing effect" of reinforcer works like this: as re approaches r, error is
reduced. This presumably happens because error drives output and the effect
of o on re is such that o brings re toward r. There are only two ways I can
make sense of the idea that re could cease to provide a "supply of error
reducing effect": 1) the effect of o on re might no longer be appropriate
(as, for example, when the the lever connection to the dispenser breaks; k1
goes to 0 so o no longer has an effect on re 2) the system might no longer be
able to perceive re (as, for example, when the pellets are dispensed into a
hidden recepticle).
So there really is no "error reducing effect" of re; error reduction is what
happens when re is under control.
Now, if a disturbance to the system could push it into an opposite error
(controlled variable above reference rather than below it), the same
"reinforcer" would now contribute to an exacerbation of the error rather
than to its reduction.
Is it really "the same" reinforcer that is doing this? A disturbance (such as
independent addition or deletion of pellets from the hopper) changes re (just
as o changes re). That is, with variable disturbances present, re = k1o +
k3d. So the value of re (and, hence, of the controlled percppetual variable)
is always changing as a result of the organism's outputs and independent
environmental disturbances. The re that produces "position" error cannot
possibly be the same re that produces "an opposite error".
To determine whether a given consequence of behavior serves under present
conditions as a reinforcer, one must assess whether the intensity of a given
action is higher when the action produces the putative reinforcer than when
the action does not produce it.
How do you measure the intensity of the action that produces the putative
reinforcer? Does the intensity of action relate to the amount of reinforcer
delivered? If so, this would mean that the organism is in a positive feedback
loop. If it is (according to reinforcement theory) then it would be very easy
to reject this model; just show that organisms CAN control variables that are
considered reinforcers. If organisms can control, say, the rate of food input
(under disturbance) they are definitely not in a positive feedback loop with
respect to reinforcers.
This is very different from comparing the intensity of action before and
after a control system has adjusted to counter a disturbance, which is the
comparison you were making. This is why I said that we seemed to be on
different wavelengths.
I think we'd be lucky if we were on different wavelengths; I think we're on
different systems concepts (basically, different planets).
Best
Rick