[From Bill Powers (950501.0715 MDT)]
Bruce Abbott (950430.1710 EST) --
RE: model of salivary conditioning
The exponential decay built into the "CS" perceptual input function
helps to prevent the effect of the CS from summing with the effect
of the "US" (food placed in the mouth). As far as I know, the
evidence would not support such an input function, but I would want
to have a much closer look at actual conditioning data before
passing judgment.
Actually I didn't have any particular kind of conditioning in mind when
I was fiddling with Rick's model. The details of the model obviously
will have to be adjusted to fit the facts (although doing it the other
way is often an attractive alternative when models refuse to behave).
As you say, the exponential decay in the CS input was specifically
intended to keep the CS and US from summing and producing twice as much
error when they both occur. I was treating negative errors as being just
as bad as positive ones. Of course in salivation conditioning, I don't
suppose it matters to the dog if its mouth gets too wet (and with the
Pavlovian fistula in place, the normal control system isn't working
anyway unless there are still other intact salivary glands). The model
is simpler if we're considering only positive values of variables and
only one-way (mouth not too dry) control.
I don't suppose it occurred to Pavlov that if he was collecting the
saliva to measure it, that same saliva couldn't have been performing its
normal function in the mouth, so he wasn't seeing the system in its
normal state.
With more experience the UR tends to occur only near the end of the
CS-US interval.
Whoa! Now we have to put a timer into the model. This probably means we
have to consider at least a two-level model. I haven't done any
experiments with control of perceived time intervals, but it might be
interesting to give it a try with human subjects.
I guess what we need to know is whether the CS and US disturbance
effects summate when the two disturbances overlap. I have no
experience with appetitive classical conditioning; perhaps someone
on CSG-L does and can help us out.
I don't think there can be any general rule here. Consider my falling-
drop thought experiment in a simple form with just one falling drop. If
the subject has to move the bucket back to a starting point between
trials, then in training trials there is a move of a certain size to the
correct position to catch the drop. If a signal occurs just before the
drop falls, this signal has to indicate a move of the same magnitude,
but then when the drop falls ANOTHER move of the same magnitude would
occur and the bucket would move twice as far as it should, missing the
drop. Or, we might say that when both the signal and the falling drop
occur, the summed effect takes the bucket to just the right place -- but
when only one of the events occurs, the bucket would move only half as
far as it needs to. The logic of the model would have to be quite
different from the salivation model. It's essential to get some real
data to see what actually happens, so we can build the right model.
... it would be more accurate if the amount of food placed in the
mouth were specified and a degree of "liquifaction" based on the
food and saliva volume computed. The food would remain in the
mouth but its disturbance value would diminish over time as
salivation continued, being proportional to its current state of
wetness.
Obviously, when you start modeling the phenomena that Pavlov
investigated, you discover all sorts of questions that he didn't try to
answer. If you really want to investigate the salivation control system,
you'll just have to do the experiments all over again, this time looking
at all the relevant data. When you start having to postulate observable
facts in order to make a model work, you know it's time to go back to
the lab and see what _really_ happens. Maybe someone has the data
already, but I wouldn't count on it.
One more comment: the "strength" of response to the CS is always
some fraction of the "strength" of response to the US (at least for
salivary conditioning) Thus, if food is capable of eliciting, say,
30 drops of saliva over the course of 15 seconds, then a fully
conditioned CS would always produce less than 30 drops in the same
time period.
If this rule holds true, it tells us that the US is being treated by the
organism as if it is the CS. But doesn't this leave some quantitative
questions unanswered? What happens if you ring the bell twice as loudly
as usual? What if you reduce the amount of US? Could it be that the size
of the response is dictated by how much action is needed to counteract
the effect of the US?
A big problem is that the response to the US makes sense for the
organism only if the CS occurs. When you terminate presentations of the
CS, pretty soon the US becomes superfluous because there's never any
disturbance of "liquidity". Of course during the "extinction" phase, the
CS produces a response that is considerably larger than what would be
appropriate for the (nonexistent) US. So this raises the question, what
would happen if you halved the size of the US -- would the response to
the CS initially be too large, and slowly decline? These questions don't
arise if you think of CS, US, and UR as "events" that just happen or
don't happen. When you look on them as quantitative processes, a whole
new batch of questions arises, and the only way to find the answers is
to do the experiments and record the right data.
···
-----------------------------------------------------------------------
Bruce Abbott (950430.1715 EST) --
RE: simultaneous vs. sequential
In a system with finite gain, finite lag, and having variables
subject to finite rates of change, the disturbance changes
propagate around the loop at finite speed. The transformations get
complex as the uncancelled remnants of the disturbance signal
recirculate like reflected and re-reflected waves, producing a
complex pattern of interference with their own "reflections" and
the current disturbance signal.
This is close to the right picture. But remember that there are also
integral lags, which mean that the output action of the system may
change by only a small amount in the time it takes for a perturbation to
make its way completely around the loop. A "response" may just be
getting started after the first "wave" is complete, and become a little
larger on the next "wave", and so forth. Likewise, a disturbance may
just have appeared by the time the response starts to oppose it, and may
get only a little larger during the next cycle, and so on, so that both
the disturbance and the output action change on a time-scale measured in
many "loop trip times."
If you look at a time plot of disturbance, error, and output for an
ordinary compensatory tracking task, you see waveforms that change at
the same time. You have to look very closely to see that there is a
small lag involved, because the lag is only a tiny fraction of the time
taken by the waveforms to go from maxima to minima. The large patterns
of change that take a second or two to occur swamp the small lags that
are really there. It's only when you bring in disturbances that jump
instantly from one value to another that you can see the lags clearly --
they are still small, but now there is no large pattern to provide an
appropriate time scale, so you tend to magnify them in your mind, giving
more importance to them than they have when continuous action is going
on.
The method of Laplace transforms, used to solve the differential
equations of control systems, handles time lags in a very simple way. If
the Laplace variable is s, a pure time lag of length t appears in the
equations as a multipler exp(-ts). This automatically takes care of any
number of "reflected and re-reflected waves", correctly giving their
effect summed over all of past time. The remainder of the terms in the
equations remain exactly as they would be if the lag were zero. The lag
does have an effect on the solutions of the equations, but the amount of
this effect declines gracefully to zero as the amount of lag declines to
zero (and exp (-ts) tends toward 1). The effect of assuming zero lag is
simply to produce solutions that are somewhat different from those found
with the lag present. When the lags are small to begin with, ignoring
the lag simply makes the unlagged solutions a more or less close
approximation of the lagged solutions -- there is no great quantitative
difference unless the lags are very long. The solutions are continuous
functions of time in any case.
If you try to reason out the effect of lags by tracing events around and
around the loop, you will just create confusion and complexity. The
actual effect is very simple, and can be compensated for by putting a
single integration in the loop to slow the rate of change of some
variable. That is the secret of our "slowing factor" and why it has to
be used in our simulations to prevent artificial oscillations caused by
our minimum one-iteration lag. If you put a perceptual lag into the
system which involves several iterations, you simply have to increase
the time-constant of the integration (make it slower) to prevent
oscillations. This applies in the real system as well as in the model,
and this is why our models with a single integration in them work so
well. What the integrator does is to assure that no variable in the real
or simulated loop is going to change too much during one loop trip time.
However, these circulating "remnants" appear at any point in the
loop merely as indistinguishable components of the current value of
each variable; from the point of view of the control system, there
are only current values, which result from current values and in
turn produce current values, both of themselves and of other
variables (as each function currently "sees" them).
That's a very clear description. These "current values" may include the
values of integrals and derivatives, but they are always current values.
What the time-lags do is modify the current values so they are a little
different from what they would be without the lags. But the whole system
functions strictly in present time.
When you see truly sequential behavior, you'll know it. It's possible
for lags to be present that are far from negligible, and the environment
can impose conditions such that one act must be complete before the next
one can even start. I suspect, however, that there are underlying
continuous variables even in such situations, and that we will find
integrators in the control systems with long enough time constants to
compensate for the destabilizing effects of long lags.
This is why, I think, that in tasks like bowling (where you don't even
have a ball to roll until it has come back from the far end of the
lane), we still speak in terms of continuous variables: I'm trying to
get my curve adjusted to come up higher on the head pin. We perceive
even the most discontinuous control actions as if there were an
underlying continuum of quantitative variation, and we make adjustments
accordingly. It's as though we adopt a subjective time-scale such that
the time between successive values of a variable seems to be negligible,
and the variable seems to change along a continuum.
-----------------------------------------------------------------------
Best,
Bill P.