# Task for comparing PCT and reinforcement theory

[From Bill Powers (950624.0620 MDT)]

Bruce Abbott and Rick Marken --

It occurs to me that we've been missing our best bet for a task to
compare PCT and reinforcement theory: a simple compensatory tracking
task. Let's set up a somewhat novel version in which cursor position is
set by

C := C + k*(H + D)

We know that with a nice slow disturbance, the subject will be able to
keep the cursor very close to, although not exactly on, a stationary
target. The errors, however, will show a very low correlation with the
handle movements. We can use a running average of error-squared, shown
as 100 minus the average error, as the "reinforcer." The cursor position
relative to the target, presumably, would be the discriminative
stimulus.

We know what the general PCT prediction is: handle position equal and
opposite to disturbance, cursor remains near target. The reinforcement
explanation would have to be something like probability of rightward
handle movement given cursor left or probability of leftward handle
movement given cursor right increases when the score moves toward 100,
decreases when it moves away from 100. Bruce can work out the details.

What will make the difference is that with an easy disturbance the
person will be controlling with an error near the noise level. The PCT
model will predict control whether the reinforcer is present or not --
or even if it varies randomly (lying about the actual error). An actual
working model using reinforcement theory will control very poorly
because of the low correlations, even with a truthful reinforcer, while
the PCT model will control as well as the subject does. I think.

This is a velocity-control task, so the PCT model just needs a
proportional gain in the output function; a very simple model with one
parameter will do well enough.

dc := dc + k*(H + D)*dt

c := c + dc*dt

This will require a first derivative in the output function of the
control system.

ยทยทยท

-------------------------------------------------------------------
Best,

Bill P.