Testing for Imagination in Experiments

[From Rick Marken (2010.09.07.0940)]

I finished an initial version of the simulation of the Marken and Martin models of a yes/no detection task. It still could use some work but I’ll distribute it now so Martin (and any other interested parties) can see how it works. The model parameters of the model are set but putting 1s or 0s in the appropriate cells that are yellow. You can also set the length of time the model spends in imagination model bu entering a number (from 0 to 30) in the appropriate cell. The spreadsheet comes set to run 100 trials of a detection experiment using Martin’s model. Each trial lasts 30 samples (you can think of a trial as lasting 3 seconds so the model operates at 10 samples/second) and the imagination phase of the model lasts 10 samples (1 sec). (The longer the imagination phase in Martin’s model, the better the performance of the model because it is during the imagination phase that the model is computing the reference that will be used for the response that is made when the imagination phase is over).

Pressing the “run” button runs the model specified in the “Model” cell (1 for Marken’s, 0 for Martin’s). The results of a run are reported in column F and G in terms of Hit Rate (HR) False Alarm Rate (FA) Proportion correct (PC) and d’ (a measure of sensitivity to the difference between signal and no signal trials that is commonly used in detection tasks).

The model behavior is also shown in a graph of the average state of a hypothetical controlled variable – the difference between the perceived state of the stimulus and response – during a trial. This variable is actually controlled by the Marken model but not by the Taylor model. The plot of the state of this variable over time suggests methods for distinguishing between the models in experimental data. The model assumes that responses are made by moving a lever one way to indicate “yes” (the value 1 in the model) and the other to indicate “no” (the value 0 in the model). During a three second trial the subject is assumed to move the lever to the appropriate state (1 or 0) based on whether the stimulus is perceived to be a tone+noise (requiring a 1 response) or noise only (a 0 response). What you see plotted in the graph is the average state of the difference between the perceived sate of the stimulus (1 for tone+noise and 0 for noise alone) and the current state of the lever (a number between 0 and 1) for tone+noise (Y = Yes) and noise only (N = No) trials.

What this plot suggests is that it should be possible to tell the difference between Martin’s and Marken’s model by just monitoring the state of the hypothetical CV during trials. According to Martin’s model there will be a period during a trial, while the person is imagining the response, during which the CV will be far from it’s presumed reference state (0). This is because no response is being made.

But it may be difficult to monitor the state of this hypothetical CV during a trial. So the next approach might be to disturb the response, as suggested a while ago in a post from Bill Powers. You can introduce a response disturbance by setting the response disturbance cell to 1. The response disturbance is just a sine wave disturbance to the position of the response lever, like the disturbance to the cursor in a tracking task. Since both models a controlling the response made on each trial (in Martin’s model, this control begins only after imagination is complete but it happens) this disturbance does not affect observed behavior of either either model in any particularly noticeable way.

Another approach to testing the model is to apply what I call a stimulus disturbance. When the stimulus disturbance is in effect, the stimulus on a trial changes 15 samples into the trial. So if the trial began as a tone trial it becomes a no tone trial and vice versa. This disturbance lowers the performance of Martin’s model considerably. This is because the response made at the end of a trial is based on the reference established during the initial imagination phase; so if the trial started as a tone+noise, the model imagines a “yes” response, which becomes the reference for the response that is actually model at the end of the trial. But now that response is inappropriate because the stimulus has changed.

This change in the stimulus – the stimulus disturbance – has almost no effect on the performance of the Marken model because the model is continuously controlling for the difference between the stimulus and response (the CV) trying to keep it at zero.

So I think that is ultimately the way to test the difference between the Martin and Marken models; change the stimulus during a trial in a detection task. This disturbance would be resisted by a person controlling for the relationship between S and R but not by a system controlling only R.

The model needs more work, I’m sure. But I’d like to get your suggestions about how to make it clearer or better before continuing.

Best

Rick

PsychExpSimwExp0907.xls (61 KB)

···


Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

[From Rick Marken (2010.09.10.1145)]

Here's a new and improved version of the preadsheet that compare's my
model of yes/no detection to Martin's. The changes are somewhat small
but they make a huge difference. Most important, the graph now shows
the state of the hypothetical controlled variable (CV) that could
actually be observed in the experiment. It's the difference between
the state of the stimulus (Tone or No Tone) and the response (Yes or
No)).

Since both stimulus state and response are binary variables the state
of the controlled variable is binary, 1 or 0. What is being graphed is
the average state of the CV on Tone and No Tone trials. The CV goes to
0 when the response is correct (matches the stimulus) but when the
tone is obscured by noise the subject is no correct every time so the
average CV over trials doesn't go to 0 on both kinds of trials.

I have added an entry called NSRatio that determines the level of
noise relative to tone amplitude. Setting this to zero means that the
subject (model) can tell easily whether it is a tone or no tone trial.
It's instructive to run the models with NSRatio set to zero. Both
models behave perfectly; the average CV always goes to 0 on both kinds
of trials and the Proportion Correct (PC) is 1.0 (as is the Hit Rate).
Most importantly, both models still behave perfectly when a
disturbance is added to the response (by setting the Response
Disturbance cell to 1). But the behavior of the two models is
completely different when the stimulus is disturbed (by setting the
Stimulus Disturbance cell to 1).

The stimulus disturbance is just a change in the stimulus from tone to
no tone (or no tone to tone) at the end to the stimulus presentation
interval. The behavior of the Marken model, which is controlling for
the difference between tone and response, is not affected at all by
the disturbance to the stimulus. The Martin model, which is
controlling for the difference between tone and response only in
imagination and then producing the controlled response, deteriorates
completely; the proportion correct goes to 0 and the average CV value
on both tone and no tone trials no longer goes to 0. The reason is
that Martin's model is imagining the correct response and generating
that response during the time when the change in the stimulus occurs.

This modeling effort suggests a very simple way to discriminate
between Martin's and my model of detection. I will propose that
experiment once I get confirmation from Martin that this is a proper
implementation of his model.

Best

Rick

PsychExpSimwExp0910.xls (106 KB)

···

--
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

Content-Type: application/vnd.ms-excel; name="PsychExpSimwExp0910.xls"
Content-Disposition: attachment; filename="PsychExpSimwExp0910.xls"
X-Attachment-Id: f_gdxdkc2v1

[From Rick Marken (2010.09.14.2135)]

A few days ago I posted this:

[From Rick Marken (2010.09.10.1145)]

Here's a new and improved version of the preadsheet that compare's my
model of yes/no detection to Martin's...
...
This modeling effort suggests a very simple way to discriminate
between Martin's and my model of detection. I will propose that
experiment once I get confirmation from Martin that this is a proper
implementation of his model.

I might have time to work on this this weekend so I don't want it to
slip through the cracks. Does my description of the models and the
results produced in the spreadsheet seem OK to you, Martin? I'm happy
to describe how the models work in more detail if the description in
the post above isn't clear.

Best

Rick

···

--
Richard S. Marken PhD
rsmarken@gmail.com
www.mindreadings.com

[Martin Taylor 2010.09.15.01.22]

[From Rick Marken (2010.09.14.2135)]

A few days ago I posted this:

[From Rick Marken (2010.09.10.1145)]

Here's a new and improved version of the preadsheet that compare's my
model of yes/no detection to Martin's...
...
This modeling effort suggests a very simple way to discriminate
between Martin's and my model of detection. I will propose that
experiment once I get confirmation from Martin that this is a proper
implementation of his model.

I might have time to work on this this weekend so I don't want it to
slip through the cracks. Does my description of the models and the
results produced in the spreadsheet seem OK to you, Martin? I'm happy
to describe how the models work in more detail if the description in
the post above isn't clear.

Sorry, I've been deliberately avoiding this thread for the last week or so. While I was off the net for a week I had prepared a couple of responses to previous messages by Bill, but when I learned he had been in hospital I didn't want to send them in case it made him angry, as has been happening a lot recently, and I didn't want that. So I haven't been reading the thread lest I was tempted to respond.

I'm going away for ten days starting today, but I ought to be in e-mail touch, so I'll have a look and see if there's anything in the thread to which I should respond, taking particular note of this message of yours.

Martin

[From Rick Marken(2010.09.16.1120)]

[Martin Taylor 2010.09.15.01.22]

[From Rick Marken (2010.09.14.2135)]

A few days ago I posted this:

[From Rick Marken (2010.09.10.1145)]

Here's a new and improved version of the preadsheet that compare's my
model of yes/no detection to Martin's...
...
This modeling effort suggests a very simple way to discriminate
between Martin's and my model of detection. I will propose that
experiment once I get confirmation from Martin that this is a proper
implementation of his model.

I might have time to work on this this weekend so I don't want it to
slip through the cracks. Does my description of the models and the
results produced in the spreadsheet seem OK to you, Martin? I'm happy
to describe how the models work in more detail if the description in
the post above isn't clear.

Sorry, I've been deliberately avoiding this thread for the last week or so. While I was off the net for a week I had prepared a couple of responses to previous messages by Bill, but when I learned he had been in hospital I didn't want to send them in case it made him angry, as has been happening a lot recently, and I didn't want that. So I haven't been reading the thread lest I was tempted to respond.

I'm going away for ten days starting today, but I ought to be in e-mail touch, so I'll have a look and see if there's anything in the thread to which I should respond, taking particular note of this message of yours.

Martin

Thanks Martin. Have a nice trip. I look forward to hearing from you.

Best

Rick