Limitations to Formal Models in concert with the Test?

[Andrew Willett 20170401. 20:26 ET]

I would like to introduce a new thread topic to CSGNet–I am unsure whether this has been discussed before. I am finishing up my undergraduate thesis, and among the many concepts in PCT I am trying to understand, I have been taking a good look at the methods and practices employed. This thread is considering the topic I am currently finding no solution to. I would greatly appreciate any feedback on this–I know I am not a regular contributor to the discussion here, but I do read the posts quite frequently.

As far as I am aware, many PCT studies (excepting maybe Robertson, Goldstein, Mermel, and Musgrave [1999]) utilize formal models to evaluate the existence of a controlled variable for a single individual. The most common way of doing this is to look at how an agent (usually human) tracks a relationship within their local environment. In pursuit tracking studies, graphs are usually bu
ilt, which can plot the pursuing agent (e.g., a cursor on a screen, a hand in a grassy field) as a function of location (in relation to target) and time (those graphs usually look like heart-rate monitors).

To be simple: The Test often is done alongside formal model-building.

My first questions concerns the mathematics. As I am not well-versed in mathematics (4 years ago I did AP AB calculus), I wanted a clarification. Tracking is usually isolated to 2 spatial dimensions with time (time and x-y space), but I imagine that it could be expanded to include a 3rd dimension (time, x, y, and z). Is this correct? Can only certain types of mathematics be used with tracking studies?
I’m trying to understand whether (if yes or no and why) there is a limitation to the formal modeling used in PCT for the Test, especially the modeling used with human-model studies (the tracking ones).

The more I think about it, the more I am feeling that beyond a certain hierarchical level (I am thinking the relationship level since this is [as far as I am aware] the highest level discussed in tracking tasks), mathematics may not accurately model what’s going on. In other words, it becomes harder to create a formal model to relate to an aspect of human behavior that is abstract (e.g., controlling a principle-level perception). This brings me to my next question.

If this is so, I am going to propose that qualitative description/methods have some worth. For example, it may not be useful to reduce an individual’s perception of “United States politics in relation to Canadian politics” over the course of a heated discussion to a mathematical formula (we lose some important rich subjective details?). I think it is even less possible that this formula could then somehow track this relationship in the local environment (like computerized tracking studies or chasing choppers)
. Am I wrong?

Many scientists, especially those in model-based disciplines like physics and chemistry, might see less worth in my claim that, at a certain point in the hierarchy, we can no longer do the Test with formal models. My (short-lived at this point) experience has been that some scientists (excepting, at least, anthropologists) see qualitative description as second to quantitative descriptions. Yet, I am sure that at some point the mathematics becomes overly-reductive. In other words, my rich experience of considering those politics or caring about being an honest person ought not to be relegated to a mathematical formula. Any thoughts on this?

This is a bit of a long post. Nonetheless, I hope it sparks some discussion.

[From Rick Marken (2017.04.02.1810)]

···

Andrew Willett (20170401. 20:26 ET)–

AW: I would like to introduce a new thread topic to CSGNet…

 RM: I think the essence of your topic is stated here:Â

AW: I’m trying to understand whether (if yes or no and why) there is a limitation to the formal modeling used in PCT for the Test, especially the modeling used with human-model studies (the tracking ones).

RM: This is a great question and the simple answer is "yes, there is a limitation to formal modeling , and it is in our ability to build models of the perceptual functions that produce the higher level perceptual variables that people control, such as principles (like “do unto others”) and system concepts (like “being an American”). This is not really a limitation of mathematics per se; it is a limitation in knowing how to use mathematics to produce those perceptual variables.

RM: We know it’s not a limitation of mathematics for a couple reasons. One is that we can perceive these principles and system concepts ourselves and we know that these perceptions are being created by the neural networks in our brains; the computations carried out by these neural networks can be described mathematically, possible examples of mathematical computations that could be carried out by the nervous system are described  in the Premises chapter of B:CP. So if your brain can produce a perception of the degree to which one is behaving according to the principle “do unto others”, a machine capable of carrying out the same computations as the brain should be able to produce such a perception as well.Â

RM: Another reason we know it’s not a limitation of mathematics but of how to use it is because we have been able to use mathematics to produce perceptions that many thought we would never be able to produce, such as the perception of speech and handwriting. Now I have a phone that recognizes what I say (with reasonably good accuracy) and an ATM that reads my checks. The mathematical computations that produce these perceptions are almost certainly nothing like the ones done by your brain but they do show that such perceptions can be produced by a system that can carry out mathematical calculations in some way. Â

RM: But it is unlikely mathematical algorithms for producing perceptions of principles and system concepts will be developed in the near future. So we won’t be able to build models that can control such variables any time soon. But I don’t think we need to build such models in order to make progress in research aimed at understanding the controlling done by systems (like humans) that are clearly capable of controlling such complicated perceptual variables. What we can do is use our own perceptual systems to see whether other systems control the same kinds of perceptions that we control. I believe this is what you would call qualitative research, but I think it is possible to do this research with a level of rigor that will produce useful results.

RM:  A hint of how to do it is in Powers’ description of the “Coin Game” in B:CP, in experiments 5 and 6 of the 1978 Psych Review paper and in my “Hierarchical Control” demo (http://www.mindreadings.com/ControlDemo/Hierarchy.html). I think the Robertson et al (1999) study is another good example of how this qualitative approach to determining the perceptions people control. In all these examples of PCT research, the variables being controlled are determined by an observer perceiving the variable, either directly with his own perceptual systems or using a computer to measure how well the perception is protected from disturbance.Â

RM: I get the gist of how such a qualitative PCT research program would proceed but I’ve never tried to hash out a formal description of what I have in mind. Maybe we could try to do that in this thread.Â

BestÂ

Rick


Richard S. MarkenÂ

"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.�
                --Antoine de Saint-Exupery

[Martin Taylor 2017.04.02.23.10]

I see Rick has offered a pretty good answer to one of your

questions. I won’t comment on that, but I’d like to take a different
approach.

[Andrew Willett 20170401. 20:26 ET]

      To be simple: The Test often is done alongside formal

model-building.

Just a comment on the word "Test". We usually use the "Test"

(capital T) to refer to a specific experiment, but you preceded this
by talking about tracking studies. It’s true that tracking studies
are in principle representative of what any control is, and indeed
any behaviour, if you accept the tenet that all intentional
behaviour is the control of perception. But the Test is usually used
to refer to a particular kind of experiment in which an experimenter
tries to determine what perception someone is controlling.

      My first questions concerns the mathematics. As I am not

well-versed in mathematics (4 years ago I did AP AB calculus),
I wanted a clarification. Tracking is usually isolated to 2
spatial dimensions with time (time and x-y space), but I
imagine that it could be expanded to include a 3rd dimension
(time, x, y, and z). Is this correct? Can only certain types
of mathematics be used with tracking studies?
I’m trying to understand whether (if yes or no and why )
there is a limitation to the formal modeling used in PCT for
the Test, especially the modeling used with human-model
studies (the tracking ones).

I could understand what you are asking in at least two ways. One is

about the mathematical toolbox that is applicable to tracking
studies, the other is about the possibilities of formal modelling.
They aren’t the same.

A mathematical toolbox is like any other toolbox. You get toolboxes

for tiny tots, toolboxes for teenagers, toolboxes for skilled
amateurs, and toolboxes for professionals, whether they be
carpenters, surgeons, or plumbers. The contents of the toolboxes at
any of these levels will be different for the different kinds of
tasks, but when you consider just “tools” the universe includes
surgeon’s scalpel, plumber’s wrench, and carpenter’s lathe. So when
you think of a mathematical toolbox, the same kind of distinction
applies. Some tools will usually be very useful to look at tracking
studies, some will seldom, if ever, be useful. Just as I can (just
about) imagine a surgeon using a hammer, I find it much easier to
imagine a carpenter using one. For looking at tracking studies, will
set theory be useful? Maybe at the higher hierarchic levels. Who
knows, I don’t think it’s been tried. Is calculus useful for
tracking studies, surely it is at lower levels where the variables
are continuous, but is it likely to be useful above the category
level where variables are discrete? Maybe not. Again, who knows.

If your question is about formal modelling, I have to guess what you

mean by “formal”. Is a software program a formal model? A lot of
pattern recognition approaches to perception use fuzzy logic. Does
that count as “formal”? The most popular model for tracking studies
is a software program, because calculus-based mathematical models
that are easy to solve tend not to represent system nonlinearities
very well, whereas they can be built into software models if
necessary. The saving grace is that when control is good, the
Behavioural Illusion obscures the internal processing that is being
modelled so that simple models are about as good as you can usefully
get, as I explained in the attachment to [Martin Taylor
2017.03.28.23.36].

      The more I think about it, the more I am feeling that beyond a

certain hierarchical level (I am thinking the relationship
level since this is [as far as I am aware] the highest level
discussed in tracking tasks), mathematics may not accurately
model what’s going on. In other words, it becomes harder to
create a formal model to relate to an aspect of human behavior
that is abstract (e.g., controlling a principle-level
perception).

It depends on what you mean by "mathematics". "It becomes harder" is

presumably true, but you can’t tell for sure, because even at the
lowest levels it is very hard to model anything but a drastically
simplified version of what goes on inside the organism. Think, for
example if Bill’s “neural current”, which is a really drastic
simplification of the firing patterns of lots of neurons that in
some way change in ways that relate to changes in sensory inputs.
The neat thing is that the behaviour of simulations that don’t even
try to mimic individual neurons is remarkably similar to the
behaviour of organic controllers that (presumably) use the neuron
firing patterns. How one might simplify the issues that arise at
higher levels, which involve imagined perceptions along with
perceptions derived from current sensory input – that’s a question
nobody has addressed as far as I know. That doesn’t mean it’s not
open to mathematical investigation. Remember that Boole considered
his binary logic to be the “Laws of Thought”.

This brings me to my next question.

      If this is so, I am going to propose that qualitative

description/methods have some worth.

Of course they do. They are simplifications that work to some

degree. The question is whether they have enough worth for serious
study, and on that I don’t care to comment. Personally, I do a lot
of qualitative analysis. In my mathematical toolbox is a tool I call
“envelope constraints”, that sometimes say that this or that is
physically unrealistic. For example, brain heating due to neural
firing provides a cooling constraint that presumably will have led
evolution to favour minimizing the computation that needs to be done
to produce an improvement in survivability. This constraint leads me
to argue qualitatively that mechanisms for improvement in control
quality, and reduction of side-effects and of internal conflict
should exist. Powers calls the mechanism “reorganization”.

      For example, it may not be useful to reduce an

individual’s perception of “United States politics in relation
to Canadian politics” over the course of a heated discussion
to a mathematical formula (we lose some important rich
subjective details?). I think it is even less possible that
this formula could then somehow track this relationship in the
local environment (like computerized tracking studies or
chasing choppers) . Am I wrong?

I'd say you are probably wrong in principle, but right in practice.

Most of the time. Would a tracking poll on the several dimensions of
the difference between the two political regimes, done on
individuals rather than on aggregates, count for you as a track? The
individuals could be subject to different disturbances on the
different dimensions if you wanted to test whether they controlled
those perceptions. I doubt you would get much out of such a study,
but I’m wondering whether it would fit your criterion.

      Many scientists, especially those in model-based

disciplines like physics and chemistry, might see less worth
in my claim that, at a certain point in the hierarchy, we can
no longer do the Test with formal models. My (short-lived at
this point) experience has been that some scientists
(excepting, at least, anthropologists) see qualitative
description as second to quantitative descriptions. Yet, I am
sure that at some point the mathematics becomes
overly-reductive. In other words, my rich experience of
considering those politics or caring about being an honest
person ought not to be relegated to a mathematical formula.
Any thoughts on this?

Yes. It depends again on what you mean by "mathematics" and "formal

models". See above. If mathematics in some case has “become overly
reductive”, quite probably the wrong tool has been used from the
toolbox. The surgeon used a laser when an infra-red lamp would have
been more better for the patient. But in general, the more precise
you can be, the better you can relate what you find to other areas
of your knowledge.

      This is a bit of a long post. Nonetheless, I hope it sparks

some discussion.

It's a difficult topic. We will see.

Martin

Fabulous question, and great answers. I would like to just add that I think it is likely that mathematics at any level of the hierarchy is still only likely to be an approximation of how any individual’s brain is controlling a perceptual variable - even though it can be extremely accurate at times. I think it is easy to underestimate the complexity that goes into perceiving some aspect of the environment and controlling it effectively, even for ‘simple’ sensations and configurations. I wonder whether there will be some circumstances (Kent can you help?) when the mathematics of modelling how a higher level variable is controlled (e.g. a principle) are actually simpler than the mathematics of how a lower level variable is controlled (e.g. a configuration), depending on the exact nature of that entity in the environment? I guess part of the issue depends on whether we just want to model the control loop and make short-cuts for how that variable is exactly perceived and acted upon in the world, or whether we want to construct a model that actually perceives the real environment and actually acts on the real environment (like Rupert’s robots)?
Warren

···

On Mon, Apr 3, 2017 at 5:03 AM, Martin Taylor mmt-csg@mmtaylor.net wrote:

[Martin Taylor 2017.04.02.23.10]

I see Rick has offered a pretty good answer to one of your

questions. I won’t comment on that, but I’d like to take a different
approach.

[Andrew Willett 20170401. 20:26 ET]

      To be simple: The Test often is done alongside formal

model-building.

Just a comment on the word "Test". We usually use the "Test"

(capital T) to refer to a specific experiment, but you preceded this
by talking about tracking studies. It’s true that tracking studies
are in principle representative of what any control is, and indeed
any behaviour, if you accept the tenet that all intentional
behaviour is the control of perception. But the Test is usually used
to refer to a particular kind of experiment in which an experimenter
tries to determine what perception someone is controlling.

      My first questions concerns the mathematics. As I am not

well-versed in mathematics (4 years ago I did AP AB calculus),
I wanted a clarification. Tracking is usually isolated to 2
spatial dimensions with time (time and x-y space), but I
imagine that it could be expanded to include a 3rd dimension
(time, x, y, and z). Is this correct? Can only certain types
of mathematics be used with tracking studies?
I’m trying to understand whether (if yes or no and why )
there is a limitation to the formal modeling used in PCT for
the Test, especially the modeling used with human-model
studies (the tracking ones).

I could understand what you are asking in at least two ways. One is

about the mathematical toolbox that is applicable to tracking
studies, the other is about the possibilities of formal modelling.
They aren’t the same.

A mathematical toolbox is like any other toolbox. You get toolboxes

for tiny tots, toolboxes for teenagers, toolboxes for skilled
amateurs, and toolboxes for professionals, whether they be
carpenters, surgeons, or plumbers. The contents of the toolboxes at
any of these levels will be different for the different kinds of
tasks, but when you consider just “tools” the universe includes
surgeon’s scalpel, plumber’s wrench, and carpenter’s lathe. So when
you think of a mathematical toolbox, the same kind of distinction
applies. Some tools will usually be very useful to look at tracking
studies, some will seldom, if ever, be useful. Just as I can (just
about) imagine a surgeon using a hammer, I find it much easier to
imagine a carpenter using one. For looking at tracking studies, will
set theory be useful? Maybe at the higher hierarchic levels. Who
knows, I don’t think it’s been tried. Is calculus useful for
tracking studies, surely it is at lower levels where the variables
are continuous, but is it likely to be useful above the category
level where variables are discrete? Maybe not. Again, who knows.

If your question is about formal modelling, I have to guess what you

mean by “formal”. Is a software program a formal model? A lot of
pattern recognition approaches to perception use fuzzy logic. Does
that count as “formal”? The most popular model for tracking studies
is a software program, because calculus-based mathematical models
that are easy to solve tend not to represent system nonlinearities
very well, whereas they can be built into software models if
necessary. The saving grace is that when control is good, the
Behavioural Illusion obscures the internal processing that is being
modelled so that simple models are about as good as you can usefully
get, as I explained in the attachment to [Martin Taylor
2017.03.28.23.36].

      The more I think about it, the more I am feeling that beyond a

certain hierarchical level (I am thinking the relationship
level since this is [as far as I am aware] the highest level
discussed in tracking tasks), mathematics may not accurately
model what’s going on. In other words, it becomes harder to
create a formal model to relate to an aspect of human behavior
that is abstract (e.g., controlling a principle-level
perception).

It depends on what you mean by "mathematics". "It becomes harder" is

presumably true, but you can’t tell for sure, because even at the
lowest levels it is very hard to model anything but a drastically
simplified version of what goes on inside the organism. Think, for
example if Bill’s “neural current”, which is a really drastic
simplification of the firing patterns of lots of neurons that in
some way change in ways that relate to changes in sensory inputs.
The neat thing is that the behaviour of simulations that don’t even
try to mimic individual neurons is remarkably similar to the
behaviour of organic controllers that (presumably) use the neuron
firing patterns. How one might simplify the issues that arise at
higher levels, which involve imagined perceptions along with
perceptions derived from current sensory input – that’s a question
nobody has addressed as far as I know. That doesn’t mean it’s not
open to mathematical investigation. Remember that Boole considered
his binary logic to be the “Laws of Thought”.

This brings me to my next question.

      If this is so, I am going to propose that qualitative

description/methods have some worth.

Of course they do. They are simplifications that work to some

degree. The question is whether they have enough worth for serious
study, and on that I don’t care to comment. Personally, I do a lot
of qualitative analysis. In my mathematical toolbox is a tool I call
“envelope constraints”, that sometimes say that this or that is
physically unrealistic. For example, brain heating due to neural
firing provides a cooling constraint that presumably will have led
evolution to favour minimizing the computation that needs to be done
to produce an improvement in survivability. This constraint leads me
to argue qualitatively that mechanisms for improvement in control
quality, and reduction of side-effects and of internal conflict
should exist. Powers calls the mechanism “reorganization”.

      For example, it may not be useful to reduce an

individual’s perception of “United States politics in relation
to Canadian politics” over the course of a heated discussion
to a mathematical formula (we lose some important rich
subjective details?). I think it is even less possible that
this formula could then somehow track this relationship in the
local environment (like computerized tracking studies or
chasing choppers) . Am I wrong?

I'd say you are probably wrong in principle, but right in practice.

Most of the time. Would a tracking poll on the several dimensions of
the difference between the two political regimes, done on
individuals rather than on aggregates, count for you as a track? The
individuals could be subject to different disturbances on the
different dimensions if you wanted to test whether they controlled
those perceptions. I doubt you would get much out of such a study,
but I’m wondering whether it would fit your criterion.

      Many scientists, especially those in model-based

disciplines like physics and chemistry, might see less worth
in my claim that, at a certain point in the hierarchy, we can
no longer do the Test with formal models. My (short-lived at
this point) experience has been that some scientists
(excepting, at least, anthropologists) see qualitative
description as second to quantitative descriptions. Yet, I am
sure that at some point the mathematics becomes
overly-reductive. In other words, my rich experience of
considering those politics or caring about being an honest
person ought not to be relegated to a mathematical formula.
Any thoughts on this?

Yes. It depends again on what you mean by "mathematics" and "formal

models". See above. If mathematics in some case has “become overly
reductive”, quite probably the wrong tool has been used from the
toolbox. The surgeon used a laser when an infra-red lamp would have
been more better for the patient. But in general, the more precise
you can be, the better you can relate what you find to other areas
of your knowledge.

      This is a bit of a long post. Nonetheless, I hope it sparks

some discussion.

It's a difficult topic. We will see.



Martin

Dr Warren Mansell
Reader in Clinical Psychology

School of Health Sciences
2nd Floor Zochonis Building
University of Manchester
Oxford Road
Manchester M13 9PL
Email: warren.mansell@manchester.ac.uk

Tel: +44 (0) 161 275 8589

Website: http://www.psych-sci.manchester.ac.uk/staff/131406

Advanced notice of a new transdiagnostic therapy manual, authored by Carey, Mansell & Tai - Principles-Based Counselling and Psychotherapy: A Method of Levels Approach

Available Now

Check www.pctweb.org for further information on Perceptual Control Theory

[Andrew Willett 2017 0403 11:40 ETA]

I appreciate all the commentary sofar–I think this is helping piece together the situation.

In particular, I would like to respond to Martin’s comments, but feel free to continue to provide commentary on the original post.

···

On Mon, Apr 3, 2017 at 12:03 AM, Martin Taylor mmt-csg@mmtaylor.net wrote:

[Martin Taylor 2017.04.02.23.10]

Just a comment on the word "Test". We usually use the "Test"

(capital T) to refer to a specific experiment, but you preceded this
by talking about tracking studies. It’s true that tracking studies
are in principle representative of what any control is, and indeed
any behaviour, if you accept the tenet that all intentional
behaviour is the control of perception. But the Test is usually used
to refer to a particular kind of experiment in which an experimenter
tries to determine what perception someone is controlling.

That is absolutely correct. In my thesis, in one section (of chapter 2) I started with talking about the Test generally, then explain that quantitative modeling methods seem to be used to confirm whether or not a controlled variable is indeed relevant. I am using the tracking studies as an example of quantitative modeling, not necessarily as an example of the Test.

I could understand what you are asking in at least two ways. One is

about the mathematical toolbox that is applicable to tracking
studies, the other is about the possibilities of formal modelling.
They aren’t the same.

A mathematical toolbox is like any other toolbox. You get toolboxes

for tiny tots, toolboxes for teenagers, toolboxes for skilled
amateurs, and toolboxes for professionals, whether they be
carpenters, surgeons, or plumbers. The contents of the toolboxes at
any of these levels will be different for the different kinds of
tasks, but when you consider just “tools” the universe includes
surgeon’s scalpel, plumber’s wrench, and carpenter’s lathe. So when
you think of a mathematical toolbox, the same kind of distinction
applies. Some tools will usually be very useful to look at tracking
studies, some will seldom, if ever, be useful. Just as I can (just
about) imagine a surgeon using a hammer, I find it much easier to
imagine a carpenter using one. For looking at tracking studies, will
set theory be useful? Maybe at the higher hierarchic levels. Who
knows, I don’t think it’s been tried. Is calculus useful for
tracking studies, surely it is at lower levels where the variables
are continuous,but is it likely to be useful above the category
level where variables are discrete? Maybe not. Again, who knows.

First: I would like just a quick clarification here (not entirely relevant for this discussion on formal models/the Test). As far I can recall, Powers proposes the analog computer (the earlier invention of von Neumann) as the model of choice with PCT (he provides a direct comment on this in B:CP, which he edited in the last 8 years of his work)–he was concerned with continuous functions, describing the various levels of the hierarchy in this way. What evidence do you have from Powers, PCT literature, or general neurophysiology literature that suggests that theres are discrete functions above the 7th-level of the hierarchy?

Second: The Tracking Mathematics: some background may be helpful~~this is what I am using as a precursor to this discussion of mathematics.

In my thesis, I consider two studies:

  1. Bourbon et al.'s (1990) study on pursuit tracking

  2. Marken’s (2013) study on pursuit tracking

Both studies considered the controlled variables involved in pursuit tracking.

Bourbon uses mathematics to define the pursuit tracking controlled variable as a control of the vertical distance between the target and the cursor.

He found a very good fit for the model (on the order of r= .99)

Marken’s study replicated this with two different participants.

When he fit the same controlled model to two participants, he found a lower correlation with the same formula (r = ~.76)

Marken then provided a secondary model, that of angle-control between cursor and target

With this secondary model, there was a stronger correlation (on the order of r = ~.96)

I considered two possible reasons for the discrepancy in results

  1. external validity~the small sample sizes: HOWEVER PCT claims that, since there is an organizing system, larger samples are not as necessary as a “net-casting” approach.

  2. apparatus differences: Bourbon et al. used a handle (e.g., like one used to control library stacks or a drawbridge), whereas Marken used a mouse (more used to using this in multi-dimensional spaces~e.g., a desktop). This seemed more likely to explain the differences

This also made me wonder what the limits of these sorts of formulae might be. I don’t speculate on whether the two studies had mathematical errors so I instead consider whether there are limitations to mathematics in the context of the tracking paradigm. I therefore followed this discussion of these studies with a paragraph (I will be revising this one), in which I consider whether the tracking paradigm (as a case example for formal models used in PCT research in general) is limited in its application. Below are a few ideas on this (again, I apologize that this is lengthy, but I want to be sure I make my ideas clear):

“While the source of different results in Marken
(2014) and Bourbon et al.’s (1990) studies may not have been due to
mathematical errors, it is important to consider the limitations of
mathematics. A majority of quantitative PCT studies default to some form of
tracking. Tracking is best understood in terms of how it is displayed
graphically. Graphs plot the pursuing agent (e.g., a cursor on a screen, a hand
in a grassy field) as a function of location (in relation to the target) and
time. These graphs purport to illustrate an agent’s control of 6th-order
relationship perceptions” (Willett, Thesis Draft April 2, p. 27).

Tracking studies are the most common research method in PCT–this paradigm provides a simple means for constructing a model and relating it to an agent.

I am assuming

  1. Tracking is the primary example for how the Test could relate an observable behavior with a formal model (Martin, your next question concerns my definition of formal models, please refer to this for clarification)

I suppose I am questioning here

  1. whether tracking concerns control of relationship perceptions only

  2. whether this illustrates a limitation to the tracking paradigm (keeps from talking of more abstract perceptions)

  3. whether there is indeed a method for considering control of higher-order perceptions and relating these to formal models

  4. whether these other methods (only I know of is, again, Robertson et al.'s (1990) study–more qualitative) can provide sufficient explanations for the perceptual control or whether they simply reduce my complex higher-level experience to mathematical formulae.

If anyone can talk of these above questions, I would very grateful. I am sure this would also help clarify this discussion.

If your question is about formal modelling, I have to guess what you

mean by “formal”.

A “formal model” is a term that is used in cognitive science generally (various disciplines, e.g., AI, psychology, etcetera). The nature of a formal model is synonymous with what Powers terms “physical models” in his 2005 revision of B:CP. If you would like more explanation on formal models, Smaldino (2017) is a good source–I will send you this through a private email since I am not certain whether this has been published online (it is forthcoming, but my professor gave me an accepted draft). Formal models differ from verbal models, according to Smaldino (2017) in a way similar to how Power describes physical models as differing from abstract generalizations.

Is a software program a formal model? A lot of

pattern recognition approaches to perception use fuzzy logic. Does
that count as “formal”? The most popular model for tracking studies
is a software program, because calculus-based mathematical models
that are easy to solve tend not to represent system nonlinearities
very well, whereas they can be built into software models if
necessary. The saving grace is that when control is good, the
Behavioural Illusion obscures the internal processing that is being
modelled so that simple models are about as good as you can usefully
get, as I explained in the attachment to [Martin Taylor
2017.03.28.23.36].

      The more I think about it, the more I am feeling that beyond a

certain hierarchical level (I am thinking the relationship
level since this is [as far as I am aware] the highest level
discussed in tracking tasks), mathematics may not accurately
model what’s going on. In other words, it becomes harder to
create a formal model to relate to an aspect of human behavior
that is abstract (e.g., controlling a principle-level
perception).

It depends on what you mean by "mathematics". "It becomes harder" is

presumably true, but you can’t tell for sure, because even at the
lowest levels it is very hard to model anything but a drastically
simplified version of what goes on inside the organism . Think, for
example if Bill’s “neural current”, which is a really drastic
simplification of the firing patterns of lots of neurons that in
some way change in ways that relate to changes in sensory inputs.
The neat thing is that the behaviour of simulations that don’t even
try to mimic individual neurons is remarkably similar to the
behaviour of organic controllers that (presumably) use the neuron
firing patterns. How one might simplify the issues that arise at
higher levels, which involve imagined perceptions along with
perceptions derived from current sensory input – that’s a question
nobody has addressed as far as I know. That doesn’t mean it’s not
open to mathematical investigation. Remember that Boole considered
his binary logic to be the “Laws of Thought”.

Alright. So what I’m hearing here is that the physical models (formal models, as Smaldino [2017] terms them) are simplified. What you say is, despite the simplification, useful information is still gleaned (again, this is something that Smaldino [2017] considers).

      For example, it may not be useful to reduce an

individual’s perception of “United States politics in relation
to Canadian politics” over the course of a heated discussion
to a mathematical formula (we lose some important rich
subjective details?). I think it is even less possible that
this formula could then somehow track this relationship in the
local environment (like computerized tracking studies or
chasing choppers) . Am I wrong?

I'd say you are probably wrong in principle, but right in practice.

Most of the time.

Not clear what this means.

Would a tracking poll on the several dimensions of

the difference between the two political regimes, done on
individuals rather than on aggregates, count for you as a track? The
individuals could be subject to different disturbances on the
different dimensions if you wanted to test whether they controlled
those perceptions. I doubt you would get much out of such a study,
but I’m wondering whether it would fit your criterion.

Indeed–I am saying that here is an example of a control of a relationship perception, which could be disturbed over time. Unlike a more physical relationship–controlling a knot over a dot (rubber band example), something may be lost by simply describing this relationship perceptual control situation with mathematics. It reduces a complex rich experience of politics to be equivalent with the control of a rubber band. This bothers me–the two are very different, no?

      Many scientists, especially those in model-based

disciplines like physics and chemistry, might see less worth
in my claim that, at a certain point in the hierarchy, we can
no longer do the Test with formal models. My (short-lived at
this point) experience has been that some scientists
(excepting, at least, anthropologists) see qualitative
description as second to quantitative descriptions. Yet, I am
sure that at some point the mathematics becomes
overly-reductive. In other words, my rich experience of
considering those politics or caring about being an honest
person ought not to be relegated to a mathematical formula.
Any thoughts on this?

Yes. It depends again on what you mean by "mathematics" and "formal

models". See above. If mathematics in some case has “become overly
reductive”, quite probably the wrong tool has been used from the
toolbox. The surgeon used a laser when an infra-red lamp would have
been more better for the patient. But in general, the more precise
you can be, the better you can relate what you find to other areas
of your knowledge.

This is a fair point.

      This is a bit of a long post. Nonetheless, I hope it sparks

some discussion.

It's a difficult topic. We will see.



Martin

[From MK (2017.04.04.1645 CET)]

At which level in the hierarchy would you place "space" as an abstraction?

M

[From Rick Marken (2017.04.05.1130)]

···

Andrew Willett (2017 0403 11:40 ETA)–

RM: I think you misunderstand my 2013 study. It was not a replication of Tom’s study. Rather, it was a demonstration of using a control model to determine the perceptual variable controlled in a pursuit tracking task where the horizontal separation, s, between a vertically moving target and cursor had been varied on different trials. I didn’t report the equivalent of Tom’s measure of fit of model to data  (his r = .99) because I was looking for the ability of the models to capture the deterioration of performance (how close the cursor was kept to the target during a trail – measured as -log2 (var(c-t)/var(d) , a measure of control in “bits” – with increasing horizontal separation, s, between cursor and target. But you can see from Figure 4 that all models fit the time varying outputs quite well (in fact, the correlation between model and data was > .98 for all separations).Â

RM: The low correlation that you note (r = ~.76, the square root of the reported R^2 of .51) is between the performance at different horizontal separations, s, of the distance model and the humans. These correlations are not the same as the correlations reported in Tom’s study, which are correlations between the time varying model and human outputs.Â

RM: It’s neither of these. The discrepancy results from the fact that Tom’s correlations are measures of correlation between time variations in model and human outputs; the correlations in my 2013 paper are measures of correlation between model and human performance (how well they kept the cursor aligned with the target) at the different horizontal separations between target and cursor.Â

RM: The simplest way to look at my 2013 study was as an attempt to explain why control (the ability to keep the cursor aligned with the target) deteriorates with increasing visual separation between cursor and target. One notion (the “distance model”) is that people are always controlling the vertical separation between target and cursor (t-c) but the ability to perceive this variable deteriorates as the horizontal distance, s, between t and c increases. An alternative possibility is that people are actually controlling, not (t-c), but the angle between t and c – arcsin((t-c)/s). And the research shows that a model controlling arcsin((t-c)/s matches the performance data better than one that is controlling (t-c) or a poorly perceived (t-c) Â – the"distance threshold" model. So the conclusion is that the the controlled variable in this tracking task was arcsin((t-c)/s rather than (t-c).Â

Â

RM: I hope my explanation of the results of Marken (2013) (reprinted in DRoP in 2014) makes it clear that there is no difference between my results and Tom’s from 1990. The difference between the correlations reported in the two studies results from the fact that they were measuring two completely  different things.Â

BestÂ

Rick


Richard S. MarkenÂ

"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.�
                --Antoine de Saint-Exupery

AW: In my thesis, I consider two studies:Â

  1. Bourbon et al.'s (1990) study on pursuit tracking

  2. Marken’s (2013) study on pursuit tracking

Both studies considered the controlled variables involved in pursuit tracking.

Bourbon uses mathematics to define the pursuit tracking controlled variable as a control of the vertical distance between the target and the cursor.

He found a very good fit for the model (on the order of r= .99)

Marken’s study replicated this with two different participants.Â

When he fit the same controlled model to two participants, he found a lower correlation with the same formula (r = ~.76)

Marken then provided a secondary model, that of angle-control between cursor and target

With this secondary model, there was a stronger correlation (on the order of r = ~.96)

AW: I considered two possible reasons for the discrepancy in results

  1. external validity~the small sample sizes: HOWEVER PCT claims that, since there is an organizing system, larger samples are not as necessary as a “net-casting” approach.

  2. apparatus differences: Bourbon et al. used a handle (e.g., like one used to control library stacks or a drawbridge), whereas Marken used a mouse (more used to using this in multi-dimensional spaces~e.g., a desktop). This seemed more likely to explain the differences

AW: This also made me wonder what the limits of these sorts of formulae might be. I don’t speculate on whether the two studies had mathematical errors so I instead consider whether there are limitations to mathematics in the context of the tracking paradigm. I therefore followed this discussion of these studies with a paragraph (I will be revising this one), in which I consider whether the tracking paradigm (as a case example for formal models used in PCT research in general) is limited in its application. Below are a few ideas on this (again, I apologize that this is lengthy, but I want to be sure I make my ideas clear):

“While the source of different results in Marken
(2014) and Bourbon et al.’s (1990) studies may not have been due to
mathematical errors, it is important to consider the limitations of
mathematics. A majority of quantitative PCT studies default to some form of
tracking. Tracking is best understood in terms of how it is displayed
graphically. Graphs plot the pursuing agent (e.g., a cursor on a screen, a hand
in a grassy field) as a function of location (in relation to the target) and
time. These graphs purport to illustrate an agent’s control of 6th-order
relationship perceptions” (Willett, Thesis Draft April 2, p. 27).

Tracking studies are the most common research method in PCT–this paradigm provides a simple means for constructing a model and relating it to an agent.

I am assuming

  1.  Tracking is the primary example for how the Test could relate an observable behavior with a formal model (Martin, your next question concerns my definition of formal models, please refer to this for clarification)

I suppose I am questioning hereÂ

  1. whether tracking concerns control of relationship perceptions onlyÂ

  2. whether this illustrates a limitation to the tracking paradigm (keeps from talking of more abstract perceptions)

  3. whether there is indeed a method for considering control of higher-order perceptions and relating these to formal models

4) whether these other methods (only I know of is, again, Robertson et al.'s (1990) study–more qualitative) can provide sufficient explanations for the perceptual control or whether they simply reduce my complex higher-level experience to mathematical formulae.Â

If anyone can talk of these above questions, I would very grateful. I am sure this would also help clarify this discussion.

If your question is about formal modelling, I have to guess what you

mean by “formal”.

A “formal model” is a term that is used in cognitive science generally (various disciplines, e.g., AI, psychology, etcetera). The nature of a formal model is synonymous with what Powers terms “physical models” in his 2005 revision of B:CP. If you would like more explanation on formal models, Smaldino (2017) is a good source–I will send you this through a private email since I am not certain whether this has been published online (it is forthcoming, but my professor gave me an accepted draft). Formal models differ from verbal models, according to Smaldino (2017) in a way similar to how Power describes physical models as differing from abstract generalizations.

Â

      The more I think about it, the more I am feeling that beyond a

certain hierarchical level (I am thinking the relationship
level since this is [as far as I am aware] the highest level
discussed in tracking tasks), mathematics may not accurately
model what’s going on. In other words, it becomes harder to
create a formal model to relate to an aspect of human behavior
that is abstract (e.g., controlling a principle-level
perception).

Is a software program a formal model? A lot of

pattern recognition approaches to perception use fuzzy logic. Does
that count as “formal”? The most popular model for tracking studies
is a software program, because calculus-based mathematical models
that are easy to solve tend not to represent system nonlinearities
very well, whereas they can be built into software models if
necessary. The saving grace is that when control is good, the
Behavioural Illusion obscures the internal processing that is being
modelled so that simple models are about as good as you can usefully
get, as I explained in the attachment to [Martin Taylor
2017.03.28.23.36].

It depends on what you mean by "mathematics". "It becomes harder" is

presumably true, but you can’t tell for sure, because even at the
lowest levels it is very hard to model anything but a drastically
simplified version of what goes on inside the organism . Think, for
example if Bill’s “neural current”, which is a really drastic
simplification of the firing patterns of lots of neurons that in
some way change in ways that relate to changes in sensory inputs.
The neat thing is that the behaviour of simulations that don’t even
try to mimic individual neurons is remarkably similar to the
behaviour of organic controllers that (presumably) use the neuron
firing patterns. How one might simplify the issues that arise at
higher levels, which involve imagined perceptions along with
perceptions derived from current sensory input – that’s a question
nobody has addressed as far as I know. That doesn’t mean it’s not
open to mathematical investigation. Remember that Boole considered
his binary logic to be the “Laws of Thought”.

Alright. So what I’m hearing here is that the physical models (formal models, as Smaldino [2017] terms them) are simplified. What you say is, despite the simplification, useful information is still gleaned (again, this is something that Smaldino [2017] considers). Â

Â

      For example, it may not be useful to reduce an

individual’s perception of “United States politics in relation
to Canadian politics” over the course of a heated discussion
to a mathematical formula (we lose some important rich
subjective details?). I think it is even less possible that
this formula could then somehow track this relationship in the
local environment (like computerized tracking studies or
chasing choppers) . Am I wrong?

I'd say you are probably wrong in principle, but right in practice.

Most of the time.

Not clear what this means.

Â

Would a tracking poll on the several dimensions of

the difference between the two political regimes, done on
individuals rather than on aggregates, count for you as a track? The
individuals could be subject to different disturbances on the
different dimensions if you wanted to test whether they controlled
those perceptions. I doubt you would get much out of such a study,
but I’m wondering whether it would fit your criterion.

Â

Indeed–I am saying that here is an example of a control of a relationship perception, which could be disturbed over time. Unlike a more physical relationship–controlling a knot over a dot (rubber band example), something may be lost by simply describing this relationship perceptual control situation with mathematics. It reduces a complex rich experience of politics to be equivalent with the control of a rubber band. This bothers me–the two are very different, no?Â

      Many scientists, especially those in model-based

disciplines like physics and chemistry, might see less worth
in my claim that, at a certain point in the hierarchy, we can
no longer do the Test with formal models. My (short-lived at
this point) experience has been that some scientists
(excepting, at least, anthropologists) see qualitative
description as second to quantitative descriptions. Yet, I am
sure that at some point the mathematics becomes
overly-reductive. In other words, my rich experience of
considering those politics or caring about being an honest
person ought not to be relegated to a mathematical formula.
Any thoughts on this?

Yes. It depends again on what you mean by "mathematics" and "formal

models". See above. If mathematics in some case has “become overly
reductive”, quite probably the wrong tool has been used from the
toolbox. The surgeon used a laser when an infra-red lamp would have
been more better for the patient. But in general, the more precise
you can be, the better you can relate what you find to other areas
of your knowledge.

This is a fair point.Â

Â

      This is a bit of a long post. Nonetheless, I hope it sparks

some discussion.

It's a difficult topic. We will see.



Martin

Sorry, I don't have much time to read all the thread so here's just a quick comment on software being formal models.

I think too that software are formal model of pct or pct-like models.
I.e.: the object oriented programming model which is behind most of today's softwares usually use the "model-view-controller" model to build the software.

This is very related to control theory, we can see it if we add the other left-out component which is the "event".

So here we have a control loop with:
Environment = event
Input = view
Reference setting = controller
Output = event

So here we have softwares as control systems, seeing them as what they really are, a bunch of control loops.

Nicolas M. Kirchberger

[Bruce Nevin (20170427.1336 ET]
Excellent question, great discussion.
One of the reasons that it's difficult to model control of a variable at a high level (such as principle or system concept) is that the means of control are perceptual variables at successively lower levels whose reference values are not exclusively determined by error output from a comparator for that one high-level variable. More than one variable may receive a given perception as part of its input, and may affect the reference value for that perception as means of control.
Another reason is that our observational access to these higher-level perceptions depends a lot on what people say about them. This is at several removes from a rate of firing in a nerve bundle.
When we observe a tracking experiment, we can assume some kind of direct quantitative relationship (such as multiplication) between rate of firing, muscle tension, mouse movement, and position of cursor. The subject has a subjective experience of this, but we can ignore it. We don't mind ignoring it, because subjective experience, qualia, awareness, etc. are unexplained by PCT. But for the politics of Canada vs. those of the USA all we have is subjective experience and various ways of trying to quantify subjective experience, such as questionnaires, polls, and refinements of these which some psychologists are pleased to call "test instruments". These instruments have the virtue sometimes of helping to distinguish what people are controlling from their narrative about what they are controlling, or should be controlling, and that distinction is a case in point.

···

/Bruce