[From Rick Marken (2016.01.09.1715)]
On Thu, Dec 29, 2016 at 3:53 AM, Warren Mansell email@example.com wrote:
WM: Jeff Vancouver has just produced a wonderfully elegant computational model of the complex effects of self-efficacy based on PCT. It is also particularly exciting because it includes an imagination loop for the first time!Â
RM: According to Vancouver and Purl, their model is motivated by a paper by Powers (1991) Commentary on Bandura’s “Human Agency”, American Psychologist. Specifically, they say: "…Powers suggested that computational modeling, which PCT facilitates, might help clarify self-efficacyâs roles in human behaviorâ?. I got a copy of the Powers’ 1991 comment and the closest thing to a suggestion that computational modeling might help clarify self efficacy’s roles in human behavior was the following: Â âControl theory is best used and tested through the method
of modeling or simulation. To devise working models, however, and especially to
test them, one needs experimental data that are more reliable than the
customary results of research in the social sciencesâ?.Â
RM: Since computational models are a form of working model, Vancouver and Purl are correct to say that Powers suggested the use of computational models, though he suggested this as the way to test control theory, not self-efficacyâs roles in human behavior. But leaving that aside, Vancouver and Purl seem to have ignored the last part of Bill’s recommendation about modeling: "To devise working models, however, and especially to test them, one needs experimental data that are more reliable than the customary results of research in the social sciencesâ?. Â Instead of testing their model against highly reliable experimental data, Vancouver and Purl test their model against the data obtained in an experiment reported by Schmidt and DeShon (2010)The Moderating Effects of Performance Ambiguity on the Relationship Between Self-Efficacy and Performance,Â J applied psychologyÂ which provides data that are anything but reliable.Â Schmidt and DeShon conducted a typical psychology experiment – a factorial experiment – where the goal was to see Â whether the effect of one independent variable (self-efficacy) on some dependent variables (performance, effort) depends on the value of another variable (ambiguity).Â
RM: There are two problems with testing a model against the data in the Schmidt and DeShon (2010) study. First, the results are averages over many subjects; PCT models are typically tested by comparing them to the behavior of one individual at a time. Second, and more important, is the fact that the results in these experiments are highly unreliable. The predicted interaction between ambiguity and self-efficacy is statistically significant (p<.01) but the reliability of this prediction (measured as R^2) is .03; only 3% of the variance in either dependent variable was accounted for by the interaction between ambiguity and self-efficacy. So the PCT model is being used to explain a very weak effect (indeed, essentially random noise) and one that is not seen in a large proportion of the subjects tested; and in the subjects where the effect is seen, it is must usually be a very small effect, if it’s there at all.Â
RM: Moreover, Vancouver and Purl don’t provide any measure of goodness of fit of the model to the Schmidt and DeShon data. What they show is that their model produces a qualitative match of the average results of the Schmidt and DeShon. We don’t consider a PCT model to be a successful model of behavior unless it accounts for about 90% of the variance in the observed behavior and comes within about 3% of that behavior of each individual tested.
RM: There are also problems with the Vancouver and Purl PCT model itself (Figure 2 in their diagram). One of the main problems is that it is not clear what the controlled variables. One is “Task Progress” Â which is apparently a combination of information ambiguity, task state, effort and self - efficacy. But I didn’t find this variable defined in the paper; that is, there is no description of how Â these inputs combined to produce a perception of “task progress”.Â This is a big omission since this variable seems to be central to the behavior of the model. The other controlled variable is not labeled but the input to the perceptual function that defines this variable is “dynamic expected utility of the task”. This input is the output of the other control system; therefore it’s not clear to me how this “dynamic expected utility” variable is controlled.Â
RM: There are other problems with the model but the basic problem with the application of PCT in this situation is that it is not being used to explain controlling. The term “controlled variable” is nowhere to be found in either the Vancouver and Purl or the Schmidt and DeShon paper. Once again, the problem for PCT comes down to the fact that PCT explains a phenomenon – control, also known as purposeful behavior – to which conventional psychological researchers areÂ essentially blind.Â
RM: Apparently J. Applied Psychology will occasionally publish Commentaries on published papers. I’m thinking of writing one on the Vancouver and Purl paper. Should I or would someone else like to do it.
Richard S. MarkenÂ
"Perfection is achieved not when you have nothing more to add, but when you
have nothing left to take away.â?
Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â --Antoine de Saint-Exupery
RM: I appreciate Jeff’s enthusiasm for PCT but I think his model (described in Vancouver, J.D. & Â Purl , A. (2016) Computational Model of
Self-Efficacy’s Various Effects on Performance: Moving the Debate Forward, The
Journal of applied psychology) gives a misleading impression of what PCT is about.Â