Defining; testing; defining

[From Bill Powers (940724.1730 MDT)]

Bob Clark (940724.1408 EDT) --

Just consider this system diagram:

       ====================================
x ---> || INTEGRAL A*2X/DT^2 + B*DX/DT +CX || --> y
       ====================================

To me, this expression expresses the "dynamic characteristics" of a
system with parameters "A," "B," and "C." As the general form of a
linear, second order differential equation, it is familiar, and has
the "solutions" that you present as "flow-chart versions."

To me, this "box" is not a "system diagram." To be such a diagram, at
least two boxes are required.

In my book, too. I meant only to illustrate what one box in a system
diagram would look like. The rest of the boxes would be defined
similarly. That seems to be something of a side-issue.

Dictionary:

"system, n. 1. an assemblage or combination of things or parts
"forming a complex or unitary whole. etc."

Thus a "system diagram" is composed of a set of "boxes" and
"interconnections among boxes." The "contents" of the boxes and
locations of the interconnections determine the overall dynamics of
the assembly.

This definition is inadequate for systems analysis. A tower made of
assorted children's blocks, or a painting, would satisfy it. It seems to
be based on superficial appearance rather than function.

Such a diagram can apply to, but need not be limited to, closed loop
situations.

Yes. It can apply to almost anything that's not merely a random jumble.

System diagrams, as they are used in B:CP and network discussions,
are consistent with this definition.

That is mainly because the definition is so broad as to be useless. It
classes together things like control systems, S-R systems, houses,
symphonies, and mountains.

Such a diagrams can also be interpreted as representing a temporal
sequence of events within the system.

Not the diagram reproduced above. I think you're missing my point. There
is a superficial visual resemblance between a system diagram and a flow-
chart. Conceptually, however, they are totally different. In a system
diagram, for example, there is no implication of temporal sequence in
the arrows connecting blocks. All the blocks are assumed to be in effect
all of the time.

I overlooked the difference between your phrase, "how a system
behaves," and the dictionary definition:

"flow chart, 1. also called FLOW SHEET, a detailed diagram or chart
"of the operations and equipment through which material passes, as in
"a manufacturing process. _Computer Technol._ a graphic
"representation, more detailed than a flow diagram, of a sequence of

"operations in a computer program."

and:

"flow diagram, _Computer Technol._ a schematic representation, less
"detailed than a flow chart, of a sequence of subroutines, indicating
"the general flow of information for solving a problem by a computer."

Both of the definitions, of "flow chart" and of "flow diagram,"
emphasize a "sequence" of "operations." This is a "temporal"
sequence: A before B, before C, etc.

I think my definitions of system diagrams and flow charts are preferable
to those from the dictionary that you cite. At least I prefer them. The
distinction between "chart" and "diagram" is completely arbitrary. And
you will note that in both definitions, there is implied a progression
through the diagram, as if one operation takes place at a time. That
aspect of the definition I agree with. In a system diagram, no such
step-by-step progression is implied. In a system diagram, we do not have
A before B before C: we have A, B and C all in effect at the same time.

If the time intervals involved are considered small compared to time
variables of interest, the operation diagrammed can be regarded as
"instantaneous." Thus a flow chart can be regarded as describing the
structure of a system, that is: a "system diagram."

I disagree. A flow chart is about the behavior of variables. A system
diagram is about the forms of functions and their interdependencies.

An open loop system is intended to provide an individual with
improved control of his environment. That is, it is intended to
improve an individual's ability to keep his perceptions of his
environment close to his desired perceptions.

When the individual user is included in the system, such open loop
systems serve as portions of the user's output function. Thus they
support the users own ("negative feedback") control systems. The
user completes the system by providing the feedback function, the
comparator and the reference variable.

Agreed.

···

-------------------------------------------------------------------
Bill Leach (940724.13:59 EDT) --

I am led to believe that the "MMPI" (Minosota Multiphasic Personality
Inventory) is actually a "decent test" Of course I have that "belief"
without much justification.

Does anyone out there have any actual correlations between MMPI scores
and performance scores on anything?

I have nothing against any test that gives usable results and does not
harm the individual through the uses made of it. I believe in the
usefulness of statistical analysis when appropriately employed. I am not
an anti-statistics ideologue. All I am doing, when I point out flaws in
the uses of statistics, is presenting what I believe to be the facts of
the matter, without introducing any gratuitous opinions as far as I
know. If you believe that misclassifying 20 or 30 or 40 percent of the
people who take a test is acceptable, be my guest. All I ask is that
there be a public acknowledgement of this bare fact. If everybody can
just agree on the actual consequences of using mass measures as the
basis for evaluating individuals, the rest of the debate is in the area
of ethics or morality or politics. Perhaps somebody can make a
convincing case for discounting the importance of the
misclassifications. One such case can be made on the basis of the

principle that the good of the organization must come before the good of
the individual -- you can't make an omelet without breaking eggs. I
don't happen to share that view, but that's a different argument.

...by the way, for what ever it is worth, in that industry questionable
results from the MMPI are the bases for an interview with a
psychiatrist.

Nice. Then when you apply for health insurance, how will you answer
questions like "Have you ever been referred to a mental health
professional on the basis of a diagnosis of mental disorder?" If you
answer "yes", you will be turned down because of a pre-existing
condition. If you answer "no" you will have lied on your application,
having affirmed the truth of a statement you know to be untrue. If that
is discovered, you will lose all possibility of being covered for
anything.

It also makes sense that "industry" would want to have a means of
determining in advance if they are potentially hiring a person with a
strong reference for killing others or a reference for something like
"whatever you can get away with is OK".

Do you think that an intelligent psychopath who believes that whatever
you can get away with is OK or who likes to kill people would hestitate
to lie about his beliefs and preferences on a pyschological test? Such
tests, of course, do not contain questions like "Is it OK to do anything
you want if you're not caught?" or "Do you like to kill people?" That
would obviously invite lying. Psychopaths are not stupid. So what is
done instead is to try to ask questions which get at underlying
motivations and tendencies indirectly, by asking about innocent-seeming
things that don't reveal what the tester is trying to discover.
Unfortunately, the indirectness of the questions and their very
innocuousness leaves room for incriminating answers that are actually
quite innocent of base motivations.

Since some psychopaths do sometimes answer certain innocent questions in
a particular way, anyone who answers the same questions in the same way
becomes tainted with "indications" of psychopathic "tendencies." And as
soon as such test scores come back, interviewers begin to perceive the
person who took the test not as just another applicant, but as a
possible psychopathic personality. Now every hestitation or indication
of nervousness takes on a new meaning, supporting the initial impression
created by the test score. Even reluctance to answer certain questions
like "How often did you masturbate when you were in high-school" is put
down not to embarrassment, but to evasiveness; these more "penetrating"
questions don't even get asked unless the test score arouses suspicions.
The interviewer then puts down in the record, "Subject uncooperative."

But the most damning entry is "Referred for psychiatric evaluation."
Look at what has happened in recent political campaigns to candidates
who admitted or were discovered to have undergone psychological
counseling. Instead of the news media treating this as an indication
that the candidate is probably now in better mental condition than most
of the other candidates, they treat it as if the candidate had some sort
of permanent disability: nobody gets psychotherapy if they aren't crazy,
do they?

These are the wheels you set in motion when you misevaluate an
individual by using psychological tests producing scores that are less
than perfectly correlated with whatever they are supposed to indicate.
Whether you continue, knowing these consequences, to evaluate people by
using tests that you know for certain will misclassify a certain
substantial percentage of them depends on your system of values. If your
loyalty is to some entity called "the company" or "the organization,"
then you will place the good of the company above the problems you

create for some minority of individuals. A dispassionate decision is
made more difficult by the fact that almost invariably it is "the
organization" that pays you for administering the test and making the
evaluations. The decision can't be free of self-interest.

What I am trying to do by laying out the facts about the reliability of
test scores and the consequences of misevaluating people is to bring
something to awareness that will create a painful conflict inside most
people in this sort of occupation. I can't tell anyone how to resolve
this conflict; all I can do is point it out. I have, in fact,
considerable confidence that any ordinary person who becomes aware of
this conflict will try quite hard to resolve it much as I would do. But
there's no predicting what reorganization will produce.

So what can PCT say to the personel manager? What direct, ready to
implement, practical improvements does PCT offer this person? I am not
being sarcastic here and am quite serious. This manager has "data"
that shows that what he is doing now improves his chances of success in
the process marginally but enough to make a significant difference "to
the bottom line".

PCT says that you will choose actions that bring your world as close as
possible to matching _all_ the standards you have chosen as desirable.
It says that in the face of serious conflict, you will suffer errors
that will probably lead to reorganization, and that when the
reorganization is successfully completed, you will find your world
closer than before to the state you prefer, at all levels. PCT says that
you will control your own world better, in the long run, if you perceive
it as accurately and consistently as you can.

PCT is not in the business of making people feel better about doing
things they themselves recognize as conflicting with their own
standards.

It is all well and good to tell him how bad his current system is and
he might even agree with you but unless you can offer him something
workable then all he will do is listen (if that).

If the personnel manager recognizes the problem and experiences a
conflict of values because of this recognition, I have done all I can.
How to resolve that conflict is up to the personnell manager: a very
private affair. Some managers will say "If I don't do it, somebody else
will," thus renouncing responsibility for the consequences of their
actions. Some will say "All those people who slip through the cracks
just have to work out their own destinies." Some will quit their jobs,
or campaign against psychological testing until they win or get fired.
Some will say, "It's a tough world and I'm not about to give up this
position because some do-gooder wants me to hire any sort of riff-raff."
As I say, there's no predicting the outcome of reorganization, or even
whether reorganization will start.

That's not a problem I can solve.
----------------------------------------------------------------------
Hank Folson (940724) --

On defining terms:

"On" refers to "concerning" or "having to do with," not a spatial
relationship.

"Defining" means offering sentences which refer to experiences of a
different detailed sort than the terms actually being defined (a
recursive definition).

"terms" refers to arbitrary words, not to lengths of time in office or
in jail.

"Refers to" means "being used to refer to"; that is, to point in some
way or indicate, or call to attention, or choose as a subject matter.

"concerning" means -- uh -- referring to, or having to do with.. Uh...

"not" means negation, applied to whatever statement follows.

"spatial" means having to do with space, or linear dimensions.

"relationship" means -- uh -- relationship.

"having to do with" means "in some way concerned with" or "in some way
referring to" or

"arbitrary" means chosen for no definable reason.

"reason" means ....

"means" means ..

I give up.
--------------------------------------------------------------------
Best to all,

Bill P.

<[Bill Leach 940725.02:24 EST(EDT)]

[ Bill Powers (940724.1730 MDT)]

Does anyone out there have any actual correlations between MMPI scores
and performance scores on anything?

That is a bleak response. I figured that if anyone might have
"substantive data" about something like that, I would find out here.

...by the way, for what ever it is worth, in that industry questionable
results from the MMPI are the bases for an interview with a
psychiatrist.

Nice. Then when you apply for health insurance, how will you answer
questions like "Have you ever been referred to a mental health
professional on the basis of a diagnosis of mental disorder?" If you
answer "yes", you will be turned down because of a pre-existing
condition. If you answer "no" you will have lied on your application,
having affirmed the truth of a statement you know to be untrue. If that
is discovered, you will lose all possibility of being covered for
anything.

No, that is not the case. You are not "referred", you will just be
interviewed. I have had that happen a couple of times and on one
occassion the psych specifically stated that I was just one of the random
selections. On another, the psych stated that he wanted to interview me
because that validity factor was exceptionally high.

The test itself is rather fascinating and I wonder about some of the
methodology employed in constructing such a test. For one thing, I
understand that the almost repeating question sequences are very
significant in evaluating the validity of the results. Because of this,
people that "follow the instructions" and take the test a rapidly as
possible by answering with the immediate thought normally have very high
"validity."

I understand that some sequences of questions are changed just slightly
because people WILL normally answer certain questions differently
depending upon the previous questions. The point being of course to
confound someone that intentionally answers questions in a way that is
perceived to be "correct". In this case the "results" might be quite
normal but the "validity" would be very low.

Of course this is what I have been told about the test or have been able
to determine from what little literature I has seen on this subject.
Naturally, now I recognize that the test must be based upon statistical
"norms" and would therefore be subject to possible serious error when
applied to any specific individual.

Not wanting to sound like a skeptic (but I am), I would likely be rather
reluctant to have much "faith" in the reported accuracy of the test
anyway. There are only a couple of occupations where I know that it
is/was routinely applied. A definate problem with accuracy statistics in
such cases is that even though there have been at least tens of thousands
of people take that exam, it is quite probable that the common nature of
the occupation has a great deal to do with the "values" of the people
that apply for such work.

Additionally, just how much effort is expended in trying to determine
that a person that was rejected actually did exhibit the "undesired"
behaviour at some time much less evaluate whether such would have been
likely if the person had been hired.

Have any of these sorts of tests been given to many thousands of people,
each of whom was studied for many years following the test, but without
even revealing the results to anyone? I realize that this is still
statistical but at least the test itself would not then be an
environmental disturbance capable of actually producing some of the
behaviour that it is supposedly "predicting".

Whether you continue, knowing these consequences, to evaluate people by
using tests that you know for certain will misclassify a certain ...

Yes, I find that I am very much in agreement with you. We may well
disagree some on the acceptability of using such "tools" but that does as
you said move into the moral and ethics realm.

What is not (as you stated) correct is to treat such tools as though they
are perfect when in fact they are flawed.

I have a good example, though with a different sort of test. I have
never been a "drug user" even though my generation is one of the "better
known" in that regard and many of my friends did use drugs. The nuclear
power industry is the first commercial industry to have employed
universal "drug screening". I remember being quite annoyed by the
prospect but was, of course, completely unworried. I was recovering from
financial disaster, had just moved and recently began work on a power
plant startup.

I took the "drug screening" test along with several hundred other people
that day, again feeling very uneasy about the entire process but not
personally worried. A few days later, a security guard found me on the
site and escorted off site. I was told that the test had come up
"positive" and that I was no longer employed with no recourse.

I really did not quite know what to do (I did not really know anyone in
the area) but decided that I needed to get my own test performed and I
was fortunate in the fact that the doctor that I contacted was one of the
few experts on the subject at the time.

I also called the local NRC (even though I did not think that there was
anything that they could or would do about the matter) but it turned out
that the regional director was in town, happened to have answered the
phone when I called AND was an old friend that I had worked with for many
years.

I ended up being called back in for a retest (the no recourse position
was appearently quite illegal -- at least then) and passed. Also, the
titration type test run on my own request showed that the results of the
first test had to have been in error.

Though ultimately, the whole matter cost me a few hundred dollars (and
might not have cost anything if I had been willing to deal with an
attorney but I have a "thing" about working with sub-human but intelligent
beings, so I did not try to recoup my costs), the entire experience has
left me with VERY bad feelings about such methods. And this is even with
a test that has some relationship to physical reality.

PCT is not in the business of making people feel better about doing
things they themselves recognize as conflicting with their own
standards.

I don't think that this is the issue. Assume for the moment, that said
personnel manager has high standards for morality and ethics. Assume
also that such manager believes that his ethics demand that he really
attempt to hire the best possible person for a particular job and not
reject a candidate improperly (that is for reasons not related to the
requirements of the position).

Then, as a practical matter, can PCT help this person develop a more
reliable evaluation/interview methodology?

I think that the answer is yes, but only theoretically at this point.

-bill

Tom Bourbon [940725.1240]

<[Bill Leach 940725.02:24 EST(EDT)]

[ Bill Powers (940724.1730 MDT)]

Does anyone out there have any actual correlations between MMPI scores
and performance scores on anything?

That is a bleak response. I figured that if anyone might have
"substantive data" about something like that, I would find out here.

If that was your hope, you really "struck out," Bill! (I gave an equally
bleak reply in a post that I see has not yet shown up on the net.)
. . .

Skipping through most of your post, we come to a fascinating section that
reveals the degree to which the "mental health" and "psyhoclogical
measurements" professions have helped create a clear and present social
danger.

Not wanting to sound like a skeptic (but I am), I would likely be rather
reluctant to have much "faith" in the reported accuracy of the test
anyway. There are only a couple of occupations where I know that it
is/was routinely applied. A definate problem with accuracy statistics in
such cases is that even though there have been at least tens of thousands
of people take that exam, it is quite probable that the common nature of
the occupation has a great deal to do with the "values" of the people
that apply for such work.

Additionally, just how much effort is expended in trying to determine
that a person that was rejected actually did exhibit the "undesired"
behaviour at some time much less evaluate whether such would have been
likely if the person had been hired.

Have any of these sorts of tests been given to many thousands of people,
each of whom was studied for many years following the test, but without
even revealing the results to anyone? I realize that this is still
statistical but at least the test itself would not then be an
environmental disturbance capable of actually producing some of the
behaviour that it is supposedly "predicting".

Bill, assessment "instruments" like the MMPI were not designed to evaluate
people for employment; they were intended to serve as aids in the assessment
of "psychopathy," a newer name for what used to be called "mental illness"
or "abnormal psychology." The norms were on people in "mental" institutions
in Minnesota. Questions were shuffled and changed until certain ones of
them were answered in a particular way by some acceptable (to the test
makers) proportion of institutionalized people who had already been placed
in one or another officially-recognized (American Psychiatric Association
Diagnostic and Statistical Manual -- DSM) diagnostic category. Multiple
"categories" of previous diagnoses were included, hence the "multiphasic"
part of the name of the test. It is bad enough that clinicians use it with
candidates for diagnostic labels; how inexcusably odious is it that some
people use it as a way to screen candidates for employment!

Later,

Tom

<[Bill Leach 940725.22:35 EST(EDT)]

Tom Bourbon [940725.1240]

... helped create a clear and present social danger.

Do you enjoy Tom Clancy too?

It is bad enough that clinicians use it with candidates for diagnostic
labels; how inexcusably odious is it that some people use it as a way to
screen candidates for employment!

Lets see how "clever" you are... Do you have any idea who first began
using this "intrument" for employment screening?

Give up?

The U.S. Government of course. It was used on candidates for people that
would work in close proximity to the President. I believe that the
Military Intelligence organizations were next (yea, I know, that is an
oxymoron but that is the lable) and then I think that the CIA started
using it (though it is actually likely that either the CIA or the FBI
were actually first.

I believe that the "justification" whipped up for "these important"
sensitive jobs was then used along with strong coercive ability that the
NRC can have against the nuclear power utilities "to encourage" them to
take such a simple "precaution".

Cookie from my last log in:

Mythology, n.:
        The body of a primitive people's beliefs concerning its
        origin, early history, heroes, deities and so forth, as
        distinguished from the true accounts which it invents later.
                -- Ambrose Bierce, "The Devil's Dictionary"

Rick especially should enjoy that one :slight_smile:

-bill