Maximization

[Bill_Powers1]

[From Bill Powers (2007.11.20.0920 MST)]

Martin Taylor 2007.11.19.17.38 –

Martin, I think you’re assuming that every interaction is a conflict. As
I think of conflicts, there is no problem until some variable is driven
to a limit and can’t increase any more, meaning that further disturbances
can no longer be resisted and control is lost. A budget constraint is
such a limit, but has no effect until one party finds that it’s
impossible to buy as much of several goods as desired because the money
runs out. I see that you have to assume that the amount of money
possessed by each party is always less than the amount wanted, or that it
would be less if the goods desired were brought to their reference levels
for consumption. I agree that this assumption is necessary to make
utility theory appear to work, but I don’t think it’s necessary on any
other grounds. However, we can certainly find out what happens if the
system does work that way.

The main problem here is trying to reason out what will happen in a
complex situation using unaided logic and words.I can’t follow your
complicated verbal arguments, and I’m not convinced (yet) that you can,
either. The best approach I know of is to construct a simulation of what
you’re talking about, set it in motion, and point to what it does. Then
there can be no arguments about how one is reaching conclusions. I’m
sorry if I’m not grasping what is obvious to you, but I really can’t see
how it all works.

If you want to postulate that people always try to increase their money
supply, or their consumption of goods either in quantity or in quality,
that’s fine, but put it into a simulation and show what the consequences
will be. First, of course, we have to agree on a model; then we can
postulate whatever details we want to try out.

In the evidently unimpressive simulation I offered some years ago, I
introduced budget constraints, reference levels for income saved, and
reference levels for goods purchased on the part of buyers; and inventory
control, costs of production, and desired capital reserves on the part of
sellers/manufacturers. Consumers were divided into those whose income
came from wages and those whose income came from capital investments (I
could have added a “mixed” category). I was planning on going
on to include multiple goods, capital goods, depreciation, multiple
manufacturers with overlaps of goods produced, and other complications
like a bank that made loans and charged interest, and wage negotiations,
but all I heard was that I couldn’t do it that way, or that it had
already been done, or that it wasn’t worth the effort because the
situation is too complicated to understand (or maybe, just for me to
understand, since I don’t have a degree in economics. Bill Williams, may
he rest in peace, could sometimes be a real pain in the butt). I got a
distinct impression that a number of people thought they had it all
figured out already, and simply knew that any approach that came out
different must be wrong anyway, so why bother? That’s the “economics
as religion” argument, which stirs up a lot of emotion.

Well, I’m not that easy to discourage.

Best.

Bill P.

[Richard_Kennaway]

[From Richard Kennaway (2007.11.21.1447 GMT)]

[From Bill Powers (2007.11.16.1103 MDT)]

Erling Jorgensen (2007.11.16 0900 EST) --

I'm starting to wonder whether control via a Kalman filter,
which is pretty far removed from effective corrective action
from the actual environment, might be a first-approximation
way to model System Concepts.

It would work at any level, wouldn't it? But I'm learning more about the Kalman Filter, because I feel a bit threatened by it. Google it and you'll get over a million hits. A lot of people think it's important.

What I'm finding is that it's basically a way to predict the next state of a system given very uncertain and noisy information about it. It gets so complicated mathematically that I can't see through to the underlying logic of this approach. But the idea that the information is very noisy makes me think this can't be useful for a model of organisms: look around and ask how noisy your experiences of the world are. It looks pretty darned noise-free to me. Of course at higher levels, particular from logic on up, there may be much more uncertainty, but you couldn't prove that by listening to Rush Limbaugh. There's nothing inherently noisy about logic, principles, or system concepts, though of course reorganization introduces random changes and not all perceptual input function work perfectly. But randomly noisy? I can understand "badly organized," but where the randomness comes from I can't imagine.

We can't get away from mental models, because we undeniably use them. But mainly we use them when we have plenty of time, and very few of them seem automatic and quick the way a control system is. I hope Richard K.takes up this subject, because I'm not feeling well-enough equipped to handle it.

Here's my go at explaining what a Kalman filter does. As Bill says, it's "a way to predict the next state of a system given very uncertain and noisy information about it".

It addresses the following situation: you are observing a process, and you have two sources of information, both unreliable. One is a model of the process, and the other is the observations you make of the process. Both have some inaccuracy. A practical example is GPS. There is a certain level of inaccuracy in the GPS signals (even the military-grade ones). You also have an inertial navigation system that gives you imperfect information about how you are moving. How can you combine the two sources of information to get the best estimate of your position? Navigation is a major application of Kalman filtering.

In the linear case, the model takes the general form:

     x(k+1) = A x(k) + B u(k) + w(k)

where
     x is the state of the process (in general a vector).
     u is some set of parameters of the process that you can set -- operating
         knobs and levers.
     A and B are matrices, constituting a model of the system.
     w is a random variable with zero mean and known standard deviation.

w is sometimes called "system noise", but what it really is is a model of the unmodelled behaviour of x.

For simplicity, let's take x to be a single variable, A=1, and ignore u and B:

     x(k+1) = x(k) + w(k)

This is a system which takes a random walk.

The observations are:

     z(k) = x(k) + v(k)

where v is the measurement noise, also assumed to have mean 0 and known standard deviation.

The Kalman filter says that you should estimate x by a series y given by:

     y(k+1) = y(k) + K (z(k+1) - y(k))

or equivalently:

     y(k+1) = y(k) (1-K) + z(k+1) K

K is called the Kalman gain, and says how much faith you should put in the observed value z(k+1) vs. your previous estimate y(k). (The process gets started by just picking any value for y(0).)

The mathematics of Kalman filters gives a way of computing a value of K that is optimal, in the sense of minimising the expected error. If the standard deviation of v is large compared with w, K should be small, so that you trust the model and only slowly update it from the noisy observations. If it is the other way round, K should be close to 1, putting more trust in the observations than in the model.

There are elaborations of this for multidimensional systems, non-linear systems, continuous-time systems, systems with time-varying models, ignorance terms (v and w) with known autocorrelation, and so on, but the fundamental idea is the same in all cases: using an observation to correct a prediction, both of known inaccuracy, in the best possible way.

It is possible, though perhaps not very illuminating, to look at the filter as a control system. The observations z are the reference signal, and y is the perception that the system tries to make track the reference. z(k+1) - y(k) is the error signal, K (z(k+1) - y(k)) is the output action, and y(k+1) = y(k) + output is the effect of the action on the perception y.

It can also be looked at as a low-pass filter, the turnover frequency chosen to optimally separate the measurement noise (v) from the unmodelled variation in the signal (w).

But the more important question is what role Kalman filters play in control systems, rather than vice versa. If you restore B and u to the equations, u is the action your controller is taking to affect the system. In the general case, z(k) is not x(k)+v(k), but C x(k) + v(k), where C is another matrix. C x(k) is then a controlled variable and z(k) a noisy measurement of it. The control problem is then to calculate what trajectory u should take in order to make C x(k) follow a desired trajectory. This is model-based control. The fundamental problem of model-based control is the imperfection of models. Kalman filters are a fix for that, given a model of the imperfections of the model. An example would be a cruise missile navigating by GPS.

I don't know what happens if there are imperfections in your model of the imperfections of the model.

I don't see any relevance of the Kalman filter to classical negative feedback control, other than as a means of noise reduction separate from the control problem. All the references I've found dealing with Kalman filters and control are about model-based control.

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

[Martin_Taylor2]

[Martin Taylor 2007.11.21.15.45]

[From Bill Powers (2007.11.20.0920 MST)]

Martin Taylor 2007.11.19.17.38 --

Martin, I think you're assuming that every interaction is a conflict.

No, only those in which there are fewer degrees of frredom for control than there are degrees of freedom for percepions to be controlled.

Other interactions may cause mutual disturbances and difficulty of control. As action vectors become more nearly aligned, the effective df are reduced (as in any standard engineering text -- a subtlety I chose to ignore in my recent message on conflict and tolerance), so mutual disturbance can shade into conflict. The boundary is not sharp.

In the evidently unimpressive simulation I offered some years ago, I introduced budget constraints, reference levels for income saved, and reference levels for goods purchased on the part of buyers; and inventory control, costs of production, and desired capital reserves on the part of sellers/manufacturers. Consumers were divided into those whose income came from wages and those whose income came from capital investments (I could have added a "mixed" category). ...

Well, I'm not that easy to discourage.

Good.

I remember you discussing the simulation. Was it written in a way that it can be run and manipulated on a non-Intel Mac? Is it on-line? If so, I wouldn't mind an opportunity to play with it.

Martin