Ratio control

[Bill_Powers1]

[From Bill Powers (2012.11.21.0951 MST)]

Rupert Young 2012.11.20 22.00 UT ]

RY: If we substitute o[t] in (2)
with the right-hand side of (1) we get,

i[t] := o[t-1] + g*e[t-1]*dt + d[t].

Yes? That is, the input to a new iteration is a function of the previous
output and a linear additive adjustment (g*e[t-1]*dt), and the
disturbance.

BP: No. I[t] is the current value of the input quantity, which is the sum
of the current values of o and d: i[t] = o[t] + d[t]. When you do the
substitution, the result is that you fail to update the value of o[t] for
use as o[t-1] in the next iteration, so you’ve left out the program step
that is written
o = o + gedt
I put the time subscripts in so it is clear that o on the left is the new
value of o, and the o on the right is the old value of the same
variable at a slightly earlier time: o[t] = o[t-1]*e[t-1]*dt. This is
a program step, not algebra. Algebra doesn’t know anything about previous
and current values of a variable. There is no time in algebra. It it were
algebra, you could solve the equation by subtracting o from both sides to
get

gedt = 0 ← that’s a “zero”.

That’s clearly not what the program step means.

My program steps were

(1) o[t] = o[t-1] + g*e[t-1]*dt /new output = old value plus (gain

• old error * dt)

Come to think of it, that time subscript is a time, not just an index. I
should have written it as

o[t] = o[t - dt] + g*e[t - dt]*dt

because the time between iterations is dt. When you write (t - 1) you are
tacitly saying that the unit of time is dt. The 1 means “1 unit of
time,” not just “one step.” Sp, keeping that in mind, I
can say:

The current value of o[t] will be the old value o[t-1] on the next
iteration. In the current iteration, the variables all have their current
values:

(2) i[t] := o[t] + d[t] /current input quantity = current output +
current disturbance

No time is required for the addition to be carried out and entered as the
new value of i[t].

RY: In ratio control,
however,

(3) i[t] := d[t] /
o[t] / new input quantity
= current disturbance / current output

BP: No. You’re thinking of what the equation would mean if its variables
were the logarithms of other variables. They aren’t. i[t] is not
d[t]/o[t]. It is just d + o. The perception is log(i), which means log(d

• o). The input i remains equal to (d + o).

RY:

and I am questioning whether equation (1) is valid in this situation,
which would give

(4) i[t] := d[t] / (o[t-1] + g*e[t-1]*dt)

BP: On the next iteration you will find that o[t-1] has the same value it
had in the previous iteration because you didn’t update o before using it
in the environment part of the equation.

Also –

It’s not correct to convert the environmental equation into antilogs. The
variables are just variables in the environment, not logarithms of
variables. Where you introduce antilogs is in translating the perceptual,
reference, and error signals into corresponding meanings in the
environment. This doesn’t change the environmental relationships into
antilogs.

RY: The point I am trying to
make is that equation (1) doesn’t appear to work with ratio control,
however, this does

(5) o[t] = o[t-1] + o[t-1]ge[t-1]*dt

Is this valid for ratio control?

BP: You have one too many o[t-1]'s in (5) – the second one. You still
seem to be treating these program steps like algebra. I don’t mean to be
beating up on you here; the reason I know what to say here is that I did
the same thing at first. That’s how I figured out that while you can use
algebra in some program steps – to expand an expression, or take out a
common factor from a parenthetical expression, for example – you can’t
do it across iterations in a simulation.
Step (5) seems to work because the effect is just like writing
o[t] = o[t-1] (1 + ge[t-1]dt)
You’re just adding an extra 1 to ge[t]*dt on every iteration. The result
will be to get too large a value of o, but unless you have the correct
value to compare it with you won’t realize it’s too large. Also, you’ll
find that the output doesn’t stop changing when the error signal is
zero.
If you want the perceptual signal to represent the ratio of d to o (as a
controlled variable), you have to have two logarithmic input functions,
on for d and one for o. You then subtract the second perception
from the first one to get the second-order perception equal to log(d) -
log(o), which is log(d/o). Now you can set up a second-order control
system for controlling that ratio, the error signal setting the reference
level (via an integrator) for log(o) which one of the first-order control
systems controls. And watch the signs so you have negative feedback
around both the first and second-order loops.

Note that the antilogs, in this last model, are automatically computed by
a high-gain negative feedback system. Negative feedback systems make the
output be the inverse feedback function of the input.

All this is very good practice at interpreting the meanings of equations
in simulations. It’s not totally simple. I was fortunate to have a genius
of a math teacher in my freshman year of engineering physics (which was
at the U of Colorado in Boulder in 1946-7), who spent a lot of time on
this very subject. He would start with a set of equations, do one step of
the derivation, and stop. “Now what does this equation mean?”
he would ask. We had to do this for every step to the end. He taught us
not to go through mathematical manipulations blindly, just following the
rules and not thinking. In simulations, every transformation of one
equation to another one produces a new set of meanings. He made us think
about them. I highly recommend learning to do this. It’s extremely
helpful in troubleshooting and can expose wrong steps in the
derivations.

Best,

Bill P.

[rupert]

[Rupert Young 2012.11.27 21.30 UT ]

Thanks, finally managed to get back to this. There a re a number of things under discussion here; the "standard" equations, ratio control and possible modified equations. For my simple brain can we take them one at a time? So, first, can we forget about ratio control and look at the standard equations?

RY: If we substitute o[t] in (2) with the right-hand side of (1) we get,
i[t] := o[t-1] + g*e[t-1]*dt + d[t].
Yes? That is, the input to a new iteration is a function of the previous output and a linear additive adjustment (g*e[t-1]*dt), and the disturbance.

BP: No. I[t] is the current value of the input quantity, which is the sum of the current values of o and d: i[t] = o[t] + d[t]. When you do the substitution, the result is that you fail to update the value of o[t] for use as o[t-1] in the next iteration, so you've left out the program step that is written
o = o + g*e*dt
...
Come to think of it, that time subscript is a time, not just an index. I should have written it as
o[t] = o[t - dt] + g*e[t - dt]*dt

RY: First a note, why is this e[t-dt] rather than e[t]? Isn't it the error we have just computed in the previous step, that is, the current error rather than the old error?

RY: Ok, if we think of things as iterative steps and start the loop, then we have,

(1) i[t] = o[t] + d[t]
(2) e[t] = r[t] - p[t] (assuming p=i)
(3) o[t] = o[t - dt] + g*e[t - dt]*dt

here we reach the end of one iteration, update t and start again,

(4) i[t] = o[t] + d[t]
(5) e[t] = r[t] - p[t] (assuming p=i)
(6) o[t] = o[t - dt] + g*e[t - dt]*dt

How is o[t] in step (4) computed? Isn't it o[t] from the previous step, i.e. the left-hand side of (3), which is a sum of two elements, a previous output and an adjustment according to the error?

So, we could say then that i is a function (a sum) of the output (from 2 steps ago) and an adjustment according to the error (from the previous step) and the current disturbance: so, algebraically i = f(o) +f(e) + d. That was the point I was trying to get to in my previous email.

Am I still barking up the wrong tree?

Regards,
Rupert

[Bill_Powers1]

[From Bill Powers (2012.11.27.1625 MST)]

Rupert Young 2012.11.27 21.30 UT --

RY: First a note, why is this e[t-dt] rather than e[t]? Isn't it the error we have just computed in the previous step, that is, the current error rather than the old error?

BP: The rate of change or velocity of the output is proportional to the error signal. That is, as a calculus statement, do/dt = g*e.

When you multiply do/dt by dt, you're extrapolating to the next value of the output as if the output will be changing at that same constant rate during the next interval dt. This is wrong because the error signal is not exactly constant during the iteration, but not far wrong when dt is small, and becomes the truth if you let dt approach zero. The amount of change is the rate of change times the length of time. Therefore the output will change by an amount g*e*dt. The new value of the output at time t + dt will be its value at time t plus the amount of change during the interval dt.

The only place where time enters into this model is in the output function. There are no delays or time variables in any other function or signal path. So everything from the new value of the output around the loop and back to the error signal entering the output function happens in zero time. The time between iterations therefore applies only to the output function's time-integration. You can put multiple time functions anywhere, of course, if you treat them properly as we're doing here.

The logical place to imagine where an iteration starts is the time when a new value of the output apppears. At time zero, once we have o[0] we can compute the input quantity from i = o+d, then the perceptual signal from p = i (unless we want a constant of proportionality there), then the error signal from e = r - p, and finally the new rate of change of output from do/dt = g*e. No time has passed yet because the model has no functions that continuously change value with time in these parts. Finally we have come to the integrating output function where time does enter, and we can compute the next value of the output at the start of a new iteration: o[dt] = o[0] + g*e[0]*dt. The next time around, we will be computing o[2dt] = o[dt] + g*e[dt]*dt, then o[3dt] = o[2dt] + g*e[2dt]*dt, and so on, step by tiny step.

RY: Ok, if we think of things as iterative steps and start the loop, then we have,

(1) i[t] = o[t] + d[t]
(2) e[t] = r[t] - p[t] (assuming p=i)
(3) o[t] = o[t - dt] + g*e[t - dt]*dt

here we reach the end of one iteration, update t and start again,

The mistake here is in not updating the output time (after step 2) to t+dt. After the error signal is computed, a new rate of change of output exists, so the next value of output, o[t+dt], will be the current value, o[t], plus the current rate of change times the duration of dt. In step 3 you used the previous values of o and e instead of the current values, so you recomputed the current value of o rather than the next value.

From here on the reasoning is off the track because time has not increased. Unless you add something to t or do something else to change it, the time remains the same. Perhaps this would be clearer if we updated t after step 3:

dt = 0.01, start with t = 1.00:

RY: (2) e[t] = r[t] - p[t] (assuming p=i) // t = 1.00
    (3) t = t + dt. // t = 1.01
    (4) o[t] = o[t - dt] + g*e[t - dt]*dt // t - dt = 1.00

BP: Now in step 4, t refers to the time at the start of the next iteration, so to use the results from the previous iteration in computing it you now have to use t - dt as the index because we have changed t.

An algebraic symbol is just a number, and if nothing is done to it to change it, it remains the same number.

Am I still barking up the wrong tree?

Yes, but now I think you will be getting the scent of the 'possum more clearly. This is all part of getting used to doing simulations. Everyone has to go through it, so don't be discouraged. This confusion is all quite normal. Once you understand what is being done, it will all make sense to you.

Best,

Bill P.

P.S. I now realize that strictly speaking I should be using parentheses rather than square brackets in all this. Square brackets in programming are conventionally used to index successive entries in arrays, and the numbers in them are assumed to be integers. Parentheses indicate the arguments of functions, which is more like what we're doing here. I feel guilty for leading you into imitating my sloppiness out of pure faith. Thank you, but don't do that. I am a guru with feet of clay. Don't imitate me until you have understood for yourself.

[rupert]

[From Rupert Young 2012.11.29 21.00 UT]

Ok, I was thinking of the time updating after the output function, as that is how I have things (implicitly) set up in my programs, that is each iteration starts with computing the input and ends with the output. Whereas you are talking about the time only being relevant to the element that is being changed internally to the control system.

In practice I don't think this changes the way I have been computing standard control systems in my programs as I don't explicitly represent a time change (dt = 1). However, I will try this out in spreadsheets in future models.

The general point I had been trying to make was that the adjustment to the output (which in turn affects the input) was of a linear nature, and didn't seem to work with (the way I had set up) ratio control. However, this may no longer be relevant with reference to the ratio control model you are suggesting.

So if we could get back to that:

BP: If you want the perceptual signal to represent the ratio of d to o (as a controlled variable), you have to have two logarithmic input functions, on for d and one for o. You then subtract the second perception from the first one to get the second-order perception equal to log(d) - log(o), which is log(d/o). Now you can set up a second-order control system for controlling that ratio, the error signal setting the reference level (via an integrator) for log(o) which one of the first-order control systems controls. And watch the signs so you have negative feedback around both the first and second-order loops.

Just to clarify then, you are suggesting two control systems the higher of which has,
* two inputs log(d) and log(o) (which is the input to the lower order system), used to compute the perceptual signal by log(d) - log(o)
* a reference which is log(r), where r = d/o,
* an output representing log(o), which is the reference for the lower order system,
and the lower which has,

* as a perceptual signal log(o),
* a reference from the higher system,
* an output integrator, which computes o, which is then fed as input to the perceptual function.
and error functions as normal?

Regards,
Rupert

[Bill_Powers1]

Hi, Rupert –

[From Rupert Young 2012.11.29
21.00 UT]

Ok, I was thinking of the time updating after the output function, as
that is how I have things (implicitly) set up in my programs, that is
each iteration starts with computing the input and ends with the output.
Whereas you are talking about the time only being relevant to the element
that is being changed internally to the control system.

BP: If there is no time function in a component of a control system
model, the output of the component changes as the input changes, with no
delay. In an integrator, the output’s rate of change changes
exactly in time with variations in the error signal – but the integrated
value of the output changes only by (do/dt)*dt during one iteration. When
the series of computations that makes up one iteration passes through
that function, the time change occurs. Time advances by one unit of
dt.

…

RY: The general point I had been trying to make was that the adjustment
to the output (which in turn affects the input) was of a linear nature,
and didn’t seem to work with (the way I had set up) ratio control.
However, this may no longer be relevant with reference to the ratio
control model you are suggesting.

So if we could get back to
that:
BP: If you want the perceptual signal to represent the ratio of d to o
(as a controlled variable), you have to have two logarithmic input
functions, on for d and one for o. You then subtract the second
perception from the first one to get the second-order perception equal to
log(d) - log(o), which is log(d/o). Now you can set up a second-order
control system for controlling that ratio, the error signal setting the
reference level (via an integrator) for log(o) which one of the
first-order control systems controls. And watch the signs so you have
negative feedback around both the first and second-order loops.

Just to clarify then, you are suggesting two control systems the higher
of which has,

• two inputs log(d) and log(o) (which is the input to the lower order
  system), used to compute the perceptual signal by log(d) - log(o)
• a reference which is log(r), where r = d/o,
• an output representing log(o), which is the reference for the lower
  order system,
  BP: the reference signal r represents the desired value of
  [log(d) - log(o)], or log(d/o). So yes.

RY: and the lower which has,

• as a perceptual signal log(o),
• a reference from the higher system,
• an output integrator, which computes o, which is then fed as input to
  the perceptual function.
  and error functions as normal?

Yes, that sounds almost right. Remember that it’s not what you
know that matters in a model, it’s what the model knows. Here the model
is just trying to control the difference between one perceptual signal,
which it calls o, and a second signal, which it calls d. It has no sensor
to tell it what kind of input function there is, so it’s just controlling
the difference between two numbers, and bringing that difference to some
reference value which may or may not be zero.

If the reference signal is zero, the system will try to make the two
signals equal, and to an external observer that will look like equality
of d and o. The d signal is equal to log(d), the o signal to log(o), so
if the logs are equal, the arguments are equal.

A nonzero reference signal r will tell the second-order system to make
the difference between the d signal and the o signal be equal to r.
Log(o) - log(d) = r, or log(d/o) = r. The external observer will see d
and o being maintained in the condition o/d = antilog(r). The
second-order error signal can become a positive reference signal
for the controller of log(d) or a negative reference signal for the
controller of log(o), or both.

Because the slope of a log function increases with the value of the
argument, the loop gain of this ratio control system will increase as r
increases. Using an integrating output function will put off instability
until the gain gets high enough to cause digital oscillations due to the
noncontinuous nature of the iteration process.

Starting to make sense?

Best,

Bill

···

At 09:04 PM 11/29/2012 +0000, Rupert Young wrote:

[rupert]

[From Rupert Young 2012.12.04 21.00 UT]

BP: Starting to make sense?

Yes, I believe so. I hope to try this out soon, probably in the New Year now, so I'll let you know how it goes. Thanks for your help.

Regards,
Rupert