Insect flight as the control of visual perception

[from Gary Cziko 2007.02.10 0915 CST]

I haven’t been able to fid the full article yet on Current Biology (http://www.current-biology.com/), but the summary sure sounds interesting at:

http://www.eurekalert.org/pub_releases/2007-02/cp-abf020107.php

including:

In their new work, the authors hypothesized that insects rely on a
visual feedback loop, termed an optic-flow regulator, to assess the
ratio of groundspeed to height, and to maintain that ratio by
controlling their vertical lift. To test this hypothesis, the
researchers engineered a fly-by-sight micro-helicopter that
incorporated control elements based on the idea of an optic-flow
regulator, and found that the robot was indeed capable of achieving
impressive aspects of insect flight, including takeoff, level flight,
and landing.

The findings, explain the authors in their paper,
help illuminate the basis for a number of previously unexplained
observations regarding insect flight, including the fact that many
insects descend in a headwind and ascend in a tailwind, and that
honeybees often drown when flying over mirror-still water.

–Gary

···

Gary Cziko
Professor
Educational Psychology & English as an International Language

University of Illinois
1310 S. Sixth Street
210F Education Building
Champaign, IL 61820-6990
USA

Telephone +1-217-333-8527
Fax: +1-217-244-7620
e-mail:
g-cziko@uiuc.edu
Web: http://garycziko.net
Skype: garyjazz (http://www.skype.com)
Google Talk: gcziko
Amateur Radio: N9MJZ

Founder, Station Manager, Program Director, Chief Engineer and Announcer

The Latino Radio Service of La Casa Cultural Latina
1660 AM on the Urbana-Champaign campus of the University of Illinois
(http://latinoradioservice.org)

Founder and Major Contributor

ATALL Wikibook for
Autonomous Technology-Assisted Language Learning
(http://en.wikibooks.org/wiki/ATALL)

[From Rick Marken (2007.02.11.1040)]

Gary Cziko ( 2007.02.10 0915 CST )

I haven’t been able to fid the full article yet on Current Biology (http://www.current-biology.com/ ), but the summary sure sounds interesting at:
http://www.eurekalert.org/pub_releases/2007-02/cp-abf020107.php

including:

In their new work, the authors hypothesized that insects rely on a visual feedback loop, termed an optic-flow regulator, to assess the ratio of groundspeed to height, and to maintain that ratio by controlling their vertical lift. To test this hypothesis, the researchers engineered a fly-by-sight micro-helicopter that incorporated control elements based on the idea of an optic-flow regulator, and found that the robot was indeed capable of achieving impressive aspects of insect flight, including takeoff, level flight, and landing.

This sounds very interesting. Of course, by building the robot they are implicitly modeling things correctly. The robot must be controlling the perceived ratio of groundspeed to height (gs/h) by varying vertical lift (the output) as necessary to protect this variable from disturbance. So they must have built into the robot a circuit that set’s the reference for gs/h. It would be interesting to know if the reference is fixed or varied as the means of controlling some other variable.

By the way, I’m trying to figure out a way to clearly insert my comments into my e-mails using Outlook – which makes things a lot tougher than my Mac Mailer. I’m reconsidering my desertion of the Mac, but each system has its plusses and minuses. If there is a real world out there it sure can be a pain in the ass.

Best

Rick

[From Richard Kennaway (2007.02.12.1038 GMT)]

[from Gary Cziko 2007.02.10 0915 CST]

I haven't been able to fid the full article yet on Current Biology (<http://www.current-biology.com/&gt;http://www.current-biology.com/\), but the summary sure sounds interesting at:
<http://www.eurekalert.org/pub_releases/2007-02/cp-abf020107.php&gt;http://www.eurekalert.org/pub_releases/2007-02/cp-abf020107.php

It's there now. PDF version at
http://download.current-biology.com/pdfs/0960-9822/PIIS0960982206026662.pdf

Also reported in New Scientist at

Author's web page with more papers on optical flow control:
http://www.laps.univ-mrs.fr/~ruffier/
(which takes a very long time to load, due to the presence of some unnecessarily large pictures -- if you can set your browser to not download pictures it will be much faster)

···

--
Richard Kennaway, jrk@cmp.uea.ac.uk, Richard Kennaway
School of Computing Sciences,
University of East Anglia, Norwich NR4 7TJ, U.K.

[from Gary Cziko 2007.02.12 10:30 CST]

[From Rick Marken (2007.02.11.1040)]

This sounds very interesting. Of course, by building the robot they are implicitly modeling things correctly. The robot must be controlling the perceived ratio of groundspeed to height (gs/h) by varying vertical lift (the output) as necessary to protect this variable from disturbance. So they must have built into the robot a circuit that set’s the reference for gs/h. It would be interesting to know if the reference is fixed or varied as the means of controlling some other variable.

The authors say on page 4:

The insect’s OF [optic-flow] set-point might
depend on either innate, internal, or external factors.

So it looks like they haven’t made the HPCT insight, but at least appear open to it.

Rick, do you want to mention this possibility to them?

–Gary

···

On 2/11/07, Rick Marken marken@mindreadings.com wrote:

[From Bill Powers (2007.02.12.0945 MST)]

Richard Kennaway (2007.02.12.1038 GMT) --

I should point out that Christian Heider, of the European CSG, has been interested for many years in the possibility of controlling optical flow as a way of controlling locomotion. Could he be part of the group working on this insect project?

Best,

Bill P.

[From Rick Marken (2007.02.12.1000)]

Now I’m trying to use gmail and I find it completely confusing. But here goes.

Rick Marken (2007.02.11.1040)

So they must have built into the robot a circuit that set’s the reference for gs/h. It would be interesting to know if the reference is fixed or varied as the means of controlling some other variable.

The authors say on page 4:

The insect’s OF [optic-flow] set-point might
depend on either innate, internal, or external factors.

So it looks like they haven’t made the HPCT insight, but at least appear open to it.

Rick, do you want to mention this possibility to them?

I got a copy of the paper and took a quick look at it. I knew that if they got the helicopter to fly then they must have built it correctly (as an input control system) but I was pleasantly surprised to see that they even seem to have gotten the diagram right.

I will definitely show this to my class as something someone might want to write a paper on (may be a bit too complex though). I may try to contact them if I ever figure out how to use my email again.

Best

Rick

···


Richard S. Marken
rsmarken@gmail.com
marken@mindreadings.com

[from Gary Cziko 2007.02.12 1230 CST]

···

On 2/12/07, Richard Marken rsmarken@gmail.com wrote:

[From Rick Marken (2007.02.12.1000)]

I got a copy of the paper and took a quick look at it. I knew that if they got the helicopter to fly then they must have built it correctly (as an input control system) but I was pleasantly surprised to see that they even seem to have gotten the diagram right.

I will definitely show this to my class as something someone might want to write a paper on (may be a bit too complex though). I may try to contact them if I ever figure out how to use my email again.

It may be interesting for your students to see how the authors use the word “control.” They seem to get it right sometimes, but then they talk about things like controlling height to maintain the optic flow when they should really be saying that the bugs (and their robot) vary height to control the perception of optic flow.

–Gary

[From Rick Marken (2007.02.12.1115)]

[from Gary Cziko 2007.02.12 1230 CST]

[From Rick Marken (2007.02.12.1000)]

I got a copy of the paper and took a quick look at it. I knew that if they got the helicopter to fly then they must have built it correctly (as an input control system) but I was pleasantly surprised to see that they even seem to have gotten the diagram right.

I will definitely show this to my class as something someone might want to write a paper on (may be a bit too complex though). I may try to contact them if I ever figure out how to use my email again.

It may be interesting for your students to see how the authors use the word “control.” They seem to get it right sometimes, but then they talk about things like controlling height to maintain the optic flow when they should really be saying that the bugs (and their robot) vary height to control the perception of optic flow.

–Gary

Great point. I saw that use of “controlling” in the abstract but forgave it because I figured they did the right thing (which they did). But it would make a good pedagogical point.

Best

Rick

···

On 2/12/07, Richard Marken < > rsmarken@gmail.com> wrote:


Richard S. Marken
rsmarken@gmail.com
marken@mindreadings.com

[From Erling Jorgensen (2007.02.13 0945 EST)]

Rick Marken (2007.02.12.1115)

Gary Cziko 2007.02.12 1230 CST

It may be interesting for your students to see how the authors use the
word "control." They seem to get it right sometimes, but then they talk
about things like controlling height to maintain the optic flow when they
should really be saying that the bugs (and their robot) vary height to
control the perception of optic flow.

Great point. I saw that use of "controlling" in the abstract but forgave it
because I figured they did the right thing (which they did). But it would
make a good pedagogical point.

Aren't you guys making an artificial distinction here? Isn't it possible
to vary height in a controlled way, by varying the reference signal going
to that control system?

I know we used to use the formulation of using a varying means to achieve
a controlled end, but wasn't that a pedagogical convention to stress the
means-ends relationship? The danger of such a formulation is that it
forgets that the means are achieved by perceptual control as well. It's
controlled perceptions all the way out the hierarchy, even if some of them
are tracking a varying reference signal. The "varying" part is really just
a relative distinction anyway, compared to signals that have a slower time
constant, and so are temporarily more stable.

Just thought we should clean up this formulation, so we don't leave the
impression that we use open-loop or non-controlled means to achieve the
higher level controlled ends.

All the best,
Erling

[From Rick Marken (2007.02.13.1115)]

Erling Jorgensen (
2007.02.13 0945 EST)

Rick Marken (2007.02.12.1115)

Gary Cziko 2007.02.12 1230 CST

It may be interesting for your students to see how the authors use the
word “control.” They seem to get it right sometimes, but then they talk

about things like controlling height to maintain the optic flow when they
should really be saying that the bugs (and their robot) vary height to
control the perception of optic flow.

Great point. I saw that use of “controlling” in the abstract but forgave it
because I figured they did the right thing (which they did). But it would
make a good pedagogical point.

Aren’t you guys making an artificial distinction here?

Gee, I hope not.

Isn’t it possible

to vary height in a controlled way, by varying the reference signal going
to that control system?

Yes. If they meant that the system varies the reference for height, which is itself a controlled variable, as the means of controlling for optical flow, then all would be well. This is what they might have meant if there robot were, indeed, a two level control system, controlling height as the means of controlling optical flow.

But, based on their diagram, this doesn’t seem to be the case. It looks like they have a one level control model that varies two lower level variables, forward thrust and vertical lift, as the means of controlling two perceptual variables, pitch angle and optical flow. So their model is apparently not setting references for lower level inputs as the means of controlling pitch and optical flow. So I think they were wrong to have said that their robot control thrust and lift (height) to control optical flow, because it doesn’t.

I think what both Gary and I were reacting too was the penchant of some
control theorists to talk about “control” in terms of outputs rather
than inputs, even when they are building robots or models that actually
are controlling their input perceptions. I would make it a lot easier
for PCT if the people who are in the best position to promulgate it
would get the message right. Saying that their robot “controls” height in order to maintain optical flow gives the wrong message about what their system is doing; it is controlling perceptions, not outputs. I think how we communicate ideas can be just as important as the implementation of those ideas. I think it’s great that Francheschini et al build a robot that moves around by controlling a perception of optical flow (and pitch angle). But I also think it would be better for those of us trying to introduce a new way of looking at behavior if they consistently made the point that the behavior of their robot is the control of perception.

Best

Rick

···


Richard S. Marken
rsmarken@gmail.com

marken@mindreadings.com

[From Erling Jorgensen (2007.02.19 2230 EST)]

I'd like to revisit the article by Franceschini, N., Ruffier, F., &
Serres, J. (2007). A Bio-Inspired Flying Robot Sheds Light on Insect
Piloting Abilities,Current Biology 17, 1–7, February 19, 2007.
Available at --
<http://download.current-biology.com/pdfs/0960-
9822/PIIS0960982206026662.pdf>

Erling Jorgensen ( 2007.02.13 0945 EST)

Rick Marken (2007.02.13.1115)

Isn't it possible
to vary height in a controlled way, by varying the reference signal
going to that control system?

Yes. If they meant that the system varies the reference for height, which
is itself a controlled variable, as the means of controlling for optical
flow, then all would be well. This is what they might have meant if
their robot were, indeed, a two level control system, controlling height
as the means of controlling optical flow.

You're right, Rick. I stand corrected.

My point was obviously a relic from some CSG netherworld, where we used
to talk about "varying" the means to achieve "controlled" ends, leaving
the impression that the means were not themselves often being controlled
as well, in relation to varying reference signals at those lower levels.
I can see that this does not seem to be the case with the micro-helicopter
model for insect flight in the Franceschini, et al. study.

When I first looked at the article, I noticed that they were defining
Optic Flow as Ground Speed divided by Ground Height, even including that
function in their schematic diagram (Figure 2). I had thought that they
must be constructing the perception of Optic Flow out of those lower
level perceptions, and thus would have to be measuring and controlling
them as well. But apparently, they were able to construct and measure
a stand-in perception for Optic Flow, derived from an Elementary Motion
Detector, and control it that way.

But, based on their diagram, this doesn't seem to be the case. It looks
like they have a one level control model that varies two lower level
variables, forward thrust and vertical lift, as the means of controlling
two perceptual variables, pitch angle and optical flow. So their model
is apparently _not_ setting references for lower level inputs as the
means of controlling pitch and optical flow. So I think they were wrong
to have said that their robot _control_ thrust and lift (height) to
control optical flow, because it doesn't.

I agree. The micro-helicopter model described in the article only
measures Optic Flow, in terms of Angular Velocity. It is not fully
clear from the article, but it seemingly also has a way to determine
whether Pitch Angle is matching the commanded reference of 10 degrees
off vertical. In terms of its own mechanisms, those are the only two
perceptions it can control.

I believe additional variables do end up being controlled, but those
have to take into account the larger system comprised of the helicopter
plus the human operator acting as a control system. In other words, the
"hierarchical" features come from the human operator helping to set the
references for at least two or three layers of HPCT control.

Figure 2 of Franceschini's article seems to use Norbert Wiener's style of
diagramming the control loop, which for me seems to collapse a lot of the
variables together. To get a clearer sense of them, I needed to rearrange
things into a HPCT style of diagram, which I have attached at the end of
this post. (There is one I produced with the Draw feature of MS Works,
version 8, & then I scanned it & saved the image as an Adobe .pdf file,
in case folks did not have access to MS Works.)

I tried to make everything explicit, instead of leaving gaps for our
imagination to (conveniently) fill in. I also had to keep asking myself
which variables were means for the achievement of other variables, thereby
placing them lower in the hierarchy, and which variables were actually
being measured, and by whom.

I really like the form of argumentation that Richard Kennaway used in his
"A Simple and Robust Hierarchical Control System for a Walking Robot,"
(available at -- <http://www2.cmp.uea.ac.uk/~jrk/Robotics/rk-c2004.pdf&gt; ),
where he analyzes Bill Powers' inverted pendulum model. I'll borrow a bit
of that style in trying to explain the diagram I came up with.

The human operator is interested in constructing a model that can mimic
insect flight characteristics in several ways, such as with take off,
landing, level flight, and terrain following. A stationary micro-
helicopter [MH]-- (seemingly with the initial vertical lift of its rotors
just balancing the weight of the MH) -- is doing none of that, with
neither Forward Velocity nor Height giving any visual appearance of
matching insect flight.

If we could control Height and Forward Velocity, then we might be able
to reproduce such aspects of insect flight. If the model had a sensor
for Height, it could control it directly. We don't have such a sensor,
but the Optic Flow of features passing below the MH can be defined as
the helicopter's Ground Speed relative to its Height off the ground.
We do have a way to approximate Optic Flow as Angular Velocity, and
control it relative to a reference setting. (Actually, the Angular
Velocity is itself approximated as a voltage, derived by two photo-
receptors feeding into an elementary motion detector, with volts
calibrated to the perceptual angle traversed by the MH.)

In order for there to be any Angular Velocity, there must be Forward
Velocity, and the output of the Optic Flow regulator cannot bring that
about. Forward Velocity can be achieved if there is Forward Acceleration.
Forward Acceleration could be brought about via Rotor Pitch, but the
micro-helicopter has no autonomous way of initiating Rotor Pitch.
However, a human operator can control the degree of Rotor Pitch, by
ramping it forward ten degrees from vertical, to initiate Forward Thrust,
and then ramping it back to zero degrees, to bring the MH to a stop.
The Forward Thrust generates Air Speed, which when summed with the
disturbance of the Wind Speed, leads to a perception of Ground Speed.
The helicopter does not directly perceive Ground Speed, but the graphs
shown in Figure 3 of the article indicate that the human operator is
monitoring it.

Once there is Ground Speed, there is a way for it to affect the MH via
the changing Ground Texture underneath the MH. As mentioned above, an
elementary motion detector is able to derive a perception of Angular
Velocity, which can be made to match a reference for Optic Flow. To
maintain Optic Flow constant when there is Ground Speed, the Ground Height
must be increased accordingly. This can be achieved by adjusting Rotor
Speed, which generates Vertical Lift as its output. We don't know how
much to vary Rotor Speed, but we could make it proportional to the error
in the Optic Flow. The resulting Flight Altitude offsets any local
changes in Relief Altitude (simulated in some studies by a ramp and
drop off underlying the Ground Texture), to control Ground Height as
needed. Again, while the MH does not perceive Ground Height, the graphs
of Figure 3 show that the human operator is monitoring it. Indeed, the
Monitored Flight Parameters of those graphs are important perceptions
to the human operator, showing that the MH's take off, landing, level
flight, and terrain following constitute a fairly good match to insect
flight characteristics.

What is interesting is that both Ground Speed and Ground Height end up
getting controlled for the human operator, even though the micro-
helicopter perceives neither one directly. For the MH, it seems only
Angular Velocity and Pitch Angle are controlled directly, and I have
indicated that in the diagram I derived by using a bold-italic font for
those variables.

It is also interesting the circuitous path of the variables involved in
the Optic Flow (i.e., Angular Velocity) control loop. Angular Velocity
is constructed via a motion detector, which is constructed out of a lens
and photoreceptors differentially recording Ground Texture when there is
Ground Speed, which is itself traceable to the Pitch of the Rotor. The
output of the Optic Flow regulator does not affect any of those variables
directly, but instead alters the Rotor Speed, which in turn generates
Vertical Lift to control the Ground Height of the MH for the human
operator, who is the one ramping the Rotor Pitch up or down ten degrees,
as a means of controlling Forward Velocity, which ultimately affects the
monitoring of Angular Velocity.

It is quite a striking system that Franceschini, et al. have devised,
which can mimic several important features of insect flight, while only
perceiving and controlling a couple of variables. There is a hierarchical
aspect of their scheme, consistent with the linkages of Hierarchical
Perceptual Control Theory. But those aspects draw in the perceptual
control of the human operator, to complete the requisite control loops.

The diagram is attached below, in both .wps (MS Works for Windows, v.8)
and .pdf (Adobe Acrobat, v.7.0) formats.

All the best,
Erling

CDocuments and SettingsCompaq_AdministratorMy DocumentsEJPCT ThreadsPCTdiag_Franceschini.wps (43 KB)

CDocuments and SettingsCompaq_AdministratorMy DocumentsEJPCT ThreadsPCTFranceschiniDiag.pdf (21.4 KB)

[From Rick Marken (2007.02.20.0825)]

Erling Jorgensen (2007.02.19 2230 EST)--

I'd like to revisit the article by Franceschini, N., Ruffier, F., &
Serres, J. (2007). A Bio-Inspired Flying Robot Sheds Light on Insect
Piloting Abilities,Current Biology 17, 1�7, February 19, 2007.

Wow, that's a really nice diagram. It must have taken quite some time
to make it. So I hate to be critical -- and maybe the diagram is
correct, based on your more careful reading of the article-- but
shouldn't the output of the optical flow control system also go to the
forward thrust output? It currently goes only to the vertical lift
output, with the human operator specifying the level of forward
thrust. Also, shouldn't vertical lift (along with forward thrust)
influence the ground texture that is transduced by the
lens/photoreceptor input function?

Still, it's a really nice piece of work; particularly useful is the
fact that the diagram shows what the diagram in the article didn't
show, which is the remote settings of references supplied by the
people on the ground.

Nice work, Erling.

Best

Rick

···

--
Richard S. Marken
rsmarken@gmail.com
marken@mindreadings.com

Erling Jorgensen (2007.02.20 1300 EST)]

Rick Marken (2007.02.20.0825)

shouldn't the output of the optical flow control system also go to
the forward thrust output? It currently goes only to the vertical lift
output, with the human operator specifying the level of forward
thrust.

I think you're right about this. I don't have the article in front
of me at the moment, but I seem to remember the fine print of the
model acknowledging a Mean Flight Force, which it partitioned into
Vertical Lift and Forward Thrust components. But it also made the case
that Vertical Lift was many more times significant, because of the
small angle of 10 degrees off vertical of the Rotor Pitch. I can't
fully reconcile that with the observation that Ground Speed increased
substantially, up to 3 meters/second, whereas Height only changed by
one meter (seemingly taking about 15 seconds to do so, if I was
extrapolating off the graph right). But then I don't understand all
the dynamics, and obviously gravity has to be counteracted by Lift,
with only Wind Speed being counteracted by Forward Thrust.

It's possible that the black box variables labelled in the article
"Heave dynamics" and "Surge dynamics" capture some of the interaction
of Rotor Speed and Rotor Pitch. These seemed to me to be environmental
feedback functions, in my diagram.

Also, shouldn't vertical lift (along with forward thrust)
influence the ground texture that is transduced by the
lens/photoreceptor input function?

I am a bit confused by how both those variables are interacting (or
embodied?) with the Ground Texture. The article spoke of a "one
dimensional" input to the photoreceptors. The authors simulate contour
on the ground with a series of lines and shadings perpendicular to the
direction of flight. Seemingly, the two photoreceptors are able to
register the time interval between a texture line crossing first one
and then the other line of sight of each receptor, which is then
transduced in an elementary motion detector into voltages corresponding
to subtended angles, to yield Angular Velocity.

None of that gets perceived, it seemed to me, without some Forward
Velocity across the texture lines. So that is why I have Ground Speed
and Ground Texture being combined in an environmental variable, but I
am not sure summation is the right way to describe that combination.

When considering the potential impact of Height on Ground Texture, I
don't think the photoreceptors are registering "degree of fuzziness,"
or some such thing, and converting that into an accounting for the
height off the ground. What is crucial is the realization that the
same Ground Speed can yield half as much Angular Velocity if the Ground
Height is twice what it was before, (I hope I'm saying that right.)
In other words, Angular Velocity is inversely proportional to Ground
Height, and that must be taken into account in some way.

This isn't the mathematical way of putting it, but the conversion of a
directional velocity into an angular velocity introduces a confound,
because it makes all the difference how far out the angle the traversed
distance is being measured. That is part of how an angle is constructed.
The way to make the measured Angular Velocity invariant with respect to
height (i.e., to control it relative to a fixed set point) is to adjust
the height up or down accordingly. So I think it is in the calibration
of Rotor Speed to the Optic Flow error, that Ground Height gets accounted
for because it thereby gets controlled.

So in answer to your (second) question, Rick -- No, I don't think there
is a direct influence of Vertical Lift on what is being measured in
terms of Ground Texture. Rather, it affects the _placement_ of the
lens and photoreceptors by controlling Ground Height, and in that way
calibrates Angular Velocity to Ground Speed in the right manner.

Good, insightful questions, Rick. Thanks for the "positive feedback."

All the best,
Erling