Modelling & Prediction

[Avery Andrews 960928]

An interesting-looking item from the comp-neuro list:

From Sat Sep 28 10:05:54 1996

Return-Path: <>
Received: from by (4.1/SMI-4.0)
        id AA05506; Sat, 28 Sep 96 10:05:52 EST
Received: from ( []) by (8.7.6/8.6.12) with SMTP id KAA05443 for <>; Sat, 28 Sep 1996 10:05:49 +1000 (EST)
Received: from ( by (4.1/SMI-4.0)
        id AA18927; Fri, 27 Sep 96 17:16:57 PDT
Received: from by (4.1/SMI-4.1)
        id AA29015; Fri, 27 Sep 96 17:16:40 PDT
Message-Id: <>
Status: OR


Date: Fri, 27 Sep 96 17:16:40 PDT
From: (Comp-Neuro Mailing List)
Subject: Tech Report: Visual Cortex as a Hierarchical Predictor

Date: Thu, 26 Sep 1996 15:07:05 -0400
From: Rajesh Rao <>
Subject: Tech Report: Visual Cortex as a Hierarchical Predictor

The following technical report on a hierarchical predictor model of
the visual cortex and the complex cell phenomenon of "endstopping" is
available for retrieval via ftp.

Comments and suggestions welcome (This message has been cross-posted -
my apologies to those who received it more than once).

Rajesh Rao Internet:
Dept. of Computer Science VOX: (716) 275-2527
University of Rochester FAX: (716) 461-2018
Rochester NY 14627-0226 WWW:


            The Visual Cortex as a Hierarchical Predictor

                 Rajesh P.N. Rao and Dana H. Ballard

                       Technical Report 96.4
   National Resource Laboratory for the Study of Brain and Behavior
        Department of Computer Science, University of Rochester
                          September, 1996


   A characteristic feature of the mammalian visual cortex is the
reciprocity of connections between cortical areas [1]. While
corticocortical feedforward connections have been well studied, the
computational function of the corresponding feedback projections has
remained relatively unclear. We have modelled the visual cortex as a
hierarchical predictor wherein feedback projections carry predictions
for lower areas and feedforward projections carry the difference
between the predictions and the actual internal state. The activities
of model neurons and their synaptic strength are continually adapted
using a hierarchical Kalman filter [2] that minimizes errors in
prediction. The model generalizes several previously proposed
encoding schemes [3,4,5,6,7,8] and allows functional interpretations
of a number of well-known psychophysical and neurophysiological
phenomena [9]. Here, we present simulation results suggesting that the
classical phenomenon of endstopping [10,11] in cortical neurons may be
viewed as an emergent property of the cortex implementing a
hierarchical Kalman filter-like prediction mechanism for efficient
encoding and recognition.

Retrieval information:

FTP-pathname: /pub/u/rao/papers/

20 pages; 302K compressed.

The following related papers are also available via ftp:

Dynamic Model of Visual Recognition Predicts Neural Response Properties
                        In The Visual Cortex

                 Rajesh P.N. Rao and Dana H. Ballard

                   (Neural Computation - in press)


The responses of visual cortical neurons during fixation tasks can be
significantly modulated by stimuli from beyond the classical receptive
field. Modulatory effects in neural responses have also been recently
reported in a task where a monkey freely views a natural scene. In
this paper, we describe a hierarchical network model of visual
recognition that explains these experimental observations by using a
form of the extended Kalman filter as given by the Minimum Description
Length (MDL) principle. The model dynamically combines input-driven
bottom-up signals with expectation-driven top-down signals to predict
current recognition state. Synaptic weights in the model are adapted
in a Hebbian manner according to a learning rule also derived from the
MDL principle. The resulting prediction/learning scheme can be viewed
as implementing a form of the Expectation-Maximization (EM) algorithm.
The architecture of the model posits an active computational role for
the reciprocal connections between adjoining visual cortical areas in
determining neural response properties. In particular, the model
demonstrates the possible role of feedback from higher cortical areas
in mediating neurophysiological effects due to stimuli from beyond the
classical receptive field. Simulations of the model are provided that
help explain the experimental observations regarding neural responses
in both free viewing and fixating conditions.

Retrieval information:

FTP-pathname: /pub/u/rao/papers/

43 pages; 569K compressed.


        A Class of Stochastic Models for Invariant Recognition,
                         Motion, and Stereo

                 Rajesh P.N. Rao and Dana H. Ballard

                       Technical Report 96.1


  We describe a general framework for modeling transformations in the
  image plane using a stochastic generative model. Algorithms that
  resemble the well-known Kalman filter are derived from the MDL
  principle for estimating both the generative weights and the current
  transformation state. The generative model is assumed to be
  implemented in cortical feedback pathways while the feedforward
  pathways implement an approximate inverse model to facilitate the
  estimation of current state. Using the above framework, we derive
  models for invariant recognition, motion estimation, and stereopsis,
  and present preliminary simulation results demonstrating recognition
  of objects in the presence of translations, rotations and scale

Retrieval information:

FTP-pathname: /pub/u/rao/papers/

7 pages; 430K compressed.


Anonymous ftp instructions:


Connected to
220 FTP server (Version wu-2.4(3)) ready.

Name: [type 'anonymous' here]
331 Guest login ok, send your complete e-mail address as password.

Password: [type your e-mail address here]

cd /pub/u/rao/papers/

[From Bill Powers (960928.0615 MDT)]

Avery Andrews 960928 --

An interesting-looking item from the comp-neuro list:

Thanks, Ave. I'll look up the papers.


Bill P.

[Hans Blom, 961011b]

(Avery Andrews 960928)

Trying to catch up. Thanks for your reference to:

An interesting-looking item from the comp-neuro list:
Subject: Tech Report: Visual Cortex as a Hierarchical Predictor

An interesting paper (and there are more interesting papers at the
site where this comes from; thanks!). What I miss in this paper is
the aspect of control. The author talks about recognition tasks but
tends to forget that recognition is _for a purpose_: how to behave or
act. Apparantly the subjects weren't able to _do_ much. If so,
recognition of what happens does not seem very urgent.

What appeals to me is the description of the brain as a "hierarchical
predictor". It is the prediction aspect that I miss in PCT, so this
paper presents what seems to me a necessary complement. What I like
and recognize in the paper is that the author describes behavior as
"prediction error minimization". The link to PCT becomes most clear
when one replaces the author's term "expectation" by the PCT term
"reference level". These two seem to play indistinguishable roles.

Where the author's view and PCT need to come together, I think, is by
understanding that one's expectation is a function of one's actions.
If one is unable to act, the laws of nature (and the behavior of
others) determine what to expect. If one is able to act, however, one
can influence the state of the world -- and hence modulate one's
expectations of what is going to happen through one's actions.

The author says, in effect, that we try to see our expectations come
true. PCT says that we try to see our wishes come true. The question
is now: is there a difference? Are our wishes indeed identical to our
expectations? In a control sense, they are, I think: if one chooses
an action such that the expectation (which is a function of the
action) coincides with the wish. This accurately describes what a
control system does...