Modeling learning

[From Rick Marken (951229.0830)]

Shannon Williams (951228)---

In _The Philosophy of Artificial Intelligence_, Andy Clark briefly
describes PDP (Parallel Distributed Processing).

I see a couple of ways to change the system:

1) Since a reference level exists, the unit should continuously 'adjust the
weights on the hidden units' until the correct output is generated...

5) I can see atleast one way that the reference generator can evolve: Let
the first letters be input...

What do you think of this project?

It's hard for me to tell from your verbal description. Could you provide a
simple diagram of what you would like to do?

Here's what I think you might mean:

                 r
                 >
                 v
            ---> C --- e
            p |
            > adjust
            > k weights
          > s | |
            ^ |
            > >
            > v
       [i-->|o = sum(ki)| -->o]

This control system adjusts the weights (k values) of a linear pattern
recognizer (o = sum(ki)). The weights of the pattern recognizer are to be
adjusted so that inputs (i, sound patterns) are converted into appropriate
outputs (o, phonemes ). The control system must perceive the relationship
between i and o; so the control system's perceptual function, s, is a
relationship recognizing function.

Let's say that s is designed so that when the relationship between any i and
any o produces a perceptual value of zero (0) the relationship is what a
linguist considers the "correct" mapping of sound input to phonemic output;
non-zero perceptual values measure the discrepency of the relationship from
this "correct" mapping. This way the reference input to the control system
can be fixed at 0. The output of the perceptual function, p, is quantitative
so that there is always a quantitative error signal, e = r-p. Non-zero error
values are turned into adjustments to the k weights of the pattern
recognizer.

This system would continuously adjust the k weights of the pattern recognizer
until the output of the control system's perceptual function, s, (the measure
of relationship between i and o) is at the reference value (0), if such a
solution exists.

Of course, the tough part of designing this automatic "learning" control
system is designing 1) the function, s, that perceives the relationship
between i and o and 2) the error-driven weight adjustment algorithm -- though
this could probably be an E. coli style random process.

Is this close to what you had in mind?

Best

Rick

[From Shannon Williams (951230)]

Rick Marken (951229.0830)--

Here's what I think you might mean:

                r
                >
                v
           ---> C --- e
           p |
           > adjust
           > k weights
         > s | |
           ^ |
           > >
           > v
      [i-->|o = sum(ki)| -->o]

This control system adjusts the weights (k values) of a linear pattern
recognizer (o = sum(ki)). The weights of the pattern recognizer are to be
adjusted so that inputs (i, sound patterns) are converted into appropriate
outputs (o, phonemes ). The control system must perceive the relationship
between i and o; so the control system's perceptual function, s, is a
relationship recognizing function.

I do not know how the pattern is getting recognized nor how it is getting
updated. All I know is that Rosenberg and Sejnowski have found a way to do
this, and I want to use whatever method that they have found.

To me, Andy Clark's description of Rosenberg and Sejnowski's project can be
drawn like this:

                              -----------> Voice Synthesizer
                              >
                              >
              ------------ |
              > Learned | |
    input --->| pattern | --- reference
  (letters) | generator| | |
              ------------ | |
                   /|\ | \|/
                    > -----------> C ---------------
                    > >
                    > >
                    ------[pattern adjustment algorithm]----

Of course, the tough part of designing this automatic "learning" control
system is designing 1) the function, s, that perceives the relationship
between i and o and 2) the error-driven weight adjustment algorithm -- though
this could probably be an E. coli style random process.

I think that Rosenberg and Sejnowski have already designed a function "s"
and they already have a workable adjustment algorithm. (I will know for
sure when my books get in). The reason that their system is not a
"learning" control system is because they have hard coded the system's
reference. In other words, Rosenberg and Sejnowski are using their own
natural language capabilities as the system's reference.

I will not explain here why it is wrong for Rosenberg and Sejnowski to
hardcode their own knowledge as the system's reference. Bill P. just sent
a wonderful post which explains why. I just want to say that the part of
this little system that remains to be designed is the designing of a
reference that is generated by the system instead of by Rosenberg and
Sejnowski.

Because of this goal, I think that the system should look something like
this:

    input (supervisor says a/some letters) --------------------------
                                                                  >
                                                                 \|/

···

----------
                                                              >Black |
                             -------> [Voice Synthesizer]---->|Box |
                             > >which |
                             > >converts>
              ------------ | ----------- |sounds |
              > Learned |___|__/ ___\ |Generated| |to |
     -------->| pattern | | / |reference| |letters |
     > > generator>---- ----------- ----------
     > ------------ | | |
     > /|\ | \|/ |
     > > ------------> C ------------- |
     > > > >
     > > > >
     > ----[pattern adjustment algorithm]---- |
     > >
     > >
     --------------------------------------------------------------

There are a few ways that this could work:

1) The system 'knows' whether the output of the pattern generator comes
from a supervisor. If it does come from a supervisor, then it is used as a
reference, else it is used as a perception.

2) The system 'knows' whether it generated the output of the pattern
generator. If it did generate the output, then it is used as a perception,
else it is used as a reference.

3) The first output generated by the pattern generator is used as a
reference. All subsequent generator outputs are used as perceptions. When
the perception finally matches the reference then the reference is cleared
and the loop is broken.

-------------------------------------------------------------------------

To me, the most important design goals are: 1) The reference must be
generated by the system 2) The system must be attempting to match its
perception to its reference.

Does this explain any better what I am visualizing? Do you think it would
work?

-Shannon

BTW- the way that I understand the pattern generator to work, it must have
new/varying input in order to improve. For example, say it learns to
output '*&^%$#@' when the letters 'Rick' are input. It then learns to
output "@@##$$%" when the letters 'Marken' are input. When it now
receives the input 'Rick' again, the pattern will not be '*&^%$#@'. It
will be something a little bit different.

I think this is significant because if animals have a similar system then
it would explain why we dislike monotony. We would need to have new stimuli
in order to improve our preceptions or references. And our perceptions and
references are improved by new stimuli even though the new stimuli are not
related to the particular perceptions/references that we what to improve.