Critique of pure reason

CSGnet Archive CSG1993
1

[From Bill Powers (930113.0730)]

Martin Taylor (930112.1830) --

You have indeed managed to introduce a disturbance, in your paper
on Ockhams's Razor. I might entitle my comments, "A critique of
pure reason."

In principle, any hypothesis can be written as a linear
assemblage of words and symbols taken from a finite set of
possible symbols. Inasmuch as we can enumerate the letters used
in the words, each hypothesis can be given a number.

I have never understood the logic of this notion, which I, too,
saw a generation ago. When I first saw it I couldn't believe that
it was taken seriously. Most such finite sets of letters must be
gibberish. The number of non-gibberish sets must be enormously
greater than meaningful nongibberish sets, including as they do
all paragraphs from all works of literature that have yet to be
written in nonexistent languages, or backward, or from the inside
out, or in various substitution codes, or encrypted ciphers, and
so on.

Merely placing letters (or even whole propositions) in one-to-one
correspondence with qualitative "facts" that can be true or false
by no means exhausts the possibilities for forming propositions
as hypotheses about real or imagined nature. Propositions can
contain quantitative references, algebraic and differential
equations, and statements of quantitative inequality. Every
hypothesis containing a quantitative statement has as many
potential variants as there are real numbers (and they are quite
statable in a finite number of symbols: consider the Pytharogean
Theorem, which entails a trancendental number. Consider the
symbols "..." and their meaning).

Even within the realm of qualitative true-false propositions,
there are enormously more propositions that can be expressed in
statements of length L; far more than N^L. That number assumes a
set of propositions that could be expressed as a sequence of 1's
and 0's of length L. But that is not the only way to calculate
the number.

You may recall my proposal that a person is never aware OF the
level of control he/she is working FROM. Presumably, you and John
Gabriel, in reasoning about this subject, have been working FROM
the logic or reasoning level. And in line with my proposition,
you have neglected to take logic into account, and have dealt
only with binary sequences, the next level down.

How many different hypotheses can be formed as logical
propositions involving L variables, each capable of being only
true or false?

The answer is N = 2^(2^L). For only 8 logical variables, this
number is ten to the 77th power. You have therefore vastly
underestimated the number of possible hypotheses, and greatly
overestimated the a priori probability of truth of any one
hypothesis, even one so simple as to rest on only 8 propositions.
In your paper on Ockham's razor, containing about 13,000 letters,
I think that there is a reliance on at least 8 propositions
capable of being independently true or false. So the a priori
probability of this entire argument being meaningful must be
extremely if not vanishingly small.

If you take the point of view, as I do, that symbols are
arbitrary pointers to (mainly) nonverbal experiences, then we
simply assign symbols on the basis that they must differentiate
between experiences. There is no danger, obviously, of running
out of conveniently short symbols, particularly as we can re-use
the same symbols with different meanings in different contexts.
Neither are we likely to run out of ways of stringing symbols
into sequences, and it is utterly impossible that we or all of
humanity could run out of potential logical propositions
concerning nature or anything else, using a different 8-variable
proposition every nanosecond for the rest of the estimated
lifetime of the universe.

The real problem is to tie our logic to the logic of direct
experience. This can't be done by inventing arbitrary rules and
then playing out their consequences by symbol-manipulation. The
number of possible games that could be played this way is
staggeringly, unthinkably, large. The proportion of such games
that has something to do, even peripherally, with the universe of
experience is too small to estimate. There is essentially no
chance that a symbolic argument using variables that have no
demonstrated experimental or experiential meaning will ever
uncover a law of nature. To find logical propositions that relate
to natural law, we must first perceive the logical relationships
that actually occur, and then if we like, translate them into
symbolic language. Trying to do it the other way around is a
losing proposition.

Best,

Bill P.

2

[From Oded Maler 930113]

ยทยทยท

*
* [Bill Powers (930113.0730)]
*
* Martin Taylor (930112.1830) --
*
* You have indeed managed to introduce a disturbance, in your paper
* on Ockhams's Razor. I might entitle my comments, "A critique of
* pure reason."
*
* >In principle, any hypothesis can be written as a linear
* >assemblage of words and symbols taken from a finite set of
* >possible symbols. Inasmuch as we can enumerate the letters used
* >in the words, each hypothesis can be given a number.
*
* I have never understood the logic of this notion, which I, too,
* saw a generation ago. When I first saw it I couldn't believe that
* it was taken seriously. Most such finite sets of letters must be
* gibberish. The number of non-gibberish sets must be enormously
* greater than meaningful nongibberish sets, including as they do
* all paragraphs from all works of literature that have yet to be
* written in nonexistent languages, or backward, or from the inside
* out, or in various substitution codes, or encrypted ciphers, and
* so on.
*
* Merely placing letters (or even whole propositions) in one-to-one
* correspondence with qualitative "facts" that can be true or false
* by no means exhausts the possibilities for forming propositions
* as hypotheses about real or imagined nature. Propositions can
* contain quantitative references, algebraic and differential
* equations, and statements of quantitative inequality. Every
* hypothesis containing a quantitative statement has as many
* potential variants as there are real numbers (and they are quite
* statable in a finite number of symbols: consider the Pytharogean
* Theorem, which entails a trancendental number. Consider the
* symbols "..." and their meaning).

I don't think you understand the point (or I don't). Suppose you
have a phenomenon with a finite number of variable and you want
to model it using a system of differetnial equations. Now the set
of all systems of differential equations (with rational coeffeicients,
with a finite basis of functions using which you express them, etc.
- in short a finite alphaet) is countable and can be enumerated
systematically - you can write a program that generates the infinite
sequence of all possible hypotheses. So, in principle, you can stick
to some hypothesis until some experimental observation contradicts
it and then switch to the next hypothesis in the sequence that does not
contradict the observation. This idealized conception of scientific
theory formation is usually used to give lower-bounds, that is,
even in this case of a fixed alphabet, under such and such condition
you will never converge to the true hypothesis.

The fact that the set of everything speakable (typable on a computer)
is countable (i.e., a number can be given to every possibly speakable
thing) is not so marginal.

--Oded

--

Oded Maler, LGI-IMAG (Campus), B.P. 53x, 38041 Grenoble, France
Phone: 76635846 Fax: 76446675 e-mail: maler@vercors.imag.fr

3

The real problem is to tie our logic to the logic of direct
experience. This can't be done by inventing arbitrary rules and
then playing out their consequences by symbol-manipulation. The
number of possible games that could be played this way is
staggeringly, unthinkably, large. The proportion of such games
that has something to do, even peripherally, with the universe of
experience is too small to estimate. There is essentially no
chance that a symbolic argument using variables that have no
demonstrated experimental or experiential meaning will ever
uncover a law of nature. To find logical propositions that relate
to natural law, we must first perceive the logical relationships
that actually occur, and then if we like, translate them into
symbolic language. Trying to do it the other way around is a
losing proposition.

Thanks! This is why I enjoy reading CSG-L! This is basically the
disagreement I have had for >30 years with the artificial intelligence
people. They seem unable to conceive of anything other than deductive
(symbolic) logic as explanative (or generative) of behavior, i.e.
thought, perception, language use, throwing a ball or courting a mate.

I believe there is another half of "thought" which has been pretty much
ignored as too difficult since the Greek focus on "logic". That is the
non-logical behavior of induction, learning, analogy and other analog
phemonema.

When I'm feeling fractious I like to say "Indeed, you have a superb
argument; valid and truth-maintaining, which has made several
propositions explicit which were not visible before. I can see that
each step of your "reasoning" (interesting word, that.) follows
inevitably from previous statements. But where did the _premises_ come
from? (And how do you plan to make the conclusions useful?)

G'day,
Ray Allis
ray@atc.boeing.com

4

[Martin Taylor 930114 -- the first time I have managed to put "93" without
using the backspace key! -- 14:30]
(Bill Powers 930113.0730)

In principle, any hypothesis can be written as a linear
assemblage of words and symbols taken from a finite set of
possible symbols. Inasmuch as we can enumerate the letters used
in the words, each hypothesis can be given a number.

I have never understood the logic of this notion, which I, too,
saw a generation ago.

The idea must have been in the air at that time. I never say it anywhere
else until the 80's, but then it was with reference to Kolmogorov, around
1970-ish. It just seemed a straightforward application of information
theory to me, not something worth publishing. I guess you still think
it not worth publishing!

When I first saw it I couldn't believe that
it was taken seriously. Most such finite sets of letters must be
gibberish. The number of non-gibberish sets must be enormously
greater than meaningful nongibberish sets, including as they do
all paragraphs from all works of literature that have yet to be
written in nonexistent languages, or backward, or from the inside
out, or in various substitution codes, or encrypted ciphers, and
so on.

Yep. The longer the string, the higher the proportion of gibberish, too.
And that includes strings that purport to make sense. That's the essence
of my argument.

Merely placing letters (or even whole propositions) in one-to-one
correspondence with qualitative "facts" that can be true or false
by no means exhausts the possibilities for forming propositions
as hypotheses about real or imagined nature.

That's not what I am doing. I'm talking about how succinctly you, the
observer of nature, can express your hypothesis in any terms whatever
that use your chosen symbol set. I used linear strings (i.e. some form
of natural, formal, or mathemathematical language.) but the argument
applies equally well to 2-D strings (graphic depictions) or 3 or 4-D
strings (animations).

Even within the realm of qualitative true-false propositions,
there are enormously more propositions that can be expressed in
statements of length L; far more than N^L. That number assumes a
set of propositions that could be expressed as a sequence of 1's
and 0's of length L. But that is not the only way to calculate
the number.

Huh? There are N possibilities for the first symbol in the string, N for
the second, ... It still seems to me that there are N^L possible strings
of length L, and among these are ALL the possible things you can say using
strings of length L, including all the hypotheses of length L, which, as
you initially pointed out, are a minuscule subset of the strings of length L.

How many different hypotheses can be formed as logical
propositions involving L variables, each capable of being only
true or false?

The answer is N = 2^(2^L).

I'm not sure what you are getting at here. Your L variables seem as if they
would be the length of the string that describes your hypothesis. But to
create a proposition using L variables would require quite a few more symbols,
at least L-1, I should think (one binary operator between each pair of
variable symbols). There are quite a few possible choices for what these
operators might be and where in the string they might be placed. But if
we assert reverse Polish notation (an assertion that should be included in
the length of the string, but I will assume that it is among the internal
prior assertions), then the number of possible strings would seem to be
2^L * S^(L-1) where S is the number of binary operators.

I think we are talking in different worlds here, and I can't find yours.

Propositions can
contain quantitative references, algebraic and differential
equations, and statements of quantitative inequality. Every
hypothesis containing a quantitative statement has as many
potential variants as there are real numbers ...

Yes, that's all included in the length of the hypothesis. In decomal notation,
a real number takes around 3 1/3 bits of information per digit. How
precise do you want the real numbers in your hypothesis statement to be?
That will affect the length L of your hypothesis. If you are only going
to have one-zero symbols, you need about 10 of them for every 3 decimal
digits of precision, unless your number can be described as "e", which
takes one symbol, or "pi" which takes 2 (or one from a larger symbol set
that includes Greek notation).

If you take the point of view, as I do, that symbols are
arbitrary pointers to (mainly) nonverbal experiences, then we
simply assign symbols on the basis that they must differentiate
between experiences.

Yes, you are describing the point I am trying to get across, that
the simplicity of a hypothesis depends on what you are adding to
(distinguishing from) what you already know. It is not absolute.

The number of possible games that could be played this way is
staggeringly, unthinkably, large. The proportion of such games
that has something to do, even peripherally, with the universe of
experience is too small to estimate. There is essentially no
chance that a symbolic argument using variables that have no
demonstrated experimental or experiential meaning will ever
uncover a law of nature.

I make no statement about how you go about finding laws of nature,
models, or even simple point-by-point descriptions. I am concerned
with how YOU (anyone, individually) can incorporate them into YOUR
model of how the world works. The argument is that Occam's razor
not only is convenient, but has a vanishingly small chance of failing.
Your points about large numbers only serve to buttress what I am
trying to get across.

Martin

5

[Chris Malcolm]

Ray Allis writes:

Thanks! This is why I enjoy reading CSG-L! This is basically the
disagreement I have had for >30 years with the artificial intelligence
people. They seem unable to conceive of anything other than deductive
(symbolic) logic as explanative (or generative) of behavior, i.e.
thought, perception, language use, throwing a ball or courting a mate.

You are a decade or so out of date (or your AI friends are). For example
"The Embodied Mind" by Varela, Rosch, and Thomson explains three of the
current major AI paradigms, of which symbolic computation is only one.
The AI section in any decent library or bookstore should these days
contain at least half-a-dozen books devoted to criticising the symbolic
computation approach and proposing alternatives.

I believe there is another half of "thought" which has been pretty much
ignored as too difficult since the Greek focus on "logic". That is the
non-logical behavior of induction, learning, analogy and other analog
phemonema.

Check out the publications of Morgan Kaufmann, publishers. They have at
least one heavily compendious collection of recent AI papers on each of
induction, learning, and analogy. I'm also rather surpised to read
someone posting from the homeland of Pierce and James complaining that
these aspects of thought have been ignored.

If your disagreement with AI is >30 years old I think it's time you
updated it a bit!

[By the way, I'm far from being the only AI researcher lurking in the
readership of CSG, and pointing students and colleagues at PCT.]

6

[Ray Allis 930115.1315 PST]

[Chris Malcolm 930115.1919 GMT]

You are a decade or so out of date (or your AI friends are). For example
"The Embodied Mind" by Varela, Rosch, and Thomson explains three of the
current major AI paradigms, of which symbolic computation is only one.
The AI section in any decent library or bookstore should these days
contain at least half-a-dozen books devoted to criticising the symbolic
computation approach and proposing alternatives.

For five years I had a standing search order with our technical library
for anything mentioning induction, analogy or generally learning, in
psychology, philosophy, computer science and any other data bases they
could get at. I cancelled the request two years ago, because I could
not find _anything_ relevant. (Well, there WAS one conference on real
induction in Britain in, I think, the early 1970's, and one PhD.
thesis.) It's ALL deduction!

I haven't read "The Embodied Mind" though. I will, on your
recommendation, rectify that as soon as I can find a copy I can read
for free. I quit buying books unread even longer than two years ago.

Check out the publications of Morgan Kaufmann, publishers. They have at
least one heavily compendious collection of recent AI papers on each of
induction, learning, and analogy.

They certainly do! (Our technical library is pretty good.) They
appear to describe the attempt to do these things using deduction.

I'm also rather surpised to read
someone posting from the homeland of Pierce and James complaining that
these aspects of thought have been ignored.

Perhaps not ignored. Just set aside until someone has an idea. Meantime,
you can _accomplish_ things with deduction. (And get paid.)

I did some work with a LOOPS-based system from U. of Ill. based on
Pierce's notion of _abduction_. Still symbolic; _not_ induction.
Ryszard Michalski ("Machine Learning" I & II) spent considerable time
here, explaining that subject. Regardless of the titles of the
publications, few people outside AI would find any induction (learning)
or analogy.

If your disagreement with AI is >30 years old I think it's time you
updated it a bit!

I'm willing. I quit following closely because I felt I had better uses
for my time, but I'll read "The Embodied Mind" and let you know. I
found other work by Varela pretty interesting, especially the famous
stuff with Maturana.

[By the way, I'm far from being the only AI researcher lurking in the
readership of CSG, and pointing students and colleagues at PCT.]

Good! Someone may fall into something yet. Because PCT was developed
by people ( Hi, Bill!) who understood about analogs and analogy.

Enjoy!
Ray Allis