Help needed (AI, perceptrons)

[yann]

Hi, i'm making a perceptron learn to make his output number six, it always makes it number 5.999... is it ok if i make number 5.999... = 6? will it affect the perceptrons performance? And why do they need to learn to calculate nuber 6(not nesseserly 6) when their output is going to be 1 or 0 anyway...? Please help.

yann

Sorry because I asked these questions before, but i never got a good answer( MK27 you helped me the most thanks...)

[MK27]

Yann, I am still pretty much positive this is a consequence of using floating point numbers.

I believe it is commonly understood that using == with floats and doubles is a bad practice.* Which means you cannot really expect a set of complex calculations to == 6.0, even if that is the mathematically correct answer.

Someone will contradict me here if I'm wrong.

*it's OK to use floats, just don't use == with them

[cyberfish]

I guess you can use == if you know ALL your intermediate results up to that point can be represented with infinite precision (eg, if they are all integers smaller than 2^51 or so).

[MK27]

I guess you can use == if you know ALL your intermediate results up to that point can be represented with infinite precision (eg, if they are all integers smaller than 2^51 or so).

But that is obviously not true.

#include <stdio.h>

int main() {
        float x=0.0f, i;
        for (i=0.0f; i<20; i+=0.1f) {
                printf("%f\n",x+i);
        }
        return 0;
}

We all know what happens:

2.600000
2.700000
2.799999
2.899999

Altho the point at which it happens depends on the compiler/system. Any reasonable person would assume that 2.7 + 0.1 == 2.8, but if I assume that in my code here, the condition (==) will never be true, since in fact in this case we have 2.7 + 0.1 = 2.79999....

[cyberfish]

Last time I checked, 2.6 is not an integer .

#include <stdio.h>

int main() {
        double a = 19683.0, i;
        for (i = 0; i < 9; ++i) {
                a /= 3;
                a *= 2;
                printf("%lf\n", a + i);
        }
}

Or

#include <stdio.h>

int main() {
        double a = 1.0, i;
        for (i = 0; i < 9; ++i) {
                a /= 2;
                printf("%lf\n", a);
        }

        a *= 8.0;
        a *= 8.0;
        a *= 8.0;

        printf("%lf\n", a);
}

This works because everything is in base 2.

But of course, these are exceptions, and in general, it's not a good idea to compare floating point numbers with == (or >, <, >=, and <= without a threshold).

[MK27]

in general, it's not a good idea to compare floating point numbers with == (or >, <, >=, and <= without a threshold).

Okay, so we agree.

BTW "2.6" never appears as an integer in that code,

float x=0.0f, i;
        for (i=0.0f; i<20; i+=0.1f) {
                printf("%f\n",x+i);

The only integer is 20.

[cyberfish]

But the intermediate results of x include non-integers not representable with perfect precision (eg 0.1).

I guess you can use == if you know ALL your intermediate results up to that point can be represented with infinite precision (eg, if they are all integers smaller than 2^51 or so).

[MK27]

But the intermediate results of x include non-integers not representable with perfect precision (eg 0.1).

A little tricky there cyberfish :roll:

Anyway, I believe I have pointed yann to a wikipedia article about floats (and 0.1). Given that, it seems to me slightly foolish to believe that "ALL your intermediate results up to that point can be represented with infinite precision" unless you are the floating point guru

The moral of the story being 5.9999 is essentially 6 for the purpose of the perceptaron, methinks.

[cyberfish]

I am no floating point guru, but it's not that complex.

In base 10, suppose we have 10 digits to represent the number, with an exponent. First of all, all integers less than 9999999999 are representable. For decimal numbers, if the decimals terminate within the precision (eg. 5.1 or 5.000001 or 12345.6789), they are representable with perfect precision.

It's the same thing for floating point numbers, except it's in base 2.

In my second example, what I was doing was, in base 2 -
1
0.1
0.01
0.001

by dividing the number by 2 each time.
Of course they are all representable.

[MK27]

For decimal numbers, if the decimals terminate within the precision (eg. 5.1 or 5.000001 or 12345.6789), they are representable with perfect precision.

Okay, now I am confused. So what is wrong with 2.6 and 0.1? Which I have heard from a few sources now that the problem is that the decimal number 0.1 is not perfectly representable in binary. But it seems to me that 0.1 terminates precisely, at one decimal point. (Or did you mean "terminate with precision" in binary, like 5.1 isn't a number you pulled out of your head? In which case the explanation is becoming circular...)

[cyberfish]

Can you convert 0.1 to binary? using a combination of 1/2, 1/4, 1/8, 1/16...

The number 0.1d in binary is like 1/3 in decimal. You need an infinite number of decimal places to represent it.

[tabstop]

Okay, now I am confused. So what is wrong with 2.6 and 0.1? Which I have heard from a few sources now that the problem is that the decimal number 0.1 is not perfectly representable in binary. But it seems to me that 0.1 terminates precisely, at one decimal point. (Or did you mean "terminate with precision" in binary, like 5.1 isn't a number you pulled out of your head? In which case the explanation is becoming circular...)

Right. So those would be perfectly good floating-point numbers if your float type used (say) binary coded decimal, because they do terminate in decimal notation. If your float type uses actual binary notation, then you're sunk. (0.5 works either way -- it terminates in a decimal notation, and since it's 1/2, it terminates in binary notation too.)

[MTK]

From Wikipedia:

In mathematics, a dyadic fraction or dyadic rational is a rational number whose denominator is a power of two, i.e., a number of the form a/2b where a is an integer and b is a natural number; for example, 1/2 or 3/8, but not 1/3. These are precisely the numbers whose binary expansion is finite.

According to this, 0.1 (or 1/10) is not a dyadic fraction, and so its binary representation in infinitely long. And because float and double obviously cannot store an infinite amount if digits, it cannot accurately store 0.1 in binary.

Right. So those would be perfectly good floating-point numbers if your float type used (say) binary coded decimal, because they do terminate in decimal notation. If your float type uses actual binary notation, then you're sunk. (0.5 works either way -- it terminates in a decimal notation, and since it's 1/2, it terminates in binary notation too.)

I didn't know that some computers store floats an BCD. That way it sure would store 0.1 properly.

http://en.wikipedia.org/wiki/Binary-coded_decimal

[cosmic_cow]

My programming for non-majors professor listed this along with our notes with the header "Why You Shouldn't Use Float Counters in Loops." I think it goes along with this.
Computer Arithmetic Tragedies

[yann]

Right, actually my question was can i use 5.999... as 6? well i think i could...or not...
but people, thank you much, this is the best forum in the world...you people helped me soooo much thank you