Thread: why can't my perceptron learn correctly?

Originally Posted by abachler

another little piece of advice is that perceptrons generally need either fixed or floating point weights, inputs and outputs. integers just won't do.

ok ...what are "fixed" values? (i don't speak English quite well...)

"Fixed" meaning constant, or never changing, I think.

Regarding zeroing of global variables: Velocity Reviews - View Single Post - global variables initialization

> When you declare some global variables(any type), do they initialized
> with zero implicitly or they have garbage (whatever was there on the
> memory location)?

Zero. It says so in 6.7.8 (10) of the C99 standard.

Originally Posted by dwks

Zero. It says so in 6.7.8 (10) of the C99 standard.

No it doesn't.

If an object that has automatic storage duration is not initialized explicitly, its value is indeterminate. If an object that has static storage duration is not initialized elicitly, then:

-- if it has a pointer type, it is initialized to a null pointer;
-- if it has arithmetic type, it is initialized to (positive or unsigned) zero;
-- if it is an aggregate, every member is initialized (recursively) according to these rules;
-- if it is a union, the first named member is initialized (recursively) according to these rules.

Global variables are not static, because you can actually declare them as such.


Quzah.

That's what I was looking for yesterday! I couldn't find where it described global scope. Thanks.


Quzah.

Originally Posted by mike_g

Why not? Every neural net I have seen so far does.

Nah, fixed point arithmetic. But its simpler to just use floating point. The weights themselves need to be adjusted to train the network.

Okay, but it is not really necessary. We had some problems with floating point "precision" -- check yann's previous threads. IMO the floats are just a pointless complication at this point. Once the thing is working satisfactorily, we can worry about aesthetic refinements such as the difference between int 0-10000 and float 0.0 - 1.0.

@ yann: "Fixed precision/fixed point" is like money, limited to a specific number of decimal places. Unfortunately, there is no fixed precision datatype in C, so it doesn't matter.

Originally Posted by yann

ok ...what are "fixed" values? (i don't speak English quite well...)

Fixed point, the ugly stepsister to floating point. They hold decimal values like floating point but are constrained to a specific predetermined range (exponent value).

They are faster to execute in hardware, but not as flexible. And as noted not available in C without a 3rd party library. Apparently this code does Fixed point, but i haven't tested it and its not my code. It might be C++ though.

In any case you need to use floating point values, because for a eprceptron to work correctly it needs decimal values for the training to work correctly. You also need a sigmoid function. My favorite sigmoid for speed is sin(atan(x)) where x is the sum of products from the inputs*weights. 2*atan(x)/pi also works.

Inputs to a perceptron must be normalized to the range of -1.0 to +1.0, so your assertion that you can just go with integers until you get it working is a infinite loop of futility.

Originally Posted by booklist

Artificial Intelligence, A Modern Approach ISBN 0-13-790395-2
Neural Networks, A Comprehensive Foundation, ISBN 0-13-273350-1
Neural Networks for Pattern Recognition, ISBN 978-0-19-853864-6
Biophysics of Computation, Information porcessing in single neurons, ISBN 978-0-19-518199-9
Theoretical Neuroscience, Computational and Mathematical Modelling of Neural Systems, ISBN 978-0-262-54185-5
C++ Neural Networks & Fuzzy Logic, ISBN 1-55851-552-6

Code:

/*
Copyright (c) 2006 Henry Strickland & Ryan Seto

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

        (* http://www.opensource.org/licenses/mit-license.php *)
*/

class Fixed {

private:

	int	g; // the guts

const static int BP= 16;  // how many low bits are right of Binary Point
const static int BP2= BP*2;  // how many low bits are right of Binary Point
const static int BPhalf= BP/2;  // how many low bits are right of Binary Point

static double STEP() { return 1.0 / (1<<BP); }  // smallest step we can represent

	// for private construction via guts
	enum FixedRaw { RAW };
	Fixed(FixedRaw, int guts) : g(guts) {}


public:
	Fixed() : g(0) {}
	Fixed(const Fixed& a) : g( a.g ) {}
	Fixed(float a) : g( int(a / (float)STEP()) ) {}
	Fixed(double a) : g( int(a / (double)STEP()) ) {}
	Fixed(int a) : g( a << BP ) {}
	Fixed(long a) : g( a << BP ) {}
	Fixed& operator =(const Fixed& a) { g= a.g; return *this; }
	Fixed& operator =(float a) { g= Fixed(a).g; return *this; }
	Fixed& operator =(double a) { g= Fixed(a).g; return *this; }
	Fixed& operator =(int a) { g= Fixed(a).g; return *this; }
	Fixed& operator =(long a) { g= Fixed(a).g; return *this; }

	operator float() { return g * (float)STEP(); }
	operator double() { return g * (double)STEP(); }
	operator int() { return g>>BP; }
	operator long() { return g>>BP; }

	Fixed operator +() const { return Fixed(RAW,g); }
	Fixed operator -() const { return Fixed(RAW,-g); }

	Fixed operator +(const Fixed& a) const { return Fixed(RAW, g + a.g); }
	Fixed operator -(const Fixed& a) const { return Fixed(RAW, g - a.g); }
#if 1
	// more acurate, using long long
	Fixed operator *(const Fixed& a) const { return Fixed(RAW,  (int)( ((long long)g * (long long)a.g ) >> BP)); }
#else
	// faster, but with only half as many bits right of binary point
	Fixed operator *(const Fixed& a) const { return Fixed(RAW, (g>>BPhalf) * (a.g>>BPhalf) ); }
#endif
	Fixed operator /(const Fixed& a) const { return Fixed(RAW, int( (((long long)g << BP2) / (long long)(a.g)) >> BP) ); }

	Fixed operator +(float a) const { return Fixed(RAW, g + Fixed(a).g); }
	Fixed operator -(float a) const { return Fixed(RAW, g - Fixed(a).g); }
	Fixed operator *(float a) const { return Fixed(RAW, (g>>BPhalf) * (Fixed(a).g>>BPhalf) ); }
	Fixed operator /(float a) const { return Fixed(RAW, int( (((long long)g << BP2) / (long long)(Fixed(a).g)) >> BP) ); }

	Fixed operator +(double a) const { return Fixed(RAW, g + Fixed(a).g); }
	Fixed operator -(double a) const { return Fixed(RAW, g - Fixed(a).g); }
	Fixed operator *(double a) const { return Fixed(RAW, (g>>BPhalf) * (Fixed(a).g>>BPhalf) ); }
	Fixed operator /(double a) const { return Fixed(RAW, int( (((long long)g << BP2) / (long long)(Fixed(a).g)) >> BP) ); }

	Fixed& operator +=(Fixed a) { return *this = *this + a; return *this; }
	Fixed& operator -=(Fixed a) { return *this = *this - a; return *this; }
	Fixed& operator *=(Fixed a) { return *this = *this * a; return *this; }
	Fixed& operator /=(Fixed a) { return *this = *this / a; return *this; }

	Fixed& operator +=(int a) { return *this = *this + (Fixed)a; return *this; }
	Fixed& operator -=(int a) { return *this = *this - (Fixed)a; return *this; }
	Fixed& operator *=(int a) { return *this = *this * (Fixed)a; return *this; }
	Fixed& operator /=(int a) { return *this = *this / (Fixed)a; return *this; }

	Fixed& operator +=(long a) { return *this = *this + (Fixed)a; return *this; }
	Fixed& operator -=(long a) { return *this = *this - (Fixed)a; return *this; }
	Fixed& operator *=(long a) { return *this = *this * (Fixed)a; return *this; }
	Fixed& operator /=(long a) { return *this = *this / (Fixed)a; return *this; }

	Fixed& operator +=(float a) { return *this = *this + a; return *this; }
	Fixed& operator -=(float a) { return *this = *this - a; return *this; }
	Fixed& operator *=(float a) { return *this = *this * a; return *this; }
	Fixed& operator /=(float a) { return *this = *this / a; return *this; }

	Fixed& operator +=(double a) { return *this = *this + a; return *this; }
	Fixed& operator -=(double a) { return *this = *this - a; return *this; }
	Fixed& operator *=(double a) { return *this = *this * a; return *this; }
	Fixed& operator /=(double a) { return *this = *this / a; return *this; }

	bool operator ==(const Fixed& a) const { return g == a.g; }
	bool operator !=(const Fixed& a) const { return g != a.g; }
	bool operator <=(const Fixed& a) const { return g <= a.g; }
	bool operator >=(const Fixed& a) const { return g >= a.g; }
	bool operator  <(const Fixed& a) const { return g  < a.g; }
	bool operator  >(const Fixed& a) const { return g  > a.g; }

	bool operator ==(float a) const { return g == Fixed(a).g; }
	bool operator !=(float a) const { return g != Fixed(a).g; }
	bool operator <=(float a) const { return g <= Fixed(a).g; }
	bool operator >=(float a) const { return g >= Fixed(a).g; }
	bool operator  <(float a) const { return g  < Fixed(a).g; }
	bool operator  >(float a) const { return g  > Fixed(a).g; }

	bool operator ==(double a) const { return g == Fixed(a).g; }
	bool operator !=(double a) const { return g != Fixed(a).g; }
	bool operator <=(double a) const { return g <= Fixed(a).g; }
	bool operator >=(double a) const { return g >= Fixed(a).g; }
	bool operator  <(double a) const { return g  < Fixed(a).g; }
	bool operator  >(double a) const { return g  > Fixed(a).g; }
};

inline Fixed operator +(float a, const Fixed& b) { return Fixed(a)+b; }
inline Fixed operator -(float a, const Fixed& b) { return Fixed(a)-b; }
inline Fixed operator *(float a, const Fixed& b) { return Fixed(a)*b; }
inline Fixed operator /(float a, const Fixed& b) { return Fixed(a)/b; }

inline bool operator ==(float a, const Fixed& b) { return Fixed(a) == b; }
inline bool operator !=(float a, const Fixed& b) { return Fixed(a) != b; }
inline bool operator <=(float a, const Fixed& b) { return Fixed(a) <= b; }
inline bool operator >=(float a, const Fixed& b) { return Fixed(a) >= b; }
inline bool operator  <(float a, const Fixed& b) { return Fixed(a)  < b; }
inline bool operator  >(float a, const Fixed& b) { return Fixed(a)  > b; }



inline Fixed operator +(double a, const Fixed& b) { return Fixed(a)+b; }
inline Fixed operator -(double a, const Fixed& b) { return Fixed(a)-b; }
inline Fixed operator *(double a, const Fixed& b) { return Fixed(a)*b; }
inline Fixed operator /(double a, const Fixed& b) { return Fixed(a)/b; }

inline bool operator ==(double a, const Fixed& b) { return Fixed(a) == b; }
inline bool operator !=(double a, const Fixed& b) { return Fixed(a) != b; }
inline bool operator <=(double a, const Fixed& b) { return Fixed(a) <= b; }
inline bool operator >=(double a, const Fixed& b) { return Fixed(a) >= b; }
inline bool operator  <(double a, const Fixed& b) { return Fixed(a)  < b; }
inline bool operator  >(double a, const Fixed& b) { return Fixed(a)  > b; }


int& operator +=(int& a, const Fixed& b) { a = (Fixed)a + b; return a; }
int& operator -=(int& a, const Fixed& b) { a = (Fixed)a - b; return a; }
int& operator *=(int& a, const Fixed& b) { a = (Fixed)a * b; return a; }
int& operator /=(int& a, const Fixed& b) { a = (Fixed)a / b; return a; }

long& operator +=(long& a, const Fixed& b) { a = (Fixed)a + b; return a; }
long& operator -=(long& a, const Fixed& b) { a = (Fixed)a - b; return a; }
long& operator *=(long& a, const Fixed& b) { a = (Fixed)a * b; return a; }
long& operator /=(long& a, const Fixed& b) { a = (Fixed)a / b; return a; }

float& operator +=(float& a, const Fixed& b) { a = a + b; return a; }
float& operator -=(float& a, const Fixed& b) { a = a - b; return a; }
float& operator *=(float& a, const Fixed& b) { a = a * b; return a; }
float& operator /=(float& a, const Fixed& b) { a = a / b; return a; }

double& operator +=(double& a, const Fixed& b) { a = a + b; return a; }
double& operator -=(double& a, const Fixed& b) { a = a - b; return a; }
double& operator *=(double& a, const Fixed& b) { a = a * b; return a; }
double& operator /=(double& a, const Fixed& b) { a = a / b; return a; }