why can't my perceptron learn correctly?

This is a discussion on why can't my perceptron learn correctly? within the C Programming forums, part of the General Programming Boards category; Originally Posted by abachler another little piece of advice is that perceptrons generally need either fixed or floating point weights, ...

  1. #16
    Registered User yann's Avatar
    Join Date
    Sep 2009
    Location
    Zagreb, Croatia
    Posts
    186
    Quote Originally Posted by abachler View Post
    another little piece of advice is that perceptrons generally need either fixed or floating point weights, inputs and outputs. integers just won't do.
    ok ...what are "fixed" values? (i don't speak English quite well...)
    Arduino rocks!

  2. #17
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,048
    "Fixed" meaning constant, or never changing, I think.

    Regarding zeroing of global variables: Velocity Reviews - View Single Post - global variables initialization
    > When you declare some global variables(any type), do they initialized
    > with zero implicitly or they have garbage (whatever was there on the
    > memory location)?

    Zero. It says so in 6.7.8 (10) of the C99 standard.
    dwk

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell


    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ: cpwiki.sf.net

    My website: http://dwks.theprogrammingsite.com/
    Projects: codeform, xuni, atlantis, nort, etc.

  3. #18
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    I think in this case "fixed" modifies "point".

  4. #19
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Quote Originally Posted by dwks View Post
    Zero. It says so in 6.7.8 (10) of the C99 standard.
    No it doesn't.

    If an object that has automatic storage duration is not initialized explicitly, its value is indeterminate. If an object that has static storage duration is not initialized elicitly, then:

    -- if it has a pointer type, it is initialized to a null pointer;
    -- if it has arithmetic type, it is initialized to (positive or unsigned) zero;
    -- if it is an aggregate, every member is initialized (recursively) according to these rules;
    -- if it is a union, the first named member is initialized (recursively) according to these rules.
    Global variables are not static, because you can actually declare them as such.


    Quzah.
    Hope is the first step on the road to disappointment.

  5. #20
    Dr Dipshi++ mike_g's Avatar
    Join Date
    Oct 2006
    Location
    On me hyperplane
    Posts
    1,218
    Oh god please no don't do this...
    Why not? Every neural net I have seen so far does.

    "Fixed" meaning constant, or never changing, I think.
    Nah, fixed point arithmetic. But its simpler to just use floating point. The weights themselves need to be adjusted to train the network.

  6. #21
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Quote Originally Posted by quzah View Post
    No it doesn't.

    Global variables are not static, because you can actually declare them as such.


    Quzah.
    Your flaw here is that you think "static" is how you get "static storage duration". Alas:
    Quote Originally Posted by C99, 6.2.4
    An object whose identifier is declared with external or internal linkage, or with the
    storage-class specifier static has static storage duration. Its lifetime is the entire
    execution of the program and its stored value is initialized only once, prior to program
    startup.
    And of course every identifier at file scope (i.e., global) has external linkage [6.2.2, para 5], unless it is modified by "static", in which case it has internal linkage (but the "static" does not make it a static storage duration variable, the file scope makes it so).

  7. #22
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    That's what I was looking for yesterday! I couldn't find where it described global scope. Thanks.


    Quzah.
    Hope is the first step on the road to disappointment.

  8. #23
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by mike_g View Post
    Why not? Every neural net I have seen so far does.

    Nah, fixed point arithmetic. But its simpler to just use floating point. The weights themselves need to be adjusted to train the network.
    Okay, but it is not really necessary. We had some problems with floating point "precision" -- check yann's previous threads. IMO the floats are just a pointless complication at this point. Once the thing is working satisfactorily, we can worry about aesthetic refinements such as the difference between int 0-10000 and float 0.0 - 1.0.

    @ yann: "Fixed precision/fixed point" is like money, limited to a specific number of decimal places. Unfortunately, there is no fixed precision datatype in C, so it doesn't matter.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  9. #24
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,189
    Quote Originally Posted by yann View Post
    ok ...what are "fixed" values? (i don't speak English quite well...)
    Fixed point, the ugly stepsister to floating point. They hold decimal values like floating point but are constrained to a specific predetermined range (exponent value).

    They are faster to execute in hardware, but not as flexible. And as noted not available in C without a 3rd party library. Apparently this code does Fixed point, but i haven't tested it and its not my code. It might be C++ though.

    In any case you need to use floating point values, because for a eprceptron to work correctly it needs decimal values for the training to work correctly. You also need a sigmoid function. My favorite sigmoid for speed is sin(atan(x)) where x is the sum of products from the inputs*weights. 2*atan(x)/pi also works.

    Inputs to a perceptron must be normalized to the range of -1.0 to +1.0, so your assertion that you can just go with integers until you get it working is a infinite loop of futility.

    Quote Originally Posted by booklist
    Artificial Intelligence, A Modern Approach ISBN 0-13-790395-2
    Neural Networks, A Comprehensive Foundation, ISBN 0-13-273350-1
    Neural Networks for Pattern Recognition, ISBN 978-0-19-853864-6
    Biophysics of Computation, Information porcessing in single neurons, ISBN 978-0-19-518199-9
    Theoretical Neuroscience, Computational and Mathematical Modelling of Neural Systems, ISBN 978-0-262-54185-5
    C++ Neural Networks & Fuzzy Logic, ISBN 1-55851-552-6
    Code:
    /*
    Copyright (c) 2006 Henry Strickland & Ryan Seto
    
    Permission is hereby granted, free of charge, to any person obtaining a
    copy of this software and associated documentation files (the "Software"),
    to deal in the Software without restriction, including without limitation
    the rights to use, copy, modify, merge, publish, distribute, sublicense,
    and/or sell copies of the Software, and to permit persons to whom the
    Software is furnished to do so, subject to the following conditions:
    
    The above copyright notice and this permission notice shall be included
    in all copies or substantial portions of the Software.
    
    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
    OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
    ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
    OTHER DEALINGS IN THE SOFTWARE.
    
            (* http://www.opensource.org/licenses/mit-license.php *)
    */
    
    class Fixed {
    
    private:
    
    	int	g; // the guts
    
    const static int BP= 16;  // how many low bits are right of Binary Point
    const static int BP2= BP*2;  // how many low bits are right of Binary Point
    const static int BPhalf= BP/2;  // how many low bits are right of Binary Point
    
    static double STEP() { return 1.0 / (1<<BP); }  // smallest step we can represent
    
    	// for private construction via guts
    	enum FixedRaw { RAW };
    	Fixed(FixedRaw, int guts) : g(guts) {}
    
    
    public:
    	Fixed() : g(0) {}
    	Fixed(const Fixed& a) : g( a.g ) {}
    	Fixed(float a) : g( int(a / (float)STEP()) ) {}
    	Fixed(double a) : g( int(a / (double)STEP()) ) {}
    	Fixed(int a) : g( a << BP ) {}
    	Fixed(long a) : g( a << BP ) {}
    	Fixed& operator =(const Fixed& a) { g= a.g; return *this; }
    	Fixed& operator =(float a) { g= Fixed(a).g; return *this; }
    	Fixed& operator =(double a) { g= Fixed(a).g; return *this; }
    	Fixed& operator =(int a) { g= Fixed(a).g; return *this; }
    	Fixed& operator =(long a) { g= Fixed(a).g; return *this; }
    
    	operator float() { return g * (float)STEP(); }
    	operator double() { return g * (double)STEP(); }
    	operator int() { return g>>BP; }
    	operator long() { return g>>BP; }
    
    	Fixed operator +() const { return Fixed(RAW,g); }
    	Fixed operator -() const { return Fixed(RAW,-g); }
    
    	Fixed operator +(const Fixed& a) const { return Fixed(RAW, g + a.g); }
    	Fixed operator -(const Fixed& a) const { return Fixed(RAW, g - a.g); }
    #if 1
    	// more acurate, using long long
    	Fixed operator *(const Fixed& a) const { return Fixed(RAW,  (int)( ((long long)g * (long long)a.g ) >> BP)); }
    #else
    	// faster, but with only half as many bits right of binary point
    	Fixed operator *(const Fixed& a) const { return Fixed(RAW, (g>>BPhalf) * (a.g>>BPhalf) ); }
    #endif
    	Fixed operator /(const Fixed& a) const { return Fixed(RAW, int( (((long long)g << BP2) / (long long)(a.g)) >> BP) ); }
    
    	Fixed operator +(float a) const { return Fixed(RAW, g + Fixed(a).g); }
    	Fixed operator -(float a) const { return Fixed(RAW, g - Fixed(a).g); }
    	Fixed operator *(float a) const { return Fixed(RAW, (g>>BPhalf) * (Fixed(a).g>>BPhalf) ); }
    	Fixed operator /(float a) const { return Fixed(RAW, int( (((long long)g << BP2) / (long long)(Fixed(a).g)) >> BP) ); }
    
    	Fixed operator +(double a) const { return Fixed(RAW, g + Fixed(a).g); }
    	Fixed operator -(double a) const { return Fixed(RAW, g - Fixed(a).g); }
    	Fixed operator *(double a) const { return Fixed(RAW, (g>>BPhalf) * (Fixed(a).g>>BPhalf) ); }
    	Fixed operator /(double a) const { return Fixed(RAW, int( (((long long)g << BP2) / (long long)(Fixed(a).g)) >> BP) ); }
    
    	Fixed& operator +=(Fixed a) { return *this = *this + a; return *this; }
    	Fixed& operator -=(Fixed a) { return *this = *this - a; return *this; }
    	Fixed& operator *=(Fixed a) { return *this = *this * a; return *this; }
    	Fixed& operator /=(Fixed a) { return *this = *this / a; return *this; }
    
    	Fixed& operator +=(int a) { return *this = *this + (Fixed)a; return *this; }
    	Fixed& operator -=(int a) { return *this = *this - (Fixed)a; return *this; }
    	Fixed& operator *=(int a) { return *this = *this * (Fixed)a; return *this; }
    	Fixed& operator /=(int a) { return *this = *this / (Fixed)a; return *this; }
    
    	Fixed& operator +=(long a) { return *this = *this + (Fixed)a; return *this; }
    	Fixed& operator -=(long a) { return *this = *this - (Fixed)a; return *this; }
    	Fixed& operator *=(long a) { return *this = *this * (Fixed)a; return *this; }
    	Fixed& operator /=(long a) { return *this = *this / (Fixed)a; return *this; }
    
    	Fixed& operator +=(float a) { return *this = *this + a; return *this; }
    	Fixed& operator -=(float a) { return *this = *this - a; return *this; }
    	Fixed& operator *=(float a) { return *this = *this * a; return *this; }
    	Fixed& operator /=(float a) { return *this = *this / a; return *this; }
    
    	Fixed& operator +=(double a) { return *this = *this + a; return *this; }
    	Fixed& operator -=(double a) { return *this = *this - a; return *this; }
    	Fixed& operator *=(double a) { return *this = *this * a; return *this; }
    	Fixed& operator /=(double a) { return *this = *this / a; return *this; }
    
    	bool operator ==(const Fixed& a) const { return g == a.g; }
    	bool operator !=(const Fixed& a) const { return g != a.g; }
    	bool operator <=(const Fixed& a) const { return g <= a.g; }
    	bool operator >=(const Fixed& a) const { return g >= a.g; }
    	bool operator  <(const Fixed& a) const { return g  < a.g; }
    	bool operator  >(const Fixed& a) const { return g  > a.g; }
    
    	bool operator ==(float a) const { return g == Fixed(a).g; }
    	bool operator !=(float a) const { return g != Fixed(a).g; }
    	bool operator <=(float a) const { return g <= Fixed(a).g; }
    	bool operator >=(float a) const { return g >= Fixed(a).g; }
    	bool operator  <(float a) const { return g  < Fixed(a).g; }
    	bool operator  >(float a) const { return g  > Fixed(a).g; }
    
    	bool operator ==(double a) const { return g == Fixed(a).g; }
    	bool operator !=(double a) const { return g != Fixed(a).g; }
    	bool operator <=(double a) const { return g <= Fixed(a).g; }
    	bool operator >=(double a) const { return g >= Fixed(a).g; }
    	bool operator  <(double a) const { return g  < Fixed(a).g; }
    	bool operator  >(double a) const { return g  > Fixed(a).g; }
    };
    
    inline Fixed operator +(float a, const Fixed& b) { return Fixed(a)+b; }
    inline Fixed operator -(float a, const Fixed& b) { return Fixed(a)-b; }
    inline Fixed operator *(float a, const Fixed& b) { return Fixed(a)*b; }
    inline Fixed operator /(float a, const Fixed& b) { return Fixed(a)/b; }
    
    inline bool operator ==(float a, const Fixed& b) { return Fixed(a) == b; }
    inline bool operator !=(float a, const Fixed& b) { return Fixed(a) != b; }
    inline bool operator <=(float a, const Fixed& b) { return Fixed(a) <= b; }
    inline bool operator >=(float a, const Fixed& b) { return Fixed(a) >= b; }
    inline bool operator  <(float a, const Fixed& b) { return Fixed(a)  < b; }
    inline bool operator  >(float a, const Fixed& b) { return Fixed(a)  > b; }
    
    
    
    inline Fixed operator +(double a, const Fixed& b) { return Fixed(a)+b; }
    inline Fixed operator -(double a, const Fixed& b) { return Fixed(a)-b; }
    inline Fixed operator *(double a, const Fixed& b) { return Fixed(a)*b; }
    inline Fixed operator /(double a, const Fixed& b) { return Fixed(a)/b; }
    
    inline bool operator ==(double a, const Fixed& b) { return Fixed(a) == b; }
    inline bool operator !=(double a, const Fixed& b) { return Fixed(a) != b; }
    inline bool operator <=(double a, const Fixed& b) { return Fixed(a) <= b; }
    inline bool operator >=(double a, const Fixed& b) { return Fixed(a) >= b; }
    inline bool operator  <(double a, const Fixed& b) { return Fixed(a)  < b; }
    inline bool operator  >(double a, const Fixed& b) { return Fixed(a)  > b; }
    
    
    int& operator +=(int& a, const Fixed& b) { a = (Fixed)a + b; return a; }
    int& operator -=(int& a, const Fixed& b) { a = (Fixed)a - b; return a; }
    int& operator *=(int& a, const Fixed& b) { a = (Fixed)a * b; return a; }
    int& operator /=(int& a, const Fixed& b) { a = (Fixed)a / b; return a; }
    
    long& operator +=(long& a, const Fixed& b) { a = (Fixed)a + b; return a; }
    long& operator -=(long& a, const Fixed& b) { a = (Fixed)a - b; return a; }
    long& operator *=(long& a, const Fixed& b) { a = (Fixed)a * b; return a; }
    long& operator /=(long& a, const Fixed& b) { a = (Fixed)a / b; return a; }
    
    float& operator +=(float& a, const Fixed& b) { a = a + b; return a; }
    float& operator -=(float& a, const Fixed& b) { a = a - b; return a; }
    float& operator *=(float& a, const Fixed& b) { a = a * b; return a; }
    float& operator /=(float& a, const Fixed& b) { a = a / b; return a; }
    
    double& operator +=(double& a, const Fixed& b) { a = a + b; return a; }
    double& operator -=(double& a, const Fixed& b) { a = a - b; return a; }
    double& operator *=(double& a, const Fixed& b) { a = a * b; return a; }
    double& operator /=(double& a, const Fixed& b) { a = a / b; return a; }
    Last edited by abachler; 09-15-2009 at 11:36 AM.
    Until you can build a working general purpose reprogrammable computer out of basic components from radio shack, you are not fit to call yourself a programmer in my presence. This is cwhizard, signing off.

  10. #25
    Registered User
    Join Date
    Sep 2010
    Posts
    16
    Oh god please no don't do this...
    priceless

  11. #26
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,596
    ...infinite loop of futility.
    Hehe. I'm definitely going to steal that phrase.

    Regardless of the standard I think we can all agree it is good practice to initialize variables before using them.

    As for the topic at hand I don't see any value in complicating the code with fixed point arithmetic when floating point works just fine.

Page 2 of 2 FirstFirst 12
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. How to learn everything
    By audinue in forum General Discussions
    Replies: 18
    Last Post: 09-01-2009, 01:29 AM
  2. Replies: 15
    Last Post: 08-09-2009, 11:20 AM
  3. how long to learn c programming
    By cmay in forum A Brief History of Cprogramming.com
    Replies: 18
    Last Post: 05-13-2009, 12:55 AM
  4. Trying to learn...C++ in VS .net
    By drez in forum C++ Programming
    Replies: 13
    Last Post: 07-27-2003, 09:40 AM
  5. Trying to learn guitar
    By Ben_Robotics in forum A Brief History of Cprogramming.com
    Replies: 12
    Last Post: 07-10-2003, 03:15 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21