Perceptron Question

This is a discussion on Perceptron Question within the Tech Board forums, part of the Community Boards category; What exactly is the point of the bias input in a perceptron?...

  1. #1
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718

    Perceptron Question

    What exactly is the point of the bias input in a perceptron?
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  2. #2
    Cat
    Cat is offline
    Registered User
    Join Date
    May 2003
    Posts
    1,571
    A perceptron is essentially an equation for a hyperplane in the N-dimentional input space. E.g. for two inputs, it describes a line; for three, it describes a plane. After learning, the hyperplane tries to partition the input space into two regions.

    The bias input is needed to create a zeroth-order term, a term which doesn't vary with any inputs. E.g. one form of the generic line equation could be written as Y = MX + B. B is a constant (zeroth order) term.

    Without a bias input, a perceptron couldn't describe an arbitrary hyperplane, because all permutations would inevitably pass through the origin. E.g. no matter what the W vector was, if the input vector was all zero, the output would be all zero, unless you have the bias.

  3. #3
    Toaster Zach L.'s Avatar
    Join Date
    Aug 2001
    Posts
    2,686
    The output of a perceptron is determined by the dot product of the input vector with its associated weight vector plus the bias.

    I*W+b > 0 -> 1
    I*W+b <= 0 -> 0

    So, as Cat was saying, it is essentially the constant "shift" in the plane which divides space between "on" and "off" for the perceptron.
    The word rap as it applies to music is the result of a peculiar phonological rule which has stripped the word of its initial voiceless velar stop.

  4. #4
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    But, according to AI Horizon, the bias has a weight itself. Doesn't that mean that the actual value of the bias is irrelevant, as long as it is non-zero, as the weight would compensate?
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  5. #5
    Cat
    Cat is offline
    Registered User
    Join Date
    May 2003
    Posts
    1,571
    There are 2 ways to add a bias:

    1) Addition after the dot product is taken
    2) As a weight (most common method)

    For example, say X and Y are variables, and your perceptron implements:

    2X + 4Y + 8 > 0

    You could do this as:

    I = [X Y], W = [2 4], B = 8

    but then you have to account for B more often, and being as B is updated just like W, it's more convenient to make it the last entry in the W-vector, like this:

    I = [X Y 1], W = [2 4 8].

    In this case, you add another input to the perceptron and fix that input's value at 1. Then the weight associated with this input acts as the bias. This is the more common method of implementing a perceptron; it reduces complexity by treating the bias like any other weight.

    The bias doesn't have a weight, it *IS* a weight in this model.

  6. #6
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    So, if I use the second model, appending the bias to the inputs, the actual value of it is meaningless, as long as it is constant?

    Edit:
    Also, does it matter what the threshold value is as long as it remains constant?
    Last edited by XSquared; 06-16-2003 at 08:01 PM.
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  7. #7
    Cat
    Cat is offline
    Registered User
    Join Date
    May 2003
    Posts
    1,571
    Originally posted by XSquared
    So, if I use the second model, appending the bias to the inputs, the actual value of it is meaningless, as long as it is constant?

    Edit:
    Also, does it matter what the threshold value is as long as it remains constant?
    1) The bias is not the last input, the bias is the last WEIGHT. Technically, you could append any nonzero constant to the input vector, but tradition says use 1.

    2) Technically, it could be anything, but 0 is the normal threshold.

  8. #8
    Pursuing knowledge confuted's Avatar
    Join Date
    Jun 2002
    Posts
    1,916
    I read (and originally posted the link to) the AI Horizon article, and I've read this thread, but it isn't enough for me to really understand what is going on with a Perceptron. Perhaps it's a bit over my head right now, or maybe I just haven't read enough... but has anyone here implemented a program that uses perceptrons for something? The simpler the better, obviously, when trying to learn, but anything will help. So... anyone?
    Away.

  9. #9
    Cat
    Cat is offline
    Registered User
    Join Date
    May 2003
    Posts
    1,571
    Yes, but it was far from simple... I did a MLP (multilayer perceptron) program once for an advanced course in neural networks.

    Dunno how much source I have left, that was years ago under a pretty bad (non-ANSI) compiler.

    The biggest thing is to get a library for vector/matrix math.

  10. #10
    Toaster Zach L.'s Avatar
    Join Date
    Aug 2001
    Posts
    2,686
    www.generation5.org has a small example I believe, as well as a decent introductory article.

    >>> The biggest thing is to get a library for vector/matrix math.

    I'd go with Blitz++.
    The word rap as it applies to music is the result of a peculiar phonological rule which has stripped the word of its initial voiceless velar stop.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Alice....
    By Lurker in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 06-20-2005, 02:51 PM
  2. Debugging question
    By o_0 in forum C Programming
    Replies: 9
    Last Post: 10-10-2004, 05:51 PM
  3. Question about pointers #2
    By maxhavoc in forum C++ Programming
    Replies: 28
    Last Post: 06-21-2004, 12:52 PM
  4. Question...
    By TechWins in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 07-28-2003, 09:47 PM
  5. Question, question!
    By oskilian in forum A Brief History of Cprogramming.com
    Replies: 5
    Last Post: 12-24-2001, 12:47 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21