Fastest Sigmoid Function

This is a discussion on Fastest Sigmoid Function within the Contests Board forums, part of the Community Boards category; It compiles just fine on VC++ 6.0, the stated target compiler. Perhaps you misread wikipedia, or just dont understand what ...

  1. #31
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,189
    It compiles just fine on VC++ 6.0, the stated target compiler. Perhaps you misread wikipedia, or just dont understand what monotonic means.

    From wiktionary.org -
    Adjective
    monotonic
    1. of or using the Greek system of diacritics which discards the breathings and employs a single accent to indicate stress.
    2. (mathematics) said of a function that either never decreases or never increases as its independent variable increases.

    From Wikipedia.org -

    In general, a sigmoid function is real-valued and differentiable, having either a non-negative or non-positive first derivative and exactly one inflection point.

    So, while I suppose a sigmoid is monotonic ( I may have mispoken earlier), not all monotonic functions are sigmoids. I think that citizens formula may produce bad results for negative values of X lets see -5/sqrt(25+1)? Nope, its a negative number that decreases as the variable decreases. It approaches 1.0 for large values of X. For a< b < c in the positive domain it satsifies f(a) < f(b) < f(c) and sa > sb > sc. It also satisfies the negative domain equivelant. So it qualifies. If he implements it in assembly it will not produce errors, although depending on the compiler, it may produce errors for values of X greater than sqrt(1.7E+308) when compiled as is.
    Last edited by abachler; 07-11-2007 at 04:34 PM.
    Until you can build a working general purpose reprogrammable computer out of basic components from radio shack, you are not fit to call yourself a programmer in my presence. This is cwhizard, signing off.

  2. #32
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    It appears that Rashakil understands the definition of a monotonic function perfectly well. He stated that sigmoidal functions are monotonic, not that all monotonic functions are sigmoidal.
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  3. #33
    aoeuhtns
    Join Date
    Jul 2005
    Posts
    581
    As I stated. Glad you agree. So, citizen's entry is not monotonic. Therefore, it's not a sigmoid. Please disqualify it.

    Also, I doubt your implementation compiles on VC++ 6.0.

    Code:
    void sigmoid(double* Input){
         double temp;
     
         while(Input[0] != 0.0){
              __asm fld2e;
              __asm fstp temp;
              temp = pow(2 , temp);
              temp = pow(temp , (0-X));
              temp += 1.0;
              Input[0] = pow(temp , -1.0);
              Input++;
              }
     
         return;
         }
    This relies on a global variable X and doesn't look at the input parameter at all.

    And on VC++ 2003, I'm getting a syntax error before then.
    Last edited by Rashakil Fol; 07-11-2007 at 04:25 PM.
    There are 10 types of people in this world, those who cringed when reading the beginning of this sentence and those who salivated to how superior they are for understanding something as simple as binary.

  4. #34
    aoeuhtns
    Join Date
    Jul 2005
    Posts
    581
    Fixing up your function so that I could compile it, I changed it to the following:

    Code:
    void sigmoid(double* Input) {
         double temp;
    
         while(Input[0] != 0.0) {
              temp = 2.71828182845904523536028747135662497757247; /* hope that's right */
              temp = pow(temp , (0-Input[0]));
              temp += 1.0;
              Input[0] = pow(temp , -1.0);
              Input++;
              }
    
         return;
         }
    It turns out that your function has a constant slope on the interval [36.737,5000]. This fails to meet the requirements you have given.
    There are 10 types of people in this world, those who cringed when reading the beginning of this sentence and those who salivated to how superior they are for understanding something as simple as binary.

  5. #35
    S Sang-drax's Avatar
    Join Date
    May 2002
    Location
    Göteborg, Sweden
    Posts
    2,072
    It's easy to prove that it's impossible to submit a valid entry to this contest:
    1. There are a finite number of possible values a double can take.
    2. The number of values a double in the interval [-1,1] can take is thus smaller.
    3. By the pigeonhole principle, there is no bijective function between R and [0,1] (using doubles).
    4. There is no sigmoid function using doubles.
    Last edited by Sang-drax : Tomorrow at 02:21 AM. Reason: Time travelling

  6. #36
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,189
    Quote Originally Posted by Rashakil Fol View Post
    It turns out that your function has a constant slope on the interval [36.737,5000]. This fails to meet the requirements you have given.

    T = 36.737
    e = 2.718281828459045235360287471356

    1 / (1 + e ^-T) = 0.99999999999999988899983654344547

    T = 5000

    1 / (1 + e ^ -T) = 1.0

    seems there is plenty of room in between seeing as how the FPU uses 80 bits internally, and even if my example fails above values of 36 or so, thats certainly better than any equation you have submitted, all of which fail at every value. Personally I woudlnt use this particular example (its just the one provided on wiki so giving it doesnt give an unfair advantage to any contestant), and the one I use right now doesnt fail until the values exceed 60 million or so, which is well beyond what our application of sigmoids would ever require. if you want a mathmatically pure solution, then implementing on hardware will never work, but a pure solution is worthless if you cant make toys with it.

    But since we now have two people claiming that sigmoids are impossible to implement in hardware, I will now tell my boss that neural networks are impossible to implement on computers and he should just burn his PhD and go farm goats in a cave. I figured it would be a nice change of pace to have a serious contest, but I can see you would rather cry about the limitations of hardware than work on anything more complicated than a tic tac toe game or shortest path algorithms.
    Last edited by abachler; 07-12-2007 at 10:09 AM.
    Until you can build a working general purpose reprogrammable computer out of basic components from radio shack, you are not fit to call yourself a programmer in my presence. This is cwhizard, signing off.

  7. #37
    Amazingly beautiful user.
    Join Date
    Jul 2005
    Location
    If you knew I'd have to kill you
    Posts
    254
    Maybe you should just change the function from a sigmoid to something more specific, so we can end this little flamewar and have a competition.
    Programming Your Mom. http://www.dandongs.com/

  8. #38
    Just Lurking Dave_Sinkula's Avatar
    Join Date
    Oct 2002
    Posts
    5,006
    >80 bit
    FWIW
    7. It is easier to write an incorrect program than understand a correct one.
    40. There are two ways to write error-free programs; only the third one works.*

  9. #39
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,189
    Quote Originally Posted by Dave_Sinkula View Post
    >80 bit
    FWIW
    The FPU uses 80 bit internally, regardless of the limits of the double, or if long doubles map to doubles, the FPU still converts doubles to 80 bit precision.

    FLD
    Load Floating Point Value

    Description
    Pushes the source operand onto the FPU register stack. The source operand can be in singleprecision,
    double-precision, or double extended-precision floating-point format. If the source
    operand is in single-precision or double-precision floating-point format, it is automatically
    converted to the double extended-precision floating-point format before being pushed on the
    stack.
    Until you can build a working general purpose reprogrammable computer out of basic components from radio shack, you are not fit to call yourself a programmer in my presence. This is cwhizard, signing off.

  10. #40
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    21,717
    Maybe you should just change the function from a sigmoid to something more specific, so we can end this little flamewar and have a competition.
    I agree, it looks like this has become a contest about the contest.
    C + C++ Compiler: MinGW port of GCC
    Version Control System: Bazaar

    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  11. #41
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,189
    I concur. Therefor I am closing the contest for new entries. Since citizen is the only one that spent 5 minutes writing an entry instead of 5 hours looking for loopholes, citizen is the winner.
    Last edited by abachler; 07-12-2007 at 10:57 AM.
    Until you can build a working general purpose reprogrammable computer out of basic components from radio shack, you are not fit to call yourself a programmer in my presence. This is cwhizard, signing off.

  12. #42
    aoeuhtns
    Join Date
    Jul 2005
    Posts
    581
    Quote Originally Posted by abachler View Post
    The FPU uses 80 bit internally, regardless of the limits of the double, or if long doubles map to doubles, the FPU still converts doubles to 80 bit precision.
    Whether the FPU uses 64 bit or 52 bit precision or whatnot is not relevant. The output of the function is an array of doubles; what the number "could have been" isn't important.

    All you need to do is clearly specify what restrictions you want to specifiy which implementations are acceptable, instead of having a vague definition. So please do.
    There are 10 types of people in this world, those who cringed when reading the beginning of this sentence and those who salivated to how superior they are for understanding something as simple as binary.

  13. #43
    Registered User
    Join Date
    Sep 2001
    Posts
    752
    The issue here is that floating point math is almost always useless in the realm of pure math, and you have put forward a pure math problem, with the requirement that we'll use floating point math to solve it.

    First we have requirements from the wiki article...
    • Real-Valued
    • Differentiable
    • Positive Slope
    • Having exactly one inflection point


    Additional requirements we've gotten from the thread.
    • The function must approach -1 as it tends to negative infinity
    • The function must approach +1 as it tends to positive infinity
    • Non-Zero slope at all points


    Now, let's look at the properties of a double precision method...
    • It is inherently not continuous.
    • -IF- you play connect-the-dots to cheat a continuous function, then it is not differentiable.

    Already, ideas like "limit", and "slope" are pretty much doomed.
    • A double function which maps from (-inf, inf) to (-1, 1) will have over 99.9% of its adjacent-point ranges have 0 slope



    The traditional way to handle this is to simply let the double function approximate a real valued function. So, you could define the problem as...
    Write a function which approximates a Sigmoid Family Curve.
    In which case, returning 0 for all values is perfectly correct.

    Accept the fact that giving pure math requirements for a double precision method is hopeless. If if is impossible to programatically verify the correctness of an answer, then the question was bad. Give us some code which the function must fulfill and we'll give answers.


    One other thing... "Weasal Code" is the spirit of mathematical programming. You can't get more weasely than some beautiful and amazing code like
    Code:
    double d;
    //...
    long i = *(long *)d;
    Callou collei we'll code the way
    Of prime numbers and pings!

Page 3 of 3 FirstFirst 123
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Compiling sample DarkGDK Program
    By Phyxashun in forum Game Programming
    Replies: 6
    Last Post: 01-27-2009, 02:07 AM
  2. Seg Fault in Compare Function
    By tytelizgal in forum C Programming
    Replies: 1
    Last Post: 10-25-2008, 03:06 PM
  3. Another syntax error
    By caldeira in forum C Programming
    Replies: 31
    Last Post: 09-05-2008, 01:01 AM
  4. Replies: 28
    Last Post: 07-16-2006, 11:35 PM
  5. const at the end of a sub routine?
    By Kleid-0 in forum C++ Programming
    Replies: 14
    Last Post: 10-23-2005, 06:44 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21