Fastest Sigmoid Function

[abachler]

The goal is to produce the smallest and/or fastest sigmoid function. The rules (updated) are:

1. The input is a 0.0f terminated array of double's between negative and positive infinity inclusive.
2. The output is a 0.0f terminated array of doubles between -1.0 and +1.0 and must exhibit the classic properties of a sigmoid. Specifically it must have an S shaped curve and approach +- 1.0 as the input approaches +- infinity.
3. You can use C, C++ or ASM. You can use any combination of those three.
4. The result must be deterministic, i.e. it must produce the same output for the same input every time.
5. For the purposes of testing all code must compile under VC++ 6.0
6. Points will be given for the fastest (most iterations per unit of time). Score will be equal to the number of iterations per second.
7. ASM only solutions will also recieve bonus points for smallest total bytes.
8. For testing the specific processor is -

Intel Xeon 3.20GHz Family F Model 4 Stepping 3 Revision N0
MMX SSE SSE2 SSE3 EM64T
Irwindale Socket 604 mPGA
90 nm technology @ 2.048v

9. No weasel code. This means you Rashakil.


The test itself will be multiple runs testing 1 billion iterations of the routine with random inputs. If the top two or more entries have run times closer than 5% in duration, an extended test will be run on those entries only.

[Rashakil Fol]

Could you be mathematically precise as to what you mean by "the classic properties of a sigmoid"? Be sure to keep in mind floating point precision (e.g. what exactly are the properties of an "inflection point" in a floating point context.)

Also, you should be precise as to what the probability distribution of inputs will be as far as "random inputs" are concerned. Otherwise, victory could end up being luck-based.

Edit: This should satisfy the rules you have given so far. It approximates the sigmoid function arctan(x)/((((((((100!)!)!)!)!)!)!)!) to the limits of double precision.

double sigmoid(double x) { return 0.0; }

[zacs7]

What GPU? I presume it's legal to utilize your GPU ?

[abachler]

Originally Posted by zacs7

What GPU? I presume it's legal to utilize your GPU ?

GPU is an 8800 GTX with 768MB. I suppose if we are going to have GPU entries I will have to provide the entire data set to make it fair. So, I will modify the rules to read -

1. The input is a 0.0f terminated array of double's between negative and positive infinity inclusive.
2. The output is a 0.0f terminated array of doubles between -1.0 and +1.0 and must exhibit the classic properties of a sigmoid. Specifically it must have an S shaped curve and approach +- 1.0 as the input approaches +- infinity.
7. ASM only solutions will also recieve bonus points for smallest total bytes.
9. No weasel code.

Originally Posted by Rashakil Fol

Could you be mathematically precise as to what you mean by "the classic properties of a sigmoid"?

I believe a sigmoid is defined as having an S shaped curve, which means your function is worthless, but just to satisfy the anal retentive nature of some people, I respecced the rule to include infinity, so you cant weasel a win with return 0.0 .

Be sure to keep in mind floating point precision (e.g. what exactly are the properties of an "inflection point" in a floating point context.)

The last time I checked an inflection point is a fairly specific mathmatical concept regardless of whether you are representing the values in double precision floating point or chickens.

Also, you should be precise as to what the probability distribution of inputs will be as far as "random inputs" are concerned. Otherwise, victory could end up being luck-based.

The distribution will be cryptographically random. Every routine will recieve the same set of data. The data will NOT include either infinity, but MAY include one or both, so no weasel code.

Edit: This should satisfy the rules you have given so far. It approximates the sigmoid function arctan(x)/((((((((100!)!)!)!)!)!)!)!) to the limits of double precision.

double sigmoid(double x) { return 0.0; }

I was going to comment on this solution, but I couldnt come up with any responses that didnt contain explitives and involve a lot of eye rolling. For one, its output isnt sigmoidal in any way shape or form, even if you extend the precision from 64 bits to infinity.

[CrazyNorman]

I'd like to compete, using some ASM, but I do not have an available copy of Windows.
Can you extend compilers to allowing something GCC based (also pertains to the latest standard, as opposed to VC++ 6.0)?

[brewbuck]

Originally Posted by Rashakil Fol

Edit: This should satisfy the rules you have given so far. It approximates the sigmoid function arctan(x)/((((((((100!)!)!)!)!)!)!)!) to the limits of double precision.

The traditional definition of sigmoid states that it must have either non-negative or non-positive slope everywhere, as well as a single inflection point. Your function doesn't fit that second part

[Rashakil Fol]

My function approximates one with a single inflection point just fine.

Anyway, here's an implementation that accurately represents the function 2arctan(x/(((((((100!)!)!)!)!)!)!)!))/pi, a sigmoid with one inflection point which has asymptotes -1 and 1:

double sigmoid(double x) { return 0.0; }

The traditional definition of sigmoid states that it must have either non-negative or non-positive slope everywhere

If you're mapping (-infinity, +infinity) monotonically into (-1,1) or [-1,1] using doubles, you're going to have either significant regions of slope zero, or multiple inflection 'points'. That's what you get when approximating functions' behavior, no matter what the function is. Suppose I wrote

double sigmoid(double x) { return atan(x); }

What do you think the behavior of this function is for large and small values of x? There it's perfectly constant.

Also, where do you think its inflection point is? And where's the inflection point of the following function:

double sigmoid(double x) {
  if (fabs(x) < -3.357e-10) {
    return x;
  }
  else {
    return atan(x);
  }
}

[abachler]

Originally Posted by CrazyNorman

Can you extend compilers to allowing something GCC based (also pertains to the latest standard, as opposed to VC++ 6.0)?

Well, Ill be compiling it on a VC 6 machine. In truth, I dont think the function should be so complex that you require Linux specific OS calls.

As for rashikil's lame attempts to prove his intelligence, Id say he has succeeded, although not in the way he probably intended.

Originally Posted by Rashakil Fol

double sigmoid(double x) { return 0.0; }

defines a line with slope 0.0 and zero inflection points. Since an input of infinity into your equation produces erronious results, it is hence disqualified.

[CrazyNorman]

I was planning on using inline ASM. As not being part of the standard language, GCC and VC++ use a different syntax for inline ASM, plus GCC uses AT&T style ASM (src dst), while VC++ uses Intel style (dst src).

[abachler]

as long as you let me know that, and the code isnt too long, I think I can manage to touch it up for testing.

[Rashakil Fol]

Originally Posted by abachler

defines a line with slope 0.0 and zero inflection points. Since an input of infinity into your equation produces erronious results, it is hence disqualified.

Now you're making up new rules that weren't in your original set of rules. Sigmoid functions are functions mapping the real line into a subset (a,b) of the real line; infinity is not part of the domain. It's a good idea to have all the rules of the contest listed, if you want the function to handle non-real doubles.

But, if you insist...

double sigmoid(double x) { return (x > 0) - (x < 0); }

This implements the function f(x) = 2atan(x*((((((((100!)!)!)!)!)!)!)!))/pi.

Edit: Of course, this would have to be modified if the input is an array of numbers now.

[abachler]

Well, im tired of you claiming this as an entry, so heres the deal. Return 0.0 does not compute a sigmoid. return (x > 0) - (x < 0) returns type bool or an integer depending on your compiler, so it violates rule #2 even as originally defined. Even if you recast it as a double, it would fail based on the requirement that the output exhibit 'the classical properties of a sigmoid'. i.e. a non-zero slope at all points.

entry disqualified.

Before you try to provide some other entry, please go read what a sigmoid function actually is. Its a very specific type of function that none of your entries even approximate as you claim.

[abachler]

Originally Posted by Rashakil Fol

Edit: Of course, this would have to be modified if the input is an array of numbers now.

You can specify if you want the function to be called sequentially, or to be passed the array. The array is to make it fair for GPU entries, since they generally process several data points at once.

[whiteflags]

I still find it confusing frankly. I can't help but feel that you are ignoring what Rakashil Fol is really attempting to say.

If you're mapping (-infinity, +infinity) monotonically into (-1,1) or [-1,1] using doubles, you're going to have either significant regions of slope zero, or multiple inflection 'points'. That's what you get when approximating functions' behavior, no matter what the function is. Suppose I wrote

Sigmoid functions are functions mapping the real line into a subset (a,b) of the real line; infinity is not part of the domain.

Do you intend to ignore these, and the other points about floating point numbers? Fuzzy math will make correct entries wrong. I don't think anyone will want to participate if the contest judge - I hate the term - doesn't know what he's doing.

If the goal of the contest were to plot a sigmoid, then the fastest entry would not actually draw a sigmoid at all, but instead calculate where to plot. Nevermind whether doubles are appropriate for this purpose. Do you intend to write code that would draw a sigmoid curve based on the output, or will those kinds be disqualified also? Would that be weasel code?

[abachler]

No, I use sigmoids all the time. Rashakil is trying to use 'return 0;' style weasel code and then claim it computes a sigmoid. It performs no calculation whatsoever, sigmoid or otherwise. The requirement only to exhibit the features of a sigmoid was intentionally left vague to allow for creative entries, not to act as a loophole for rules lawyers. Yes I understand that a floating point number cant quantize infinite granularity. However the data set returned by ALL his entries fail on multiple points, and yet he refuses to accept that they are not valid entries. I changed the rules, not because they were flawed, but just to spell it out in really small words that even a retarded gorilla can understand that entries must actually compute a sigmoid, not just return a fixed value. Hey im sorry I didnt put in the rule about no weasel code up front. I honestly expected people to submit realistic entries, not to try to claim that since some specific subset of a function cannot be computed within the limits of the hardware, that therefor all attempts at computing any function of that type in hardware can be represetned by a single fixed value. so for those of you that want ot exploit the limitations of the hardware to avoid actual computation, here is a specific requirement of the function.

For all sets of 3 points a , b , and c where a < b < c and all 3 points are in either the positive or negative domain ,the following three conditions must be met:
1. f(a) < f(b) < f(c)
2. the slopes of the function of a , b , c (sa , sb , sc) must exhibit the quality of sa > sb > sc for sets of positive numbers
3. the slopes of the function of a , b , c (sa , sb , sc) must exhibit the quality of sa < sb < sc for sets of negative numbers