Thread: C# calculation problem

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    What do you think the result should be if you calculate:
    2.6666666666666665 + -4.6666666666666661 + 2.0

    Those are the partial results from your calculation:
    narray[y] = (a * (frac[y] * frac[y])) + (b * (frac[y] * -1)) + c;

    I find that the result should be zero - but because of rounding errors, you get a "very small number" - this is perfectly normal when using floating point calculations, especially subtracting or adding the result of other calculations.

    Remember, floating point (double or float) calculations are approximations - not exact results.

    Edit: I'm pretty sure it's more efficient to use -frac[y] instead of frac[y] * -1...

    --
    Mats
    Last edited by matsp; 10-18-2007 at 04:09 PM.
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. A question related to strcmp
    By meili100 in forum C++ Programming
    Replies: 6
    Last Post: 07-07-2007, 02:51 PM
  2. WS_POPUP, continuation of old problem
    By blurrymadness in forum Windows Programming
    Replies: 1
    Last Post: 04-20-2007, 06:54 PM
  3. Calculation problem....
    By badran in forum C++ Programming
    Replies: 6
    Last Post: 10-17-2006, 04:07 AM
  4. Laptop Problem
    By Boomba in forum Tech Board
    Replies: 1
    Last Post: 03-07-2006, 06:24 PM
  5. Trigonometry Sin rule calculation problem
    By Harry in forum C++ Programming
    Replies: 3
    Last Post: 02-06-2006, 12:14 PM