Thread: Round off error?

  1. #1
    Registered User rogster001's Avatar
    Join Date
    Aug 2006
    Location
    Liverpool UK
    Posts
    1,472

    Round off error?

    The result of testvar2 below keeps returning as 1.000, when it should be 1.040, anybody any ideas why? i have tried it with allowing more decimal points when printing. I have

    instead of doing a single line calculation i even broke it into seperate statements to make sure no precedence issues.
    and the last statement shows me the correct answer so something is going on with storage.
    I also compiled the code below which names the variables a bit differently after restarting the ide in a new file to get rid of any evil spirits and /or Hexes on the program and same result

    Code:
    int main()
    {
    
    double testvar1 = 0;
    double testvar2 = 0;
    double Xacr = 1;
    double Hres = 80;
    
                    testvar1 = Xacr / Hres;
                    testvar2 = testvar1 * Hres;
    
    		printf("%.3f, %.3f\n",testvar1, testvar2);
    		printf("%.3f\n", 0.013 * 80);
    
                                    
    system("PAUSE");
    
    return 0;
    }

  2. #2
    Registered User rogster001's Avatar
    Join Date
    Aug 2006
    Location
    Liverpool UK
    Posts
    1,472
    cancel this post, output was correct, i was reading it all wrong. and putting 4decimal places output on the CORRECT variable fixed it.

  3. #3
    The larch
    Join Date
    May 2006
    Posts
    3,573
    1.0 divided by 80.0 equals 0.0125, so using 0.013 in the last line is indeed a significant round-off error (on your part, not the program's).
    I might be wrong.

    Thank you, anon. You sure know how to recognize different types of trees from quite a long way away.
    Quoted more than 1000 times (I hope).

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Getting an error with OpenGL: collect2: ld returned 1 exit status
    By Lorgon Jortle in forum C++ Programming
    Replies: 6
    Last Post: 05-08-2009, 08:18 PM
  2. An error is driving me nuts!
    By ulillillia in forum C Programming
    Replies: 5
    Last Post: 04-04-2009, 09:15 PM
  3. Making C DLL using MSVC++ 2005
    By chico1st in forum C Programming
    Replies: 26
    Last Post: 05-28-2008, 01:17 PM
  4. We Got _DEBUG Errors
    By Tonto in forum Windows Programming
    Replies: 5
    Last Post: 12-22-2006, 05:45 PM
  5. Learning OpenGL
    By HQSneaker in forum C++ Programming
    Replies: 7
    Last Post: 08-06-2004, 08:57 AM