issues in float division a/b

This is a discussion on issues in float division a/b within the C# Programming forums, part of the General Programming Boards category; Thanks Mats, Looks like the register can not hold a decimal, right? Decimal is larger than 64-bit. :-) Originally Posted ...

  1. #16
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Thanks Mats,


    Looks like the register can not hold a decimal, right? Decimal is larger than 64-bit. :-)

    Quote Originally Posted by matsp View Post
    Actually, x87 registers are 80-bit, so even intermediate results in 64-bit (double) would potentially have further precision[1], SSE registers are 64- or 32-bit. (In Windows-64, you'd be using SSE, not x87 for math).

    [1] Although I think Windows defaults the x87 settings to "round to 64-bit on intermediate results".
    --
    Mats

    regards,
    George

  2. #17
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by George2 View Post
    Thanks Mats,


    Looks like the register can not hold a decimal, right? Decimal is larger than 64-bit. :-)




    regards,
    George
    I had a look at the specification of "decimal", and it seems like it's not implemented using the FPU - not quite sure how it's actually implemented. It's also stated that its a 128 bit value, so no, you can't use FPU registers to hold such a value.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #18
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Thanks for your great answers, Mats!


    You are so knowledgable and nice, all of my questions are answered in this thread. Cool!! :-)

    Quote Originally Posted by matsp View Post
    I had a look at the specification of "decimal", and it seems like it's not implemented using the FPU - not quite sure how it's actually implemented. It's also stated that its a 128 bit value, so no, you can't use FPU registers to hold such a value.

    --
    Mats

    regards,
    George

Page 2 of 2 FirstFirst 12
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 2
    Last Post: 05-13-2009, 03:25 PM
  2. Replies: 14
    Last Post: 06-28-2006, 01:58 AM
  3. Could somebody please help me with this C program
    By brett73 in forum C Programming
    Replies: 6
    Last Post: 11-25-2004, 01:19 AM
  4. Half-life SDK, where are the constants?
    By bennyandthejets in forum Game Programming
    Replies: 29
    Last Post: 08-25-2003, 11:58 AM
  5. How do you search & sort an array?
    By sketchit in forum C Programming
    Replies: 30
    Last Post: 11-03-2001, 04:26 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21