long int issue

when i have
this:

Code:

---

int row=100000;
int col = 100000;
int prec =1;
long selection = ((row * col) * prec)/100;

---

selection end up being:
14100654.

i should be getting 100000000;

My guess is when you multiply 100000 * 100000 * 1 the result is past the limit of a long. So it screws up the number when it passes the roof then divides that screwed up number by 100. If you drop a 0 of the end of row and column you notice it will display the appropriate result.

climits (limits.h) - C++ Reference

the limit listed of a long is
2147483647
the result of 100000 * 100000
10000000000

I don't know for sure that this is causing the problem but it would be my first guess.

What he said. :)

You can explore various datatype limits like so:

Code:

---

#include <iostream>
#include <limits>

using namespace std;

int main(void)
{
    // Replace long as you please.
    cout << "Upper limit of long datatype: " << numeric_limits<long>::max() << endl;
    cout << "Lower limit of long datatype: " << numeric_limits<long>::min() << endl;

}

---

In the code provided in the original post, row and col are int. so row*col is computed as an int. If an int has smaller range than a long, that will also overflow.

"long selection = ((row * col) * prec)/100;"
Even if your compiler's "long" is 64bit, and can store 10^10, this will not work, because all of the operands are int, the result will be truncated to int.
You need to cast one of "row" or "col" to "long" first, for example
"long selection = ((long)row)*col*prec/100;"

((row * col) * prec)/100l might work as well.
(Notice the trailing l.)

Quote:

---

Originally Posted by Elysia

((row * col) * prec)/100l might work as well.
(Notice the trailing l.)

---

No it wouldn't. row*col will be computed as an int, as will ((row*col)*prec). The result would then be promoted to long, in order to divide by 100L.

Incidentally, from a readability perspective, I personally prefer 100L over 100l. I realise others may have different preferences, but lower case L looks too much like the bitwise "or" operator (or like the digit 1) on a lot of screens.

log2(100000 * 100000) is about 33.2 which means that the intermediate value would exceed a 32bit limit. You're also doing division so I wonder if floating point numbers would be more appropriate for your needs? Or else you could use 64bit integer values:

Code:

---

int row = 100000;
int col = 100000;
int prec = 1;
long long selection = (((static_cast<long long>(row) * col)) * prec) / 100;

---

Notice that row is upcast to a 64bit integer so that the intermediate result doesn't get truncated.

Sometimes it is confusing on where to cast or specify the trailing L or d. But I think the rule is that

type1 operator type2 -> type3, where type3 the biggest in size of type1, type2.
if there are integral vs decimal types, the integral are casted to decimals.

I am not sure if float * int would result in float or to double, but I would think to float.

Quote:

---

But I think the rule is that

type1 operator type2 -> type3, where type3 the biggest in size of type1, type2.
if there are integral vs decimal types, the integral are casted to decimals.

---

That is a rough summary, but the actual rules are:

Quote:

---

Originally Posted by C++03 Section 5 Paragraph 9

Many binary operators that expect operands of arithmetic or enumeration type cause conversions and yield result types in a similar way. The purpose is to yield a common type, which is also the type of the result. This pattern is called the usual arithmetic conversions, which are defined as follows:

• If either operand is of type long double, the other shall be converted to long double.
• Otherwise, if either operand is double, the other shall be converted to double.
• Otherwise, if either operand is float, the other shall be converted to float.
• Otherwise, the integral promotions shall be performed on both operands.
• Then, if either operand is unsigned long the other shall be converted to unsigned long.
• Otherwise, if one operand is a long int and the other unsigned int, then if a long int can represent all the values of an unsigned int, the unsigned int shall be converted to a long int; otherwise both operands shall be converted to unsigned long int.
• Otherwise, if either operand is long, the other shall be converted to long.
• Otherwise, if either operand is unsigned, the other shall be converted to unsigned.

[Note: otherwise, the only remaining case is that both operands are int ]

---