when i have

this:

selection end up being:Code:`int row=100000;`

int col = 100000;

int prec =1;

long selection = ((row * col) * prec)/100;

14100654.

i should be getting 100000000;

Printable View

- 09-11-2010spikestarlong int issue
when i have

this:

Code:`int row=100000;`

int col = 100000;

int prec =1;

long selection = ((row * col) * prec)/100;

14100654.

i should be getting 100000000; - 09-11-2010Lesshardtofind
My guess is when you multiply 100000 * 100000 * 1 the result is past the limit of a long. So it screws up the number when it passes the roof then divides that screwed up number by 100. If you drop a 0 of the end of row and column you notice it will display the appropriate result.

climits (limits.h) - C++ Reference

the limit listed of a long is

2147483647

the result of 100000 * 100000

10000000000

I don't know for sure that this is causing the problem but it would be my first guess. - 09-11-2010msh
What he said. :)

You can explore various datatype limits like so:

Code:`#include <iostream>`

#include <limits>

using namespace std;

int main(void)

{

// Replace**long**as you please.

cout << "Upper limit of long datatype: " << numeric_limits<long>::max() << endl;

cout << "Lower limit of long datatype: " << numeric_limits<long>::min() << endl;

}

- 09-11-2010grumpy
In the code provided in the original post, row and col are int. so row*col is computed as an int. If an int has smaller range than a long, that will also overflow.

- 09-12-2010chameleons
"long selection = ((row * col) * prec)/100;"

Even if your compiler's "long" is 64bit, and can store 10^10, this will not work, because all of the operands are int, the result will be truncated to int.

You need to cast one of "row" or "col" to "long" first, for example

"long selection = ((long)row)*col*prec/100;" - 09-12-2010Elysia
((row * col) * prec)/100

**l**might work as well.

(Notice the trailing**l**.) - 09-12-2010grumpy
No it wouldn't. row*col will be computed as an int, as will ((row*col)*prec). The result would then be promoted to long, in order to divide by 100L.

Incidentally, from a readability perspective, I personally prefer 100L over 100l. I realise others may have different preferences, but lower case L looks too much like the bitwise "or" operator (or like the digit 1) on a lot of screens. - 09-12-2010BMJ
log2(100000 * 100000) is about 33.2 which means that the intermediate value would exceed a 32bit limit. You're also doing division so I wonder if floating point numbers would be more appropriate for your needs? Or else you could use 64bit integer values:

Code:`int row = 100000;`

int col = 100000;

int prec = 1;

long long selection = (((static_cast<long long>(row) * col)) * prec) / 100;

- 09-12-2010C_ntua
Sometimes it is confusing on where to cast or specify the trailing L or d. But I think the rule is that

type1 operator type2 -> type3, where type3 the biggest in size of type1, type2.

if there are integral vs decimal types, the integral are casted to decimals.

I am not sure if float * int would result in float or to double, but I would think to float. - 09-12-2010laserlightQuote:

But I think the rule is that

type1 operator type2 -> type3, where type3 the biggest in size of type1, type2.

if there are integral vs decimal types, the integral are casted to decimals.

Quote:

Originally Posted by**C++03 Section 5 Paragraph 9**