This is a sequel to my 2009 post about division of long integers.
I am occasionally asked why this code produces a bizarre error message:
Console.WriteLine(Math.Round(i / 6000000000, 5));
i is an integer.
The error is:
The call is ambiguous between the following methods: 'System.Math.Round(double, int)' and 'System.Math.Round(decimal, int)'
Um, what the heck?
What is going on here is: division of an
int by a
long converts the
int to a
long and produces a
long as its result. That’s the fundamental problem here: the user wants a fraction rounded to five decimal places, but what they are getting is not even a fraction; integer arithmetic rounds automatically. So this code is just wrong.
Unfortunately the compiler is allowing the bogus division to pass without warning, and then informing the developer that there is no overload of
Round that takes a
long. There are applicable overloads that take
decimal but there is no basis upon which to deduce which one the developer wants, so that’s an error.
The real danger here is that the developer will attempt to make the fix by attacking the stated error, rather than attacking the actual conceptual error. I have seen this “fix”:
Console.WriteLine(Math.Round((double)(i / 6000000000), 5));
Again, that does the integer arithmetic rounding first, and then converts the result to
double, which will already be rounded off to a whole number, not to five decimal places.
The right way to do this is to not use integers:
Console.WriteLine(Math.Round(i / 6000000000.0, 5)); // double Console.WriteLine(Math.Round(i / 6000000000.0m, 5)); // decimal
We see this sort of error so often that we’ve actually made a checker for it in the Coverity code quality analyzer; this turns up more frequently than you might think in real-world code.