Trying to round result of a computation to 2 digits of precision

Database
Enthusiast

Trying to round result of a computation to 2 digits of precision

Hi,  I read through the forum on such topics as CAST and ROUND, but somehow I am not having any luck.  I have this expression in my select:

Select cast (PROJ_INV_CS as float) / case

when GRS_POS_FCST_CS > 0 then cast(GRS_POS_FCST_CS as float)

else .0000000001 end

as WOS

This works great and is very accurate, however I want to round the result to 2 digits of precision.  When I try to cast the expression, I get numeric overflow.  when I try to round, I get numbers with the same degree of precision, they just differ in the digits after the number of digits I am rounding to.  Thanks in advance...

1 REPLY
Junior Contributor

Re: Trying to round result of a computation to 2 digits of precision

What is the maximum values returned by that calculation?

I would not use FLOAT as it's precision is limited to approx. 15 digits.

Better use DECIMAL instead, which supports up to 38 digits.

Dieter