Database

turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

06-11-2013
04:51 AM

06-11-2013
04:51 AM

Hi

Below things wonder me.When I perform

SEL (100.00/210.00) GIVES 0.48

But if perform

SEL (29781215.00/29894915.00) gives 1.00 instead of 0.99

Then i did in the below way.It worked

SEL 100* CAST(29781215 AS DECIMAL(10,2))/29894915 gives 99.62

The only thing is the first two queries.

Regards

KVB

4 REPLIES

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

06-11-2013
05:32 AM

06-11-2013
05:32 AM

Hi KVB,

these are not Integers, but Decimals.

Decimals get rounded and the result of your division is 0.9961966775955041..., which is correctly rounded to 1.00.

Dieter

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

06-11-2013
05:56 AM

06-11-2013
05:56 AM

Oh sorry.Yes the yare decimals.Then also for the first query it is not being rounded off and for the second one it is being rounded.Here is the confusion arised for me.

KVB

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

06-11-2013
06:43 AM

06-11-2013
06:43 AM

Hi KVB,

the first dvision is also rounded: SEL (100.00/210.00) = 0.476190476190476

When you add/substract/divide two decimals the resulting precision is the higher of the two operands, e.g.

dec(a,b) + dec(x,y) -> fractional digits = max(b,y)

Multiplying results in adding the fractional digits:

dec(a,b) * dec(x,y) -> fractional digits = b+y

Dieter

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

06-11-2013
09:05 PM

06-11-2013
09:05 PM

Thanks Dieter.