simple numeric calculation doent work -
12-03-2009
, 06:55 PM
why in gods name doesnt this simple sql calculation not work in a view
98/912500 = 0.0001073972603
if I do a simple view with the following calcualted field:
SELECT 98 / 912500 AS aaa
FROM dbo.tbl_CA_Cases
I get 0??????
If I explicitly convert to decimal
SELECT CONVERT(decimal(38, 16), 98) AS v, CONVERT(decimal(38, 16),
912500) AS vt, CONVERT(decimal(38, 16), 98) / CONVERT(decimal(38, 16),
912500)
AS aaa, CONVERT(decimal(38, 16), CONVERT(decimal
(38, 16), 98) / CONVERT(decimal(38, 16), 912500)) AS bbb
FROM dbo.tbl_CA_Cases
I get 0.000107
Why is it only going to 6 decimal places????
If I do
SELECT 118993 * 98 / 912500 * 0.8468 AS aaa
FROM dbo.tbl_CA_Cases
I get 10.1616 where did SQL get this value from??? the correct
calculation is 10.82169939200
If I do
SELECT CONVERT(decimal(38, 16), 118993) * 98 / 912500 * 0.8468 AS
aaa
FROM dbo.tbl_CA_Cases
I get 10.821699392000 whcih is correct
What exactly is SQL calculating?????????????????
I'm assuming there is some logical explanation.....
Cheers
Grant |