Question about significant figures -
07-16-2004
, 04:44 PM
I'm working on a query that has to provide values with a specified number of
significant figures. What I've done so far is to use:
select
test_number,
value,
sig_figs_number,
decode
(
sig_figs_number,
-1,
value,
0,
value,
round(value, sig_figs_number - trunc(ln(value)/ln(10)) - 1)
) formatted
from
result
Output of this looks like:
TEST_NUMBE VALUE SIG_FIGS_NUMBER FORMATTED
---------- ---------- --------------- ----------
130 3.14159 2 3.1
130 3.14159 2 3.1
131 3 3 3
131 3 3 3
132 3.14159 -1 3.14159
132 3.14159 -1 3.14159
133 3.14159 1 3
133 3.14159 1 3
134 3.14159 2 3.1
134 3.14159 2 3.1
135 314.59 1 300
135 314.59 1 300
136 314.59 0 314.59
136 314.59 0 314.59
The problem is the value of 3 -- it should be 3.00 and I don't know a good
way to do that.
The "-1" in the sig_figs_number column is a default and we don't want any
formatting or rounding of the result, same for zero, hence the decode.
Thanks!
Brad
--
************************************************** *********************
Brad Millard On-line ballistics for small arms...
jbm (AT) eskimo (DOT) com www.eskimo.com/~jbm
************************************************** ********************* |