Re: Significant digits - any ideas ?

#11

I was unclear as to how the trailing zeros could be "significant," too.
'Course, I'm unclear how the calc would be used at all, so I'm just workin'
blind, here. ;-)

--
Cain
MBATM.06-26.MTM273B-LAW529
shadenfroh (AT) email (DOT) uophx.edu
"Bridget Eley" <bridgeteley (AT) ihug (DOT) com.au> wrote

Quote:

Hi Shadenfroh

I understood Peter to mean that trailing zeroes were significant for the
purpose of identifying the digits on which to perform the round function.
I
do not understand why he would need to leave the trailing zeroes in the
result, since 5.00023 = 5.0002300.

Where my calc does fall down is on whole numbers (returned ?), and
negative
numbers (rounding up instead of down and down instead of up), both easy
enough to fix.

My revised calc:

If(Int(Number) = Number, Number,Left(Number, Position(Number, ".", 1, 1) -
1) & Abs(Round(TextToNum(If(Number < 0, "-.",".") & Right(Number,
Length(Number) - Position(Number, ".", 1, 1))),SignificantDigits))
/ TextToNum(Left("10000000000000000000", Length(Right(Number, 1 +
Length(Number) - Position(Number, ".", 1, 1))) -
Length(NumToText(TextToNum(Right(Number, Length(Number) - Position(Number,
".", 1, 1))))))))

Bridget Eley

in article hetZa.5162$M6.376337 (AT) newsread1 (DOT) prod.itd.earthlink.net,
Shadenfroh
at shadenfroh (AT) yahoo (DOT) com wrote on 11/8/03 1:09 AM:

F'rinstance, 5.00023000999 calculates to 5.00023 instead of 5.0002300

From Peter's post, "As to the seeming contradiction: I initially
indicated
that *leading* zeroes (0 before the first non-zero digit) were
insignificant, but *trailing* zeroes (zero after the 3rd non-zero digit)
were not."

#12

Okay, in the process of fixing the "rounding negative numbers" problem (see
revised calc below) I discovered that it makes no difference: Filemaker
rounds numbers up if 5 or greater regardless or whether they are positive or
negative. I (being a humanities graduate) wondered if this was correct,
mathematically speaking, so I checked the forum at mathquest.com (first
google hit) and found this:

Quote:

"Rounding" means finding the nearest, most reasonable approximation.
As 7.6 (not your example) is "towards" 8, while 7.4 is "towards" 7,
so is -7.6 approximated by -8, and -7.4 by -7, (-7.4) being closer to
-7 than to -8. The "middle ground" of 7.5 is arbitrarily rounded up
stilll, I think, except over large sets of values where it may be
alternated up and down to average out differences (?)?

Which makes sense, in spite of the insubstantiality of the author
(nobody (AT) nowehere (DOT) ..). So I learned something new, the problem was a red
herring and the calc below while technically not incorrect is apparently
quite unnecessary. The one in my previous post will suffice (that one that
fixed the whole number problem).

If(Int(Number) = Number, Number,Left(Number, Position(Number, ".", 1, 1) -
1) &

Abs(If(Number < 0, Round("-." & TextToNum(Right(Number, Length(Number) -
Position(Number, ".", 1, 1))),SignificantDigits), Round("." &
TextToNum(Right(Number, Length(Number) - Position(Number, ".", 1,
1))),SignificantDigits)))

/ TextToNum(Left("10000000000000000000", Length(Right(Number, 1 +
Length(Number) - Position(Number, ".", 1, 1))) -
Length(NumToText(TextToNum(Right(Number, Length(Number) - Position(Number,
".", 1, 1))))))))

Bridget Eley

in article BB5CFE8E.261D%bridgeteley (AT) ihug (DOT) com.au, Bridget Eley at
bridgeteley (AT) ihug (DOT) com.au wrote on 11/8/03 8:04 AM:

Quote:

Where my calc does fall down is on whole numbers (returned ?), and negative
numbers (rounding up instead of down and down instead of up), both easy
enough to fix.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Re: Significant digits - any ideas ?

comp.databases.filemaker comp.databases.filemaker

Re: Significant digits - any ideas ? - 08-10-2003 , 05:37 PM

Re: Significant digits - any ideas ? - 08-10-2003 , 06:06 PM