dbTalk Databases Forums  

Select Case confusion

comp.databases.ms-access comp.databases.ms-access


Discuss Select Case confusion in the comp.databases.ms-access forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Steve P.
 
Posts: n/a

Default Select Case confusion - 09-07-2011 , 03:37 PM






Public Function PPF(Cost As Double, Price As Double) As String
Dim x As Long
'Dim x As Double

x = Round((1 - (Cost / Price)) * 100, 0)

Select Case x

Case 45 To 49.99
PPF = "45"

Case 50 To 54.99
PPF = "50"

Case Else
PPF = "DEFAULT"

End Select

End Function


Declaring x as long if I pass Cost/Price values of 4.05/8.10 then PPF
returns "45". If I declare x as Double then PPF returns "50".

In both cases x displays a value of 50 in the locals window.

I don't understand why this happens.

Reply With Quote
  #2  
Old   
Gene Wirchenko
 
Posts: n/a

Default Re: Select Case confusion - 09-07-2011 , 04:43 PM






On Wed, 7 Sep 2011 13:37:17 -0700 (PDT), "Steve P."
<x (AT) monkeybutler (DOT) com> wrote:

Quote:
Public Function PPF(Cost As Double, Price As Double) As String
Dim x As Long
'Dim x As Double

x = Round((1 - (Cost / Price)) * 100, 0)

Select Case x

Case 45 To 49.99
PPF = "45"

Case 50 To 54.99
PPF = "50"

Case Else
PPF = "DEFAULT"
Which includes >49.99 to <50.

Quote:
End Select

End Function


Declaring x as long if I pass Cost/Price values of 4.05/8.10 then PPF
returns "45". If I declare x as Double then PPF returns "50".

In both cases x displays a value of 50 in the locals window.

I don't understand why this happens.
Rounding. The values can not be represented exactly in floating
point. In the case of double, the rounding just happens to go in the
direction you are expecting.

Sincerely,

Gene Wirchenko

Reply With Quote
  #3  
Old   
Steve P.
 
Posts: n/a

Default Re: Select Case confusion - 09-08-2011 , 08:49 AM



On Sep 7, 5:43*pm, Gene Wirchenko <ge... (AT) ocis (DOT) net> wrote:
Quote:
On Wed, 7 Sep 2011 13:37:17 -0700 (PDT), "Steve P."





x... (AT) monkeybutler (DOT) com> wrote:
Public Function PPF(Cost As Double, Price As Double) As String
Dim x As Long
'Dim x As Double

x = Round((1 - (Cost / Price)) * 100, 0)

Select Case x

Case 45 To 49.99
* *PPF = "45"

Case 50 To 54.99
* *PPF = "50"

Case Else
* *PPF = "DEFAULT"

* * *Which includes >49.99 to <50.

End Select

End Function

Declaring x as long if I pass Cost/Price values of 4.05/8.10 then PPF
returns "45". If I declare x as Double then PPF returns "50".

In both cases x displays a value of 50 in the locals window.

I don't understand why this happens.

* * *Rounding. *The values can not be represented exactly in floating
point. *In the case of double, the rounding just happens to go in the
direction you are expecting.

Sincerely,

Gene Wirchenko
Thanks for the reply. I did a little research on floating point and
while I can't say that I fully understand exactly what is happening at
least I can accept that it is possible.

Reply With Quote
  #4  
Old   
Gene Wirchenko
 
Posts: n/a

Default Re: Select Case confusion - 09-08-2011 , 05:26 PM



On Thu, 8 Sep 2011 06:49:46 -0700 (PDT), "Steve P."
<x (AT) monkeybutler (DOT) com> wrote:

[snip]

Quote:
Thanks for the reply. I did a little research on floating point and
while I can't say that I fully understand exactly what is happening at
least I can accept that it is possible.
It has to do with how the number is represented in binary.
One-third can not be represented exactly in decimal. Similarly,
one-tenth and one-hundredth can not be represented exactly in binary.

One way to avoid the problem (or much of it) is to deal with a
decimal type or to represent the dollar values in pennies.

Sincerely,

Gene Wirchenko

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.