![]() | |
#1
| |||
| |||
|
#2
| |||
| |||
|
|
Public Function PPF(Cost As Double, Price As Double) As String Dim x As Long 'Dim x As Double x = Round((1 - (Cost / Price)) * 100, 0) Select Case x Case 45 To 49.99 PPF = "45" Case 50 To 54.99 PPF = "50" Case Else PPF = "DEFAULT" |
|
End Select End Function Declaring x as long if I pass Cost/Price values of 4.05/8.10 then PPF returns "45". If I declare x as Double then PPF returns "50". In both cases x displays a value of 50 in the locals window. I don't understand why this happens. |
#3
| |||
| |||
|
|
On Wed, 7 Sep 2011 13:37:17 -0700 (PDT), "Steve P." x... (AT) monkeybutler (DOT) com> wrote: Public Function PPF(Cost As Double, Price As Double) As String Dim x As Long 'Dim x As Double x = Round((1 - (Cost / Price)) * 100, 0) Select Case x Case 45 To 49.99 * *PPF = "45" Case 50 To 54.99 * *PPF = "50" Case Else * *PPF = "DEFAULT" * * *Which includes >49.99 to <50. End Select End Function Declaring x as long if I pass Cost/Price values of 4.05/8.10 then PPF returns "45". If I declare x as Double then PPF returns "50". In both cases x displays a value of 50 in the locals window. I don't understand why this happens. * * *Rounding. *The values can not be represented exactly in floating point. *In the case of double, the rounding just happens to go in the direction you are expecting. Sincerely, Gene Wirchenko |
#4
| |||
| |||
|
|
Thanks for the reply. I did a little research on floating point and while I can't say that I fully understand exactly what is happening at least I can accept that it is possible. |
![]() |
| Thread Tools | |
| Display Modes | |
| |