![]() | |
![]() |
| | Thread Tools | Display Modes |
#1
| |||
| |||
|
|
fxignal=# SELECT 12.345::real::float; float8 ------------------ 12.3450002670288 |
|
fxignal=# SELECT 12.345::real::float::real; float4 -------- 12.345 fxignal=# SELECT ((12.345::real::float8)-(0.345::float8))::real; float4 -------- 12 |
#2
| |||
| |||
|
|
i've noticed an problem with conversion from real to float - this produces ugly values. (in fact it seems to be real->float8) |
#3
| |||
| |||
|
|
Enrico Weigelt <weigelt (AT) metux (DOT) de> writes: i've noticed an problem with conversion from real to float - this produces ugly values. (in fact it seems to be real->float8) You do realize that float4/real is only good to six decimal places on most platforms? You can take the issue up with the glibc boys if you like, but I think you'll get laughed off. |
#4
| |||
| |||
|
|
Tom Lane wrote: You do realize that float4/real is only good to six decimal places on most platforms? and importantly, are stored as binary fractions. 0.10000 decimal is a repeating fraction in binary, its like .1100110011.... x 2^-2 or something as a FLOAT. |
#5
| |||
| |||
|
|
BTW: real is an alias to float4 ? |
#6
| |||
| |||
|
|
John R Pierce <pierce (AT) hogranch (DOT) com> writes: Tom Lane wrote: You do realize that float4/real is only good to six decimal places on most platforms? and importantly, are stored as binary fractions. 0.10000 decimal is a repeating fraction in binary, its like .1100110011.... x 2^-2 or something as a FLOAT. Right, and conversely a fraction that's cut off at a certain number of base-2 digits often corresponds to a repeating pattern of base-10 digits. I once had a clue about the exact laws for this, but it was many years ago :-( |
#7
| |||
| |||
|
|
Enrico Weigelt <weigelt (AT) metux (DOT) de> writes: i've noticed an problem with conversion from real to float - this produces ugly values. (in fact it seems to be real->float8) You do realize that float4/real is only good to six decimal places on most platforms? |
|
You can take the issue up with the glibc boys if you like, but I think you'll get laughed off. ehm, why ? |
![]() |
| Thread Tools | |
| Display Modes | |
| |