dbTalk Databases Forums  

[BUGS] float conversion / presentation problem ?

mailing.database.pgsql-bugs mailing.database.pgsql-bugs


Discuss [BUGS] float conversion / presentation problem ? in the mailing.database.pgsql-bugs forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Enrico Weigelt
 
Posts: n/a

Default [BUGS] float conversion / presentation problem ? - 04-06-2005 , 08:33 PM







Hi folks,


i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)

Quote:
fxignal=# SELECT 12.345::real::float;
float8
------------------
12.3450002670288
Maybe its not really an conversion problem, but an bug in sprintf(),
(there were some other float related bugs known in glibc), as these
examples let me suspect:

Quote:
fxignal=# SELECT 12.345::real::float::real;
float4
--------
12.345

fxignal=# SELECT ((12.345::real::float8)-(0.345::float8))::real;
float4
--------
12
I noticed this behaviour on postgresql-7.4.1 and 7.4.3 on glibc-2.3.2


BTW: real is an alias to float4 ?

regards,
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service

phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact (AT) metux (DOT) de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings


Reply With Quote
  #2  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] float conversion / presentation problem ? - 04-06-2005 , 11:26 PM






Enrico Weigelt <weigelt (AT) metux (DOT) de> writes:
Quote:
i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)
You do realize that float4/real is only good to six decimal places
on most platforms?

You can take the issue up with the glibc boys if you like, but
I think you'll get laughed off.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings


Reply With Quote
  #3  
Old   
John R Pierce
 
Posts: n/a

Default Re: [BUGS] float conversion / presentation problem ? - 04-06-2005 , 11:43 PM



Tom Lane wrote:
Quote:
Enrico Weigelt <weigelt (AT) metux (DOT) de> writes:

i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)


You do realize that float4/real is only good to six decimal places
on most platforms?

You can take the issue up with the glibc boys if you like, but
I think you'll get laughed off.
and importantly, are stored as binary fractions. 0.10000 decimal is a
repeating fraction in binary, its like .1100110011.... x 2^-2 or something as a
FLOAT. Float4 has like 22 bits of significants for the mantissa, while Float8
has like 51 bits or something (this is from rusty memory, so I could have
slipped a few bits here).

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo (AT) postgresql (DOT) org so that your
message can get through to the mailing list cleanly


Reply With Quote
  #4  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] float conversion / presentation problem ? - 04-07-2005 , 12:05 AM



John R Pierce <pierce (AT) hogranch (DOT) com> writes:
Quote:
Tom Lane wrote:
You do realize that float4/real is only good to six decimal places
on most platforms?

and importantly, are stored as binary fractions. 0.10000 decimal is a
repeating fraction in binary, its like .1100110011.... x 2^-2 or
something as a FLOAT.
Right, and conversely a fraction that's cut off at a certain number of
base-2 digits often corresponds to a repeating pattern of base-10
digits. I once had a clue about the exact laws for this, but it was
many years ago :-(

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo (AT) postgresql (DOT) org so that your
message can get through to the mailing list cleanly


Reply With Quote
  #5  
Old   
Neil Conway
 
Posts: n/a

Default Re: [BUGS] float conversion / presentation problem ? - 04-07-2005 , 12:19 AM



Enrico Weigelt wrote:
Quote:
BTW: real is an alias to float4 ?
Yes.

-Neil

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo (AT) postgresql (DOT) org)


Reply With Quote
  #6  
Old   
David Fetter
 
Posts: n/a

Default Re: [BUGS] float conversion / presentation problem ? - 04-07-2005 , 11:13 AM



On Thu, Apr 07, 2005 at 01:02:50AM -0400, Tom Lane wrote:
Quote:
John R Pierce <pierce (AT) hogranch (DOT) com> writes:
Tom Lane wrote:
You do realize that float4/real is only good to six decimal
places on most platforms?

and importantly, are stored as binary fractions. 0.10000 decimal
is a repeating fraction in binary, its like .1100110011.... x 2^-2
or something as a FLOAT.

Right, and conversely a fraction that's cut off at a certain number
of base-2 digits often corresponds to a repeating pattern of base-10
digits. I once had a clue about the exact laws for this, but it was
many years ago :-(
As with base-10, any denominator that can't be expressed as a power of
the prime factors in the base results in a repeating number. So
basically any denominator that can't be expressed as 2^n turns into a
repeating binary doohicky.

Cheers,
D
--
David Fetter david (AT) fetter (DOT) org http://fetter.org/
phone: +1 510 893 6100 mobile: +1 415 235 3778

Remember to vote!

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq


Reply With Quote
  #7  
Old   
Enrico Weigelt
 
Posts: n/a

Default Re: [BUGS] float conversion / presentation problem ? - 04-07-2005 , 08:09 PM



* Tom Lane <tgl (AT) sss (DOT) pgh.pa.us> wrote:
Quote:
Enrico Weigelt <weigelt (AT) metux (DOT) de> writes:
i've noticed an problem with conversion from real to float -
this produces ugly values. (in fact it seems to be real->float8)

You do realize that float4/real is only good to six decimal places
on most platforms?
Thats okay for me. I need only four.

Quote:
You can take the issue up with the glibc boys if you like, but
I think you'll get laughed off.
ehm, why ?


cu
--
---------------------------------------------------------------------
Enrico Weigelt == metux IT service

phone: +49 36207 519931 www: http://www.metux.de/
fax: +49 36207 519932 email: contact (AT) metux (DOT) de
cellphone: +49 174 7066481
---------------------------------------------------------------------
-- DSL ab 0 Euro. -- statische IP -- UUCP -- Hosting -- Webshops --
---------------------------------------------------------------------

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo (AT) postgresql (DOT) org)


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.