dbTalk Databases Forums  

Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column

Go Back dbTalk Databases Forums Databases comp.databases.oracle.misc Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column

comp.databases.oracle.misc comp.databases.oracle.misc


Discuss Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column in the comp.databases.oracle.misc forum.



Reply
Page 5 of 5 < 1 2 3 4 5
 
Thread Tools Display Modes
  #41  
Old   
Fleetie
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column - 05-22-2008 , 03:35 AM





"Shakespeare" <whatsin (AT) xs4all (DOT) nl> wrote
Quote:
Or it reserves one digit for the sign..

Shakespeare
I don't think so, because a NUMBER(1) column can take negative values,
i.e. values from -9 to +9.

Here are some more findings I've just made, in the C domain:

---

Additionally, In C domain, you start losing the 6th place right of the point
after the number to the left of the D.P. exceeds about 2^33=8589934592:

TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455

So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
you can't represent 9999999999.123456 without losing precision at the right.

I guess that's why Pro*C plays safe and rounds to 15 places.

---


Martin




Reply With Quote
  #42  
Old   
Shakespeare
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column - 05-22-2008 , 04:00 AM






"Fleetie" <fleetie (AT) nospam (DOT) com> schreef in bericht
news:g13b8g$ul9$1 (AT) aioe (DOT) org...
Quote:
"Shakespeare" <whatsin (AT) xs4all (DOT) nl> wrote
Or it reserves one digit for the sign..

Shakespeare

I don't think so, because a NUMBER(1) column can take negative values,
i.e. values from -9 to +9.

Here are some more findings I've just made, in the C domain:

---

Additionally, In C domain, you start losing the 6th place right of the
point
after the number to the left of the D.P. exceeds about 2^33=8589934592:

TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455

So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
you can't represent 9999999999.123456 without losing precision at the
right.

I guess that's why Pro*C plays safe and rounds to 15 places.

---


Martin
Ok thank you. Now I see what you mean with 'safe decimal places'. Looks like
a bit falling off somewhere ;-)

Shakespeare




Reply With Quote
  #43  
Old   
Shakespeare
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column - 05-22-2008 , 04:00 AM



"Fleetie" <fleetie (AT) nospam (DOT) com> schreef in bericht
news:g13b8g$ul9$1 (AT) aioe (DOT) org...
Quote:
"Shakespeare" <whatsin (AT) xs4all (DOT) nl> wrote
Or it reserves one digit for the sign..

Shakespeare

I don't think so, because a NUMBER(1) column can take negative values,
i.e. values from -9 to +9.

Here are some more findings I've just made, in the C domain:

---

Additionally, In C domain, you start losing the 6th place right of the
point
after the number to the left of the D.P. exceeds about 2^33=8589934592:

TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455

So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
you can't represent 9999999999.123456 without losing precision at the
right.

I guess that's why Pro*C plays safe and rounds to 15 places.

---


Martin
Ok thank you. Now I see what you mean with 'safe decimal places'. Looks like
a bit falling off somewhere ;-)

Shakespeare




Reply With Quote
  #44  
Old   
Shakespeare
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column - 05-22-2008 , 04:00 AM



"Fleetie" <fleetie (AT) nospam (DOT) com> schreef in bericht
news:g13b8g$ul9$1 (AT) aioe (DOT) org...
Quote:
"Shakespeare" <whatsin (AT) xs4all (DOT) nl> wrote
Or it reserves one digit for the sign..

Shakespeare

I don't think so, because a NUMBER(1) column can take negative values,
i.e. values from -9 to +9.

Here are some more findings I've just made, in the C domain:

---

Additionally, In C domain, you start losing the 6th place right of the
point
after the number to the left of the D.P. exceeds about 2^33=8589934592:

TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455

So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
you can't represent 9999999999.123456 without losing precision at the
right.

I guess that's why Pro*C plays safe and rounds to 15 places.

---


Martin
Ok thank you. Now I see what you mean with 'safe decimal places'. Looks like
a bit falling off somewhere ;-)

Shakespeare




Reply With Quote
  #45  
Old   
Shakespeare
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Var into NUMBER(16,6) Column - 05-22-2008 , 04:00 AM



"Fleetie" <fleetie (AT) nospam (DOT) com> schreef in bericht
news:g13b8g$ul9$1 (AT) aioe (DOT) org...
Quote:
"Shakespeare" <whatsin (AT) xs4all (DOT) nl> wrote
Or it reserves one digit for the sign..

Shakespeare

I don't think so, because a NUMBER(1) column can take negative values,
i.e. values from -9 to +9.

Here are some more findings I've just made, in the C domain:

---

Additionally, In C domain, you start losing the 6th place right of the
point
after the number to the left of the D.P. exceeds about 2^33=8589934592:

TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455

So with a C double, you can't represent a FULL 16 digits in decimal, i.e.
you can't represent 9999999999.123456 without losing precision at the
right.

I guess that's why Pro*C plays safe and rounds to 15 places.

---


Martin
Ok thank you. Now I see what you mean with 'safe decimal places'. Looks like
a bit falling off somewhere ;-)

Shakespeare




Reply With Quote
  #46  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Varinto NUMBER(16,6) Column - 05-22-2008 , 11:00 AM


On May 22, 10:35 am, "Fleetie" <flee... (AT) nospam (DOT) com> wrote:

Quote:
TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455
I don't see a loss, unless you made some copy/paste error:
I see .123455 - that is 6 digits after the decimal point...


Reply With Quote
  #47  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Varinto NUMBER(16,6) Column - 05-22-2008 , 11:00 AM


On May 22, 10:35 am, "Fleetie" <flee... (AT) nospam (DOT) com> wrote:

Quote:
TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455
I don't see a loss, unless you made some copy/paste error:
I see .123455 - that is 6 digits after the decimal point...


Reply With Quote
  #48  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Varinto NUMBER(16,6) Column - 05-22-2008 , 11:00 AM


On May 22, 10:35 am, "Fleetie" <flee... (AT) nospam (DOT) com> wrote:

Quote:
TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455
I don't see a loss, unless you made some copy/paste error:
I see .123455 - that is 6 digits after the decimal point...


Reply With Quote
  #49  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: Pro*C: Loss of Decimal Places INSERTing From "double" Host Varinto NUMBER(16,6) Column - 05-22-2008 , 11:00 AM


On May 22, 10:35 am, "Fleetie" <flee... (AT) nospam (DOT) com> wrote:

Quote:
TVTime=(double)8589934590.123456; -> TVTime=8589934590.123456
TVTime=(double)8589934591.123456; -> TVTime=8589934591.123456
TVTime=(double)8589934592.123456; -> TVTime=8589934592.123455 <-- Loss
starts now.
TVTime=(double)8589934593.123456; -> TVTime=8589934593.123455
TVTime=(double)8589934594.123456; -> TVTime=8589934594.123455
I don't see a loss, unless you made some copy/paste error:
I see .123455 - that is 6 digits after the decimal point...


Reply With Quote
Reply
Page 5 of 5 < 1 2 3 4 5

« Previous Thread | Next Thread »



Thread Tools
Display Modes
Linear Mode Linear Mode

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.


Contact Us - dbTalk Databases Forums - Archive - Top