dbTalk Databases Forums  

Re: AUTOINCREMENT fields and rolling over behavior--

comp.databases.btrieve comp.databases.btrieve


Discuss Re: AUTOINCREMENT fields and rolling over behavior-- in the comp.databases.btrieve forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Bill Bach
 
Posts: n/a

Default Re: AUTOINCREMENT fields and rolling over behavior-- - 08-27-2003 , 09:51 AM






"Will DeWitt Jr." wrote:

Quote:
I have a database with a 2 byte (in Pascal lingo, a 'word', in C/C++
lingo, a 'short') AUTOINCREMENT field. I've observed that once the
counter reaches 32,768, it begins giving Status code 5 errors on every
attempt to INSERT beyond that point. (Which makes sense and is
expected, of course.) My question is, I also delete records earlier
in the database that are no longer needed, and was wondering if
there's a way of resetting the counter used for the AUTOINCREMENT
field so I can begin INSERT'ing as usual?
There is no "counter" per se -- the database just checks for the current
largest value, then adds one to it.

Quote:
FYI-- I'm using the MKDE directly via BTRCALL API calls, I've tried it
with v6.15 as well as the newer Pervasive SQL V8 SP1, and have not
found a solution to this.
Correct -- due to backward compatibility constraints, all versions work
the same way.

Quote:
Two other related questions-- is there a
way to get the MKDE to treat the 2 byte field AUTOINCREMENT as
unsigned, allowing for a maximum counter of 65535?
The definition of the AUTOINC data type is that it is stored and collated
as an absolute value. This allows a developer to use the negative value
as a flag value. Unfortunately, I don't know what was going through the
original developers' minds when they decided this. It is a very cool
option, but needless to say, this does cause some havoc from time to time,
especially with the SQL engine when it tries to create temporary sort
files and the numbers suddenly sort differently. (I think all these
issues may have been corrected, but there may be another one yet.)

Quote:
And is there a
way, if I end up ditching AUTOINCREMENT for this field, to quickly
find gaps in manually generated fields? (Well, this would actually
apply to AUTOINCREMENT as well, but if there's no way to reset the
counter then the AUTOINCREMENT field won't work for what I'm doing.)
The only way to change this behavior is to do it in code -- search for an
available value, then force the field to that value, which will insert
correctly. You can make this a bit quicker by using the following loop:
for(i=0;i<32768;i++)
if(Btrv(Operation 5 + 50,i)==4)
break;
Obviously pseudocode, but you get the idea. The GetKey bias (+50) does
not retrieve the entire record, so it will run faster than a real GetEqual
operation. For even faster operation, store your last position in a file
somewhere, then loop to 32768 first, then back to the initial position in
a second loop.

Quote:
(I've done extensive Google searches on this subject, both in these
NG's and the web in general-- I haven't come across any viable
solutions beyond expanding the field size (which is at best a
temporary solution).)
Temporary? Even a 4-byte AutoInc field gives you 2 billion records. If
you insert 1 record every second, it'll still take 63 years to fill up the
data type. Of course, with the latest engines, you can use the 8-byte
AutoInc. I won't do the math, but I'll bet that the data file will be
purged (and possibly rebuilt) long before that happens.

Another solution is to write a rebuild application -- one that reads
records out of the old file and inserts them into a new file. Of course,
if these are key values, then you'd also need to start renumbering foreign
keys in other tables from the old value to the new value as you go along.
It can be difficult and time-consuming, but not impossible. (I once wrote
a purge/renumber function that hit 50+ files, which was loads of fun.)
Goldstar Software Inc.
Building on Btrieve(R) for the Future(SM)
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive.SQL Service & Support Classes ***
Chicago: September 16-18: See our web site for details!



Reply With Quote
  #2  
Old   
Will DeWitt Jr.
 
Posts: n/a

Default Re: AUTOINCREMENT fields and rolling over behavior-- - 08-27-2003 , 05:41 PM






Bill Bach <bbach (AT) cncdsl (DOT) com> wrote


Quote:
There is no "counter" per se -- the database just checks for the current
largest value, then adds one to it.
Ahh.. yeah, I realized this behavior when I started removing entries
from the end of the database (at the largest value).

Quote:
The only way to change this behavior is to do it in code -- search for an
available value, then force the field to that value, which will insert
correctly. You can make this a bit quicker by using the following loop:
for(i=0;i<32768;i++)
if(Btrv(Operation 5 + 50,i)==4)
break;
Obviously pseudocode, but you get the idea. The GetKey bias (+50) does
not retrieve the entire record, so it will run faster than a real GetEqual
operation. For even faster operation, store your last position in a file
somewhere, then loop to 32768 first, then back to the initial position in
a second loop.
Yeah, I was beginning to think that was the best solution. I'd
forgotten about the GetKey bias-- as you say, that should speed it up
significantly I'd think. =)

Quote:
Temporary? Even a 4-byte AutoInc field gives you 2 billion records. If
you insert 1 record every second, it'll still take 63 years to fill up the
data type. Of course, with the latest engines, you can use the 8-byte
AutoInc. I won't do the math, but I'll bet that the data file will be
purged (and possibly rebuilt) long before that happens.
Well, I'm potentially inserting dozens of records a second, but it
would certainly take *much* longer for the issue to crop up (where
with the current 2 byte AutoInc field it takes 6-8 hours to hit the
issue). As for the 8-byte AutoInc, unfortunately the engines I would
need to conform to are versions 5.1x (DOS environment) and 6.15 (Win32
environment).

Quote:
Another solution is to write a rebuild application -- one that reads
records out of the old file and inserts them into a new file. Of course,
if these are key values, then you'd also need to start renumbering foreign
keys in other tables from the old value to the new value as you go along.
It can be difficult and time-consuming, but not impossible. (I once wrote
a purge/renumber function that hit 50+ files, which was loads of fun.)
Yeah, the AUTOINCREMENT field is referenced in other files and
renumbering them would mean having to update the other files as well
(which, depending on how fast/slow it was, would possibly be too much
of a performance hit to be feasible). I'm going to be doing some more
testing to see if I can get decent performance out of the idea you
mentioned ('get equal' with the 'key only' bias), perhaps coupled with
an expansion of the field to 4 bytes (which should alleviate the
problem for a much longer period, but potentially make the 'get equal'
with 'key only' bias solution slower once the limit is finally
reached).

Thank you again for your comments, they've been most helpful. =)


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.