dbTalk Databases Forums  

Re: Max size of btrieve file.

comp.databases.btrieve comp.databases.btrieve


Discuss Re: Max size of btrieve file. in the comp.databases.btrieve forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Bill Bach
 
Posts: n/a

Default Re: Max size of btrieve file. - 10-15-2004 , 10:45 PM






I'd strongly recommend one of two options:
1) Start looking into the ability to purge data
2) Split the file into a few smaller pieces, using a hash function if
necessary

There will likely be plans to increase the size limit in the future, but
having files of this size is just asking for trouble...
Goldstar Software Inc.
Building on Btrieve(R) for the Future(SM)
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive.SQL Service & Support Classes ***
Chicago: November 16-19, 2004: See our web site for details!


"Karl A. Sørensen" wrote:

Quote:
I know the max size of a btrieve file in Pervasive.SQL 8 is 64GB. Does
anybody know if there are any plans to expand this limit ? (at a customer
site we have a file that is now 50GB, and this file will run full in a
couple of months if we don't delete some old data)

Regards
Karl


Reply With Quote
  #2  
Old   
Leonard
 
Posts: n/a

Default Re: Max size of btrieve file. - 10-18-2004 , 10:54 PM






I know of one company that has a "current" file and a "history" file
(or files). When the current file gets full (actually they schedule
this when it is near full at ~60GB) it is rotated to a history file.
For them it works well as their data has to stay on-line for a given
amount of time. When they no longer need their oldest they just stop
reading it.

As to the expanding the records.
1) If you can run it on the server there will not be a round trip
network delay (300 million network delays is quite significant).
2) When populating the new file, drop the indexes in it first,
populate it, then add the indexes back. Dropping the indexes in the
new file is "free" and prevents the database engine from having to
update all the indexes for every record inserted. This can save a
tremendous amount of time. Of course you will have to add the indexes
back after populating the file, but this is typically quite a bit
faster than adding the records into the indexes one by one.

I do not believe that the new database engine will be released before
you predict the file to be full, expecially if you are increasing the
size of the records.

As a thought, could you store the new portions of the records in a
separate file? It would eliminate having to change the record size.

Leonard

On Mon, 18 Oct 2004 08:21:49 GMT, "Karl A. Sørensen"
<kas (AT) kamstrup-ems (DOT) no> wrote:

Quote:
We are going to write a program that purges data from this file. The problem
is that our customer is getting so much data, that even if we purge old
data, this customer will likely hit the file size limitation on 64GB within
a year or two. It is not easy for us to split the file in several files,
because we use a tool (Visual Dataflex), that don't lend itself very easy to
this.

We know we are looking for trouble with this file We are now going to
upgrade this file (expanding a couple of fields), and to be able to do this,
we have to make a service program that copies data from the original file,
to a new file with the new structure. Code that adds 1 millisecond to the
record copy process, makes the whole process take about 80 hours longer to
complete (about 300 million records).

Regards
Karl


"Bill Bach" <bbach (AT) cncdsl (DOT) com> wrote in message
news:4170996F.1E54FB33 (AT) cncdsl (DOT) com...
I'd strongly recommend one of two options:
1) Start looking into the ability to purge data
2) Split the file into a few smaller pieces, using a hash function if
necessary

There will likely be plans to increase the size limit in the future, but
having files of this size is just asking for trouble...
Goldstar Software Inc.
Building on Btrieve(R) for the Future(SM)
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive.SQL Service & Support Classes ***
Chicago: November 16-19, 2004: See our web site for details!


"Karl A. Sørensen" wrote:

I know the max size of a btrieve file in Pervasive.SQL 8 is 64GB. Does
anybody know if there are any plans to expand this limit ? (at a customer
site we have a file that is now 50GB, and this file will run full in a
couple of months if we don't delete some old data)

Regards
Karl




Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.