dbTalk Databases Forums  

Huge secondary db

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss Huge secondary db in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
foobaraudio@yahoo.com
 
Posts: n/a

Default Huge secondary db - 11-17-2005 , 03:35 AM






Hi
I've written an application that stores lot of events in bdb
Among others, each event has a modification time
and I'm using a secondary db based on that modification time
Over time the secondary db is getting bigger and bigger.
On 100,000 records I get to 120 Mb and potentialy more

If the application restarts and the secondaty db is rebuilt
those 120 Mb will shrink to around 6 Mb

Is there any configuration I missed here?

Thanks ahead
Shahar


Reply With Quote
  #2  
Old   
Michael Cahill
 
Posts: n/a

Default Re: Huge secondary db - 11-17-2005 , 05:16 AM






Hi Shahar,

What type of database is the secondary, and how is it configured? Can
you check whether the secondary contains the number of keys you expect?

Regards,
Michael.


Reply With Quote
  #3  
Old   
foobaraudio@yahoo.com
 
Posts: n/a

Default Re: Huge secondary db - 11-17-2005 , 06:17 AM



The secondary is a BTREE while the primary is HASH
It contains the exact number of keys I expect(and they are accurate as
well)
here is db_stat of two secondary db's

They represent almost the same amount of (primary) keys
the first is 120Mb
and the second is 5.8Mb

First db_stat (120Mb db file)
=========================
53162 Btree magic number.
9 Btree version number.
Flags: duplicates, little-endian
2 Minimum keys per-page.
4096 Underlying database page size.
3 Number of levels in the tree.
78992 Number of unique keys in the tree.
101066 Number of data items in the tree.
9 Number of tree internal pages.
13868 Number of bytes free in tree internal pages (62% ff).
1248 Number of tree leaf pages.
1617884 Number of bytes free in tree leaf pages (68% ff).
9 Number of tree duplicate pages.
36344 Number of bytes free in tree duplicate pages (1% ff).
0 Number of tree overflow pages.
0 Number of bytes free in tree overflow pages (0% ff).
0 Number of pages on the free list.


Second - (5.8 Mb db file)
====================
53162 Btree magic number.
9 Btree version number.
Flags: duplicates, little-endian
2 Minimum keys per-page.
4096 Underlying database page size.
3 Number of levels in the tree.
89630 Number of unique keys in the tree.
118686 Number of data items in the tree.
9 Number of tree internal pages.
10412 Number of bytes free in tree internal pages (72% ff).
1449 Number of tree leaf pages.
1868534 Number of bytes free in tree leaf pages (69% ff).
8 Number of tree duplicate pages.
21900 Number of bytes free in tree duplicate pages (33% ff).
0 Number of tree overflow pages.
0 Number of bytes free in tree overflow pages (0% ff).
0 Number of pages on the free list.



Thanks again
Shahar


Reply With Quote
  #4  
Old   
foobaraudio@yahoo.com
 
Posts: n/a

Default Re: Huge secondary db - 11-17-2005 , 06:18 AM



The secondary is a BTREE while the primary is HASH
It contains the exact number of keys I expect(and they are accurate as
well)
here is db_stat of two secondary db's

They represent almost the same amount of (primary) keys
the first is 120Mb
and the second is 5.8Mb

First db_stat (120Mb db file)
=========================
53162 Btree magic number.
9 Btree version number.
Flags: duplicates, little-endian
2 Minimum keys per-page.
4096 Underlying database page size.
3 Number of levels in the tree.
78992 Number of unique keys in the tree.
101066 Number of data items in the tree.
9 Number of tree internal pages.
13868 Number of bytes free in tree internal pages (62% ff).
1248 Number of tree leaf pages.
1617884 Number of bytes free in tree leaf pages (68% ff).
9 Number of tree duplicate pages.
36344 Number of bytes free in tree duplicate pages (1% ff).
0 Number of tree overflow pages.
0 Number of bytes free in tree overflow pages (0% ff).
0 Number of pages on the free list.


Second - (5.8 Mb db file)
====================
53162 Btree magic number.
9 Btree version number.
Flags: duplicates, little-endian
2 Minimum keys per-page.
4096 Underlying database page size.
3 Number of levels in the tree.
89630 Number of unique keys in the tree.
118686 Number of data items in the tree.
9 Number of tree internal pages.
10412 Number of bytes free in tree internal pages (72% ff).
1449 Number of tree leaf pages.
1868534 Number of bytes free in tree leaf pages (69% ff).
8 Number of tree duplicate pages.
21900 Number of bytes free in tree duplicate pages (33% ff).
0 Number of tree overflow pages.
0 Number of bytes free in tree overflow pages (0% ff).
0 Number of pages on the free list.



Thanks again
Shahar


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.