dbTalk Databases Forums  

dramatic increase of log size for queue DB when activating extents

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss dramatic increase of log size for queue DB when activating extents in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
pweemeeuw@softhome.net
 
Posts: n/a

Default dramatic increase of log size for queue DB when activating extents - 02-08-2006 , 09:13 AM






Hi,

I use the 4.3.28 version of Berkeley DB (Debian platform) in
combination with the Perl BerkeleyDB 0.27 module.

I noticed that when using a queue database created with the parameters
as shown below, my test application takes about 30 seconds, while when
enabling extents (the line commented out) increases the time to about
80 seconds. I could narrow the most likely cause down to the size of
the logs: instead of 10 Mb, I get now 120 Mb of logs (and as these are
written synchronously, this might explain the time increase).

When I compare the two versions of the logs, the most eye catching
difference is that each __qam_delext line is about 12000 (!)
characters long. Is this a bug, or a known issue, or something that I
misunderstood?

Here is the perl code (the actual record size that i use is 12000, and
the page_size 16k):

sub new {
my ($class, %args) = @_;

my $env = new BerkeleyDB::Env
-Flags => DB_CREATE | DB_INIT_MPOOL
Quote:
DB_INIT_LOG | DB_INIT_LOCK
DB_INIT_TXN,
-ErrFile => *STDERR;

$env->set_flags(DB_TXN_WRITE_NOSYNC, 1);

my $rec_size = $args{recsize} || 1000;
my $page_size = $args{pagesize} || 4096;

my $txn = $env->txn_begin();
my $db = new BerkeleyDB::Queue
-Filename => 'queue.db',
-Flags => DB_CREATE,
-Len => $rec_size,
-Pagesize => $page_size,
-Env => $env,
# -ExtentSize => 2**14,
-Txn => $txn
or die "ERROR: $! $BerkeleyDB::Error\n";
$txn->txn_commit() == 0
or die "? $! $BerkeleyDB::Error\n";

my $self = { env => $env,
db => $db
};
return bless $self, $class;
}

Thank you in advance for your time. And thanks for the great Berkeley
DB code!

Patrick Weemeeuw.



Reply With Quote
  #2  
Old   
pweemeeuw@softhome.net
 
Posts: n/a

Default Re: dramatic increase of log size for queue DB when activating extents - 02-10-2006 , 09:02 AM






Answering my own question...

I think I figured out what is happening. When extents are enabled,
each deleted entry has to be recorded in the log, as the extents are
reused by the operating system. Without extents, the entry has only to
be marked as deleted; since the contents is still there, it has not
to be recorded in the log.

This seems to be consistent with the huge difference in the log file
size.

-- Patrick


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.