By way of introduction, here at The AGE, we have a home-brewed
advertising system, written in Perl, running under Linux, that uses
BerkeleyDB (4.1.24) via Paul Marquess's excellent BerkeleyDB.pm. The
system maintains a no. of Btree databases and uses the Locking
Subsystem for concurrency control.

Just y'day, we experienced an interesting problem whereby certain
programs were failing when attempting to open databases for update. At
the same time, programs that were already running were happily
updating the database seemingly oblivious to the problem. When I
attempted to run a "db_stat -d" on one of the databases, the program
failed with an "ENOMEM" error, and the following message: "The maximum
number of locks has been reached.". I take it we exhausted a resource
in the locking subsystem?

Unfortunately, because this was a production system and I needed to
fix the problem in a hurry, I didn't save the output from the command
nor did I have a chance to further analyse the database stats. To fix
the problem, all I did was stop the system, delete the "__db.001" file
in the database home directory, and restart the system. Deleting the
"__db.001" file had the effect of forcing the database environment to
be re-created thus causing the Locking Subsystem to be re-initialized.

The Locking Subsystem is running with the default configuration,
except for the "Lock timeout value", and had been "up" for just over 3
months. My questions are:

1. Should I change the database configuration and increase the size of
the lock memory region from the default 360KB?

2. Would I be best advised to "delete" the database environment
periodically, say once a week?

3. Come to think of it, the deadlock detector (db_deadlock) and RPC
database server (berkeley_db_svc) had also been running for a no. of
months before I killed the environment. I suppose I should be
restarting these processes once a week in case of memory leaks etc.?

Anyway, I'd be interested in other peoples' experiences.

TIA

Peter

In article <a2d32c48.0308191800.71f68157 (AT) posting (DOT) google.com>,
pgiorgilli (AT) theage (DOT) fairfax.com.au (Peter Giorgilli) writes:
I wonder if changing the values of set_lk_max_lockers, set_lk_max_locks,
and set_lk_max_objects would be helpful in this case? I searched
BerkeleyDB.pm and BerkeleyDB.xs and did not find any way to set these
from perl. Maybe Paul can comment on whether or not there is a way, or
even a need to do this?

I think that the default values for these are like 1000 or so. Do
you think that you had that many locks at the time? Actually, I'm not
100% sure how the values for set_lk_max_xxx map to actual locks?


That is a question that I would like to know the answer to as well. Is
it adequate simply to close down the environment and start a new one to
release any 'leaked' resources, or are more drastic measures needed. Is
it neccessary for every process that owns an environment to close it down,
or can they be closed and reopened on a process by process basis so that
the clients don't know the difference?

Also, I would think that it would be possible to detect a memory leak by
using something like MRTG to plot a particular process' core image size in
time. Ideally, it should reach some nominal value and fluctuate thereabouts.
If it is not asymptotic, but rather continues to grow over time ad infinitum,
then that would indicate a memory leak? I have not tried this, but it might
be nice to know.

Restarting the server code periodically might be a nice safeguard against this
if it could be done without interruption to the clients.

/Nathan.

You can modify the default settings of most things by
putting a file called DB_CONFIG in your environment
directory. see:
http://www.sleepycat.com/docs/ref/env/db_config.html

Michael Ubell
Sleepycat Software.