Hi,

I need to create an application-level lock for my BerkeleyDB
application. I'm accessing my DB environment (several Berkeley
databases) from Java, using the Java wrapper library. I see that it is
possible to create a unique lock ID by calling

Environment.createLockerID

Then I could use 'getLock' and 'putLock' to acquire/release the lock.
However, my application is distributed and I would to publish this
locker ID on all nodes in the cluster before using it. This makes
things a bit complicated. I was wondering whether there is a way to
manually predefine a locker id, at compilation time and then simply use
it.

If not, I was thinking to maintain a separate database for my
"application level locks" where the key would be some logical lock name
and the value, an ID obtained by createLockerID. Would that work? Are
those locker IDs persistent, so that next on the next application
startup I can just read off the locker table?

Thanks a lot in advance,
Borislav

Borislav,

Locker ids are assigned by the Bdb library, you cannot just create one
on your own. If your application is intermixing application locks with
Bdb requests then it is important that these operations use the same
locker id (e.g. the Bdb transaction id) or else threads can deadlock in
a way that cannot be detected by the Bdb library.

It is not clear to me why a distributed application would share a
locker id. If all processes have the same locker id then they do not
block each other. If the processes are on different physical nodes in
a network, then they cannot share a Bdb environment and the sharing of
a locker id makes no sense at all.

A locker id persists until it is freed or the environment is recreated
or recovered.

Michael Ubell
Sleepycat Software.

Hi Michael,

Why can't processes on different physical nodes share the same
environment? An environment is simply a disk location, so if the
database is stored on a network drive, more than one machine will have
access to it.

If locker ids are assigned by bdb, why expose an API to create new
ones? I can live with the risk of getting a deadlock without bdb
detecting it for me, but the question is: can I use the BDB locking
system for my own application level locks. For example, I would like
some node in my cluster to be able to say: nobody else should be
changing that record until I finish some processing, after which I will
release the lock.

Perhaps I don't understand the concepts of locker id and locks. You are
saying that if process share the same locker id, they do not block each
other. Isn't the whole mechanism of locking based on the idea of
"resource aqcuisition/release" where a resource is identified
abstractly by such a locker id? My understanding was that when you
call 'Environment.createLockerId', BDB creates internally some lock
object on the filesystem. Then, you call 'getLock' to acquire that lock
and putLock to release it. And getLock would block if somebody else is
currently owning the lock. Is my understanding correct?

Thanks!
Boris

Hi Borislav,

Please see:

http://www.sleepycat.com/docs/ref/env/remote.html

The basic problem is that if multiple nodes mmap a file from a network
filesystem, the data generally isn't kept consistent between nodes.

You can ask Berkeley DB to allocate a new locker ID for you, but you
can't make up your own locker ID.

Berkeley DB does not include a distributed lock manager -- to implement
what you're talking about, you'd need to have one node configured as
the lock manager (using the Berkeley DB API), and other nodes making
requests to that one using some sort of RPC/RMI.

You're confusing "lock objects" with "locker IDs". Basically, a locker
ID is Berkeley DB's way of identifying the thing *requesting* the lock
(which is usually associated with a single thread of control).

A *lock object* is a key that identifies the thing being locked. The
application can use almost any naming scheme for identifying lock
objects (as long as it doesn't overlap with Berkeley DB's scheme).

When you make a getLock request, you need to supply both the locker ID
and the lock object. If there is already an entry in the lock table
for the specified lock object, the request will block if at least one
of the locks is exclusive and the locker IDs don't match.

Regards,
Michael.

Michael,

Thanks a lot - it's much clearer now. So, to sum:

1) To have a BerkeleyDB reside in a remote filesystem, we need to make
sure it's fully POSIX compliant. Otherwise, we may have data
inconsistencies. As long as our remote filesystem support correct
fsync semantics, we are fine.

2) Basically, if we want distributed locks, we need to roll our own.
One idea that comes to mind is to define a naming schema for BDB lock
objects that does not conflict with BDB's own naming, and work with
those as "distributed locks". However, this assums that the lock table
is immediately fsync-ed, it's never cached so that all nodes in the
cluster have access to the latest "lock information". Is there a way to
make sure the lock table is not cached, but maintained directly on
disk? Actually, I should be reading the source code for that
probably...

Thanks again for all those clarifications!
Boris

Borislav.Iordanov (AT) ubs (DOT) com wrote:
I suppose the O_DIRECT flag may apply here, depending on the operating
system. You should realize that what you want to do here is totally
contrary to the design objectives of most distributed filesystems. That
is, they try to maximize use of caching to improve performance, and they
generally use a separate channel for maintaining their filesystem locks.
Propagating BerkeleyDB locks via a distributed filesystem will be
incredibly slow if it works at all.


--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/

What would be that separate channel for maintaining a filesystem lock?

I'm not insisting on "leveraging" BDB locks. I need application-level
locks (BDB by default locks at the page or record level, but I need
more control) in my distributed application the same way one needs
mutexes in a multithreaded application. That's all. If BDB locks are
not suitable for this, I'll have to figure out something else, I
guess...

Boris

Borislav.Iordanov (AT) ubs (DOT) com wrote:

It depends. BDB locks are not suitable as-is as a distributed lock
manager.
(because they are by definition not distributed: only local to one
machine,
even if the disk storage happens to be on a second machine).
However, if you want to build your own DLM, then you could I suppose
use
BDB locks in your implementation. I guess if it were me, I'd be looking
at my application-level locking semantics to see if there's any
potential
to build something relatively simple that will do the trick for your
app, without the need for a full-blown distributed locking mechanism.