dbTalk Databases Forums  

beginner questions on locking BTrees in Ruby

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss beginner questions on locking BTrees in Ruby in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Gary Lawrence Murphy
 
Posts: n/a

Default beginner questions on locking BTrees in Ruby - 10-11-2005 , 11:40 AM







Are there any good rank-beginner tutorials anywhere on using the BDB
locking system? I've read through the sleepycat docs, but I got very
lost -- although I have used bdb before, I've only used it as a
persistent hash-table; this new project has been a lesson in humility

I'm also using the ruby-bdb from ftp://moulon.inra.fr/pub/ruby/; the
code is the same in /essence/ as the C/C++ docs, but the ruby
abstraction probably adds to my confusion. Is this the best Ruby
binding for a BDB beginner?

What I need is one webapp that reads ordered ranges of BTree entries,
and another that posts new entries at the end of the ordered list.
The app receives new documents and POSTs a (timestamp,filename) for
each, the other app receives GET queries like "all filenames between
T1 and T2" where T2 is most often, but now always, the current time.
A third async crontab runs to periodically purge entries where there
is no corresponding physical file (they have expired and been removed
by another system)

the ruby-bdb docs seemed to suggest I open a BTree with something like

db = BDB::Btree.open $FILEDB, nil,
BDB::CREATE|BDB::NOMMAP, 0644,
"set_pagesize" => 1024

and right now the above line is almost pure voodoo to me I have
no reason to choose that pagesize or to select NOMMAP, I'm just copying
the examples until I get a better grasp of the bdb system.

Not surprising, since there isn't any kind of locking, my db quickly
became corrupt, but how do I add multiprocess locking to the btree.
Is it sufficient to add BDB::INIT_LOCK to the options bitmap?

Is this even the right way to do this? My naive understanding of the
sleepycat docs is that multiprocess locking should require creating
additional files in a db directory, but this BTree.open method only
allows for a single filename parameter. It also does not let me
distinguish the read-only open from the write-only open.

Should I be using some other more generic method and setting my
db-type options explicitly?

any tips to best starting point docs online or elsewhere are greatly
appreciated. I'm 99% certain bdb is the way to go with this app,
but I'm just having a mental block about this locking stuff.

--
Gary Lawrence Murphy <garym (AT) teledyn (DOT) com> ==============================
www.teledyn.com - blog.teledyn.com - irish.teledyn.com - sbp.teledyn.com
====================== The present moment is a powerful goddess (Goethe)

Reply With Quote
  #2  
Old   
Gary Lawrence Murphy
 
Posts: n/a

Default Re: beginner questions on locking BTrees in Ruby - 10-12-2005 , 10:19 AM







I /think/ this is what I'm looking for:

in the write-processes I am using

def open_db
@env ||= BDB::Env.open( @@FILEDB,
BDB::CREATE|BDB::INIT_MPOOL|BDB::INIT_CDB, 0664,
"set_pagesize" => 1024 )

@db ||= @env.open_db BDB::BTREE, 'files', nil,
BDB::CREATE|BDB::NOMMAP, 0664

end

in the read processes, and use the same but add BDB::RDONLY on the
database.

Is this /all/ I need to do to get basic multiprocess lock protection?

Or do I need to explicitly call the env.lock method before I write?

Would I gain more reliability by wrapping all my writes in a transaction?

What about the cursor range reads, should these be inside a transaction?
What happens if cursor.next leads to a record that has since been deleted?

I also don't quite understand why INIT_MPOOL is required when I am
using NOMMAP; it threw an execution exception without it.

--
Gary Lawrence Murphy <garym (AT) teledyn (DOT) com> ==============================
www.teledyn.com - blog.teledyn.com - irish.teledyn.com - sbp.teledyn.com
====================== The present moment is a powerful goddess (Goethe)

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.