dbTalk Databases Forums  

Q: multi process writes, multi thread reads

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss Q: multi process writes, multi thread reads in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Malcolm Dew-Jones
 
Posts: n/a

Default Q: multi process writes, multi thread reads - 12-09-2005 , 01:25 AM






This is in a sendmail milter environment, I have FC4, which appears to
mean "Sleepycat Software: Berkeley DB 4.3.27". The person for whom I am
setting this up uses AIX, and I'm not sure their versions, and I don't
know if that matters much or not.

The idea is for the milter to receive information from another program
(the updater) by having the updater program add entries to a db file, and
then having the milter read the db file. There might be multiple copies
of the updater running at the same time.

My understanding of the berkeley db software is that several processes can
be "put"ing key/value pairs into a db file, and another process can be
reading values from the db file, and that I do not have to do anything in
particular for that to work, other than waiting until I "know" a certain
key has been added before I try to look it up. But perhaps my
understanding is wrong.

The problem for the person for whom I am setting this up is that
apparently the value is not being reliably found by the milter, I don't
know if the value is not being reliably set by the updater program, or
whether the milter is not reliably accessing the added value. Running the
updater program by itself works fine, the data is always correctly added.

Anyway, the point of this message is to know if there is some aspect of
using the db software that I misunderstand that is needed to make this
work.


The milter fork/execs to run the updater to update the .db file.

fork/exec("updater-program","the-key");

The milter waits (i.e. wait(&status)) and then does a lookup of "the-key"
in the db file.

The milter opens the db file for read access just once when it starts.
It uses the DB_THREAD|DB_RDONLY flags. Each thread of the milter uses
that same handle to access the open db file when the thread tries to read
the value that its updater should have "put" into the db file.

When launched, the updater program opens the db file for write access
using just the DB_THREAD flag. It does a lookup of the key to check the
key is not already in the db file, and if not there then it does a
dbp->put to insert the key with a value. Then the db file is closed and
the program exits.

When run from the milter the updater program could conceivably be running
multiple instances of itself at the same time (as different processes).
In theory, two or more instances of the updater might all put a value for
the same key at approximately the same time.

But from a logical point of view that doesn't matter. The only issue is
that each milter thread should see a value for a key after its instance of
the updater program has exited.

I am not doing anything to coordinate any of the reads and writes except
the step where each thread in the milter waits for the exit of the updater
that it launched.


So, the question is, is there some fundamental flaw, missing step, or
incorrect understanding, in using this the db software in this manner?


Feedback welcome.

Malcolm



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.