 | |
DB::put function is getting hanged
comp.databases.berkeley-db comp.databases.berkeley-db
 |
| | Thread Tools | Display Modes |
#1
 
| |||
| |||
|
DB::put function is getting hanged -
06-10-2007
, 10:06 PM
#2
| |||
| |||
|
|
Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? |
#3
| |||
| |||
|
|
Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? |
and the put are ordered in some way unknown to Berkeley DB. If you've
got a single process that tries to do these things, they are trivially
ordered, so it's not allowed. Another source of orderings is IPC.
#4
| |||
| |||
|
|
Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? It's allowed, but only if the threads or processes executing the open and the put are ordered in some way unknown to Berkeley DB. If you've got a single process that tries to do these things, they are trivially ordered, so it's not allowed. Another source of orderings is IPC. |
The number of database files in not defined in my work.
So i think I need to use a single transaction in my process. Will it
be OK to open more than 1 database files in a single transaction?
#5
| |||
| |||
|
|
On Jun 12, 4:33 am, Florian Weimer <f... (AT) deneb (DOT) enyo.de> wrote: Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? It's allowed, but only if the threads or processes executing the open and the put are ordered in some way unknown to Berkeley DB. If you've got a single process that tries to do these things, they are trivially ordered, so it's not allowed. Another source of orderings is IPC. Thanks. My concern is in a single process. The number of database files in not defined in my work. So i think I need to use a single transaction in my process. Will it be OK to open more than 1 database files in a single transaction? |
in
its own transaction (use autocommit flag so you don't even have to
manage
the transaction). The only reasons I know to keep a transaction going
past the end of the open would be 1) you are creating your set of
databases atomically and you want the behavior where either all your
databases were created or none. or 2) performance - you are opening
1000 databases, I'd guess you'd see an open using a single transaction
to be faster then separate transactions. Same advice goes for your
original question if I understand it. Open the database in its own
separate transaction (use autocommit), and then do puts in their own
transaction. That's the typical approach anyway.
#6
| |||
| |||
|
|
On Jun 11, 9:20 pm, rohithr... (AT) gmail (DOT) com wrote: On Jun 12, 4:33 am, Florian Weimer <f... (AT) deneb (DOT) enyo.de> wrote: Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? It's allowed, but only if the threads or processes executing the open and the put are ordered in some way unknown to Berkeley DB. If you've got a single process that tries to do these things, they are trivially ordered, so it's not allowed. Another source of orderings is IPC. Thanks. My concern is in a single process. The number of database files in not defined in my work. So i think I need to use a single transaction in my process. Will it be OK to open more than 1 database files in a single transaction? It's okay, but probably not your best bet. Why not open each database in its own transaction (use autocommit flag so you don't even have to manage the transaction). The only reasons I know to keep a transaction going past the end of the open would be 1) you are creating your set of databases atomically and you want the behavior where either all your databases were created or none. or 2) performance - you are opening 1000 databases, I'd guess you'd see an open using a single transaction to be faster then separate transactions. Same advice goes for your original question if I understand it. Open the database in its own separate transaction (use autocommit), and then do puts in their own transaction. That's the typical approach anyway. |
auto-commit may be needed. So the implicit transaction which we get by
auto-commit will work, am I right?
I have one more doubt. Hope if the database count is less, then there
is no problem(even for performance) in creating multiple transaction
under the same environment, am i right?
#7
| |||
| |||
|
|
On Jun 18, 8:58 pm, Don Anderson <don.ander... (AT) gmail (DOT) com> wrote: On Jun 11, 9:20 pm, rohithr... (AT) gmail (DOT) com wrote: On Jun 12, 4:33 am, Florian Weimer <f... (AT) deneb (DOT) enyo.de> wrote: Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? It's allowed, but only if the threads or processes executing the open and the put are ordered in some way unknown to Berkeley DB. If you've got a single process that tries to do these things, they are trivially ordered, so it's not allowed. Another source of orderings is IPC. Thanks. My concern is in a single process. The number of database files in not defined in my work. So i think I need to use a single transaction in my process. Will it be OK to open more than 1 database files in a single transaction? It's okay, but probably not your best bet. Why not open each database in its own transaction (use autocommit flag so you don't even have to manage the transaction). The only reasons I know to keep a transaction going past the end of the open would be 1) you are creating your set of databases atomically and you want the behavior where either all your databases were created or none. or 2) performance - you are opening 1000 databases, I'd guess you'd see an open using a single transaction to be faster then separate transactions. Same advice goes for your original question if I understand it. Open the database in its own separate transaction (use autocommit), and then do puts in their own transaction. That's the typical approach anyway. So you are saying that explicit transaction is not necessary, simply auto-commit may be needed. So the implicit transaction which we get by auto-commit will work, am I right? |
begin transaction
ret = db.operation.....
if (ret is error)
abort transaction
else
commit transaction
It's a convenience, you can do it either way.
The important point in avoiding the hang is that you finish
(commit) the transaction that you used in the open,
and start a new transaction for any put operation.
Always doing an autocommit on an open is an easy way to
remember to avoid that mistake.
A transaction holds locks on various pages in the database
during its lifetime. Each operation for a transaction may acquire
new locks (for example, a Db::get will acquire locks on the page
containing the data). The only way for the transaction to give up
locks is to commit or abort the transaction. This is way we recommend
to keep
transactions short, and minimize the number of actions within a
given transaction.
The common idiom is to open a database autocommit and keep
the handle open, often for a long time, like the lifetime of the
application.
Since the transaction is done, no locks are held.
Then when you are doing a put, you can either do autocommit if it's
one operation, or if you have multiple puts/gets that need to be
isolated
or atomic, you group them all in a transaction. There is still the
potential for deadlock (depending on what you are doing with various
threads), but you are much better off than keeping the open within
your transaction. (....and there are ways around deadlocks.)
Quote:
|
I have one more doubt. Hope if the database count is less, then there is no problem(even for performance) in creating multiple transaction under the same environment, am i right? |
the same environment. If you keep your transactions short, you have
better chance at parallelism, which may increase your overall
performance.
For example, thread 1 with transaction T1 may be writing key "ABC" in
the database -- the thread may be blocked waiting on the read of a
disk block containing
the data to be updated. But meanwhile, thread 2 with transaction T2
can proceed
reading or writing key "GHI", which might already be in the cache or
it can
make its own disk request. The fewer locks you hold (i.e. smaller
your transaction),
the more chance of this kind of parallelism.
- Don
#8
| |||
| |||
|
|
On Jun 21, 5:13 am, rohithr... (AT) gmail (DOT) com wrote: On Jun 18, 8:58 pm, Don Anderson <don.ander... (AT) gmail (DOT) com> wrote: On Jun 11, 9:20 pm, rohithr... (AT) gmail (DOT) com wrote: On Jun 12, 4:33 am, Florian Weimer <f... (AT) deneb (DOT) enyo.de> wrote: Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? It's allowed, but only if the threads or processes executing the open and the put are ordered in some way unknown to Berkeley DB. If you've got a single process that tries to do these things, they are trivially ordered, so it's not allowed. Another source of orderings is IPC. Thanks. My concern is in a single process. The number of database files in not defined in my work. So i think I need to use a single transaction in my process. Will it be OK to open more than 1 database files in a single transaction? It's okay, but probably not your best bet. Why not open each database in its own transaction (use autocommit flag so you don't even have to manage the transaction). The only reasons I know to keep a transaction going past the end of the open would be 1) you are creating your set of databases atomically and you want the behavior where either all your databases were created or none. or 2) performance - you are opening 1000 databases, I'd guess you'd see an open using a single transaction to be faster then separate transactions. Same advice goes for your original question if I understand it. Open the database in its own separate transaction (use autocommit), and then do puts in their own transaction. That's the typical approach anyway. So you are saying that explicit transaction is not necessary, simply auto-commit may be needed. So the implicit transaction which we get by auto-commit will work, am I right? Using the autocommit flag for an operation is roughly equivalent to: begin transaction ret = db.operation..... if (ret is error) abort transaction else commit transaction It's a convenience, you can do it either way. The important point in avoiding the hang is that you finish (commit) the transaction that you used in the open, and start a new transaction for any put operation. Always doing an autocommit on an open is an easy way to remember to avoid that mistake. A transaction holds locks on various pages in the database during its lifetime. Each operation for a transaction may acquire new locks (for example, a Db::get will acquire locks on the page containing the data). The only way for the transaction to give up locks is to commit or abort the transaction. This is way we recommend to keep transactions short, and minimize the number of actions within a given transaction. The common idiom is to open a database autocommit and keep the handle open, often for a long time, like the lifetime of the application. Since the transaction is done, no locks are held. Then when you are doing a put, you can either do autocommit if it's one operation, or if you have multiple puts/gets that need to be isolated or atomic, you group them all in a transaction. There is still the potential for deadlock (depending on what you are doing with various threads), but you are much better off than keeping the open within your transaction. (....and there are ways around deadlocks.) I have one more doubt. Hope if the database count is less, then there is no problem(even for performance) in creating multiple transaction under the same environment, am i right? If I understand the question, you can have multiple transactions under the same environment. If you keep your transactions short, you have better chance at parallelism, which may increase your overall performance. For example, thread 1 with transaction T1 may be writing key "ABC" in the database -- the thread may be blocked waiting on the read of a disk block containing the data to be updated. But meanwhile, thread 2 with transaction T2 can proceed reading or writing key "GHI", which might already be in the cache or it can make its own disk request. The fewer locks you hold (i.e. smaller your transaction), the more chance of this kind of parallelism. - Don |
#9
| |||
| |||
|
|
On Jun 21, 5:13 am, rohithr... (AT) gmail (DOT) com wrote: On Jun 18, 8:58 pm, Don Anderson <don.ander... (AT) gmail (DOT) com> wrote: On Jun 11, 9:20 pm, rohithr... (AT) gmail (DOT) com wrote: On Jun 12, 4:33 am, Florian Weimer <f... (AT) deneb (DOT) enyo.de> wrote: Am a fresher in BerkeleyDB. If i open a database file(using transaction) for put(adding to DB) and if i tried to open the same file in different transaction the put is hanging. It seems a deadlock is occuring. Is it not allowed, the same database file to be opened in more than one transactions? It's allowed, but only if the threads or processes executing the open and the put are ordered in some way unknown to Berkeley DB. If you've got a single process that tries to do these things, they are trivially ordered, so it's not allowed. Another source of orderings is IPC. Thanks. My concern is in a single process. The number of database files in not defined in my work. So i think I need to use a single transaction in my process. Will it be OK to open more than 1 database files in a single transaction? It's okay, but probably not your best bet. Why not open each database in its own transaction (use autocommit flag so you don't even have to manage the transaction). The only reasons I know to keep a transaction going past the end of the open would be 1) you are creating your set of databases atomically and you want the behavior where either all your databases were created or none. or 2) performance - you are opening 1000 databases, I'd guess you'd see an open using a single transaction to be faster then separate transactions. Same advice goes for your original question if I understand it. Open the database in its own separate transaction (use autocommit), and then do puts in their own transaction. That's the typical approach anyway. So you are saying that explicit transaction is not necessary, simply auto-commit may be needed. So the implicit transaction which we get by auto-commit will work, am I right? Using the autocommit flag for an operation is roughly equivalent to: begin transaction ret = db.operation..... if (ret is error) abort transaction else commit transaction It's a convenience, you can do it either way. The important point in avoiding the hang is that you finish (commit) the transaction that you used in the open, and start a new transaction for any put operation. Always doing an autocommit on an open is an easy way to remember to avoid that mistake. A transaction holds locks on various pages in the database during its lifetime. Each operation for a transaction may acquire new locks (for example, a Db::get will acquire locks on the page containing the data). The only way for the transaction to give up locks is to commit or abort the transaction. This is way we recommend to keep transactions short, and minimize the number of actions within a given transaction. The common idiom is to open a database autocommit and keep the handle open, often for a long time, like the lifetime of the application. Since the transaction is done, no locks are held. Then when you are doing a put, you can either do autocommit if it's one operation, or if you have multiple puts/gets that need to be isolated or atomic, you group them all in a transaction. There is still the potential for deadlock (depending on what you are doing with various threads), but you are much better off than keeping the open within your transaction. (....and there are ways around deadlocks.) I have one more doubt. Hope if the database count is less, then there is no problem(even for performance) in creating multiple transaction under the same environment, am i right? If I understand the question, you can have multiple transactions under the same environment. If you keep your transactions short, you have better chance at parallelism, which may increase your overall performance. For example, thread 1 with transaction T1 may be writing key "ABC" in the database -- the thread may be blocked waiting on the read of a disk block containing the data to be updated. But meanwhile, thread 2 with transaction T2 can proceed reading or writing key "GHI", which might already be in the cache or it can make its own disk request. The fewer locks you hold (i.e. smaller your transaction), the more chance of this kind of parallelism. - Don |
I don't get the last few lines. Is the transactions T1 and T2 operate
on the same file?
|
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
| |
Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.