 | |
Re: Log buffer and Cache tuning for perfromance
comp.databases.berkeley-db comp.databases.berkeley-db
 |
| | Thread Tools | Display Modes |
#1
 
| |||
| |||
|
Re: Log buffer and Cache tuning for perfromance -
10-15-2006
, 07:00 PM
|
My understaning was - add operations perfromance shoule improve due to log buffer usage. |
only make a difference if the log buffer is overflowing, causing
unnecessary I/O.
Quote:
|
But did not happen |
transaction, but if they were separate operations, each one would log
less than 1KB on average, and flush the buffer at commit time, so the
default 64KB setting would be more than adequate.
Quote:
|
- cache increase must help read operations. But read performance degraded and add operation performance improved. |
cache size won't help (the working set is basically the current page
that data is read from).
That said, I am surprised that performance decreased substantially when
cache size was increased. Are those numbers reproducible, and if so,
would you be able to send us a copy of your test code?
Regards,
Michael.
#2
| |||
| |||
|
|
Hi Kiran, My understaning was - add operations perfromance shoule improve due to log buffer usage. That is not correct in general. Increasing the log buffer size will only make a difference if the log buffer is overflowing, causing unnecessary I/O. But did not happen You didn't specify whether all of the updates were occuring in a single transaction, but if they were separate operations, each one would log less than 1KB on average, and flush the buffer at commit time, so the default 64KB setting would be more than adequate. - cache increase must help read operations. But read performance degraded and add operation performance improved. Reading in sequential order is a special case where increasing the cache size won't help (the working set is basically the current page that data is read from). That said, I am surprised that performance decreased substantially when cache size was increased. Are those numbers reproducible, and if so, would you be able to send us a copy of your test code? Regards, Michael. |
#3
| |||
| |||
|
|
My configuration parameters are as follows: env flags = DB_RECOVER |DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN|DB_THREAD |
files (__db.*) will all be removed and recreated. On most systems, the
blocks for those files will be only be allocated as the pages of the
files are actually written to, which will take place during the normal
get and put operations. A larger cache will result in more disk
operations for region file allocation during the timed period of the
test. Similarly, a larger cache may require more page table updates by
the kernel VM subsystem. If you set the DB_REGION_INIT flag on the
environment (using DBENV->set_flags()) before calling DBENV->open(),
then the region files will be completely touched and paged in as part
of the DBENV->open() call, so you can measure them as one-time-only
costs instead of as part of the normal operations.
....
Quote:
|
My understaning was - add operations perfromance shoule improve due to log buffer usage. But did not happen |
before you consider tuning the logging subsystem.
Quote:
|
- cache increase must help read operations. But read performance degraded and add operation performance improved. |
performance will go up because fewer page of the .db files will need to
be flushed to disk. Again, the db_stat utility, this time with the -m
option, should be used to analyze the behavior of the subsystem when
you want to tune performance.
Philip Guenther
#4
| |||
| |||
|
|
On Oct 13, 8:19 am, "Kiran Kumar M R" <kira... (AT) gmail (DOT) com> wrote: My configuration parameters are as follows: env flags = DB_RECOVER |DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN|DB_THREAD One side-effect of the DB_RECOVER flag is that the environment region files (__db.*) will all be removed and recreated. On most systems, the blocks for those files will be only be allocated as the pages of the files are actually written to, which will take place during the normal get and put operations. A larger cache will result in more disk operations for region file allocation during the timed period of the test. Similarly, a larger cache may require more page table updates by the kernel VM subsystem. If you set the DB_REGION_INIT flag on the environment (using DBENV->set_flags()) before calling DBENV->open(), then the region files will be completely touched and paged in as part of the DBENV->open() call, so you can measure them as one-time-only costs instead of as part of the normal operations. ... My understaning was - add operations perfromance shoule improve due to log buffer usage. But did not happen You should use db_stat with the -l option to obtain the logging stats before you consider tuning the logging subsystem. - cache increase must help read operations. But read performance degraded and add operation performance improved. See my comment above for why read performance could go down. Write performance will go up because fewer page of the .db files will need to be flushed to disk. Again, the db_stat utility, this time with the -m option, should be used to analyze the behavior of the subsystem when you want to tune performance. Philip Guenther |
|
« Previous Thread
|
Next Thread »
| Thread Tools | |
| Display Modes | |
Linear Mode
| |
Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.