Hi All,

I feel like a goofball for not figuring this out, but I've been
trying to see what I've been missing for a while. I'm ready to
ask for help!

I have an environment containing just one DB:
] % ls -1sh
] total 573M
] 11M log.0000000598
] 9.3M log.0000000599
] 553M relayQueue-hash

This DB has grown pretty big, and is basically crowding out all
of the system's available RAM. So I'm trying to get a sense of
what I can tune. (I think a lot of it's in overflow pages, yikes.)

In order to do that, I want to call db_stat -m to learn more
about what's going on with my cache. But I can't.

The first usage in: http://www.sleepycat.com/docs/utility/db_stat.html
works well:
] % db_stat -f -d relayQueue-hash
] 61561 Hash magic number.
] 8 Hash version number.
] Flags: little-endian
] 4096 Underlying database page size.
] 0 Specified fill factor.
] 83283 Number of keys in the database.
] 83283 Number of data items in the database.
] 30506 Number of hash buckets.
] 0 Number of bytes free on bucket pages (100% ff).
] 0 Number of overflow pages.
] 0 Number of bytes free in overflow pages (0% ff).
] 0 Number of bucket overflow pages.
] 0 Number of bytes free in bucket overflow pages (0% ff).
] 0 Number of duplicate pages.
] 0 Number of bytes free in duplicate pages (0% ff).
] 0 Number of pages on the free list.

Buuut, the second usage? No dice!
] % db_stat -h . -m
] db_stat: DB_ENV->open: .: No such file or directory

I haven't been able to figure out a way to fool bdb. I've tried
giving it the full path, relative paths, nothing. I must be
missing something, but I haven't been able to find it.

I'm really hoping this isn't relevant, but just in case:
] % db_stat -V
] Sleepycat Software: Berkeley DB 4.2.52: (December 3, 2003)
] % uname -a
] Linux sdb5000 2.4.22-y19 #1 SMP Mon Oct 18 13:47:11 PDT 2004 i686 unknown

Any clue for the poor?

--
__________________________________________________ _____________________
Marco E. Nicosia | http://www.escape.org/~marco/ | marco (AT) escape (DOT) org

Hi,

The directory has log files in it, but I don't see the environment
backing files. You should search for __db.*. For example, __db.001.
Those are the environment files and may be in another directory. If
you can't find them, then maybe you specified DB_PRIVATE for your
environment?

I noticed you are using DB_HASH for a relatively small number of keys.
The most common access method used DB_BTREE.

http://www.sleepycat.com/docs/ref/am_conf/select.html

Ron

Bingo. I had specified DB_PRIVATE. Thank you, I didn't realize.
Annoyingly, the python interface (bsddb3) doesn't have a function
to call memp_stat(). I guess I will remove DB_PRIVATE and hope
nobody ever goofs up.

In article <1141141237.111278.326640 (AT) i39g2000cwa (DOT) googlegroups.com>,
Ron <ron (AT) sleepycat (DOT) com> wrote:
When I read this page the first time, I didn't realize that 80K
keys is a relatively small number. The page doesn't give any guidance?
Since my keys don't have an ordering, I chose hash. I imagine
converting is not trivial.

Expanding a little bit on my original post:
I've been using Berkeley DB for a little while now, but this is the
first time I've needed to pay attention to tuning.

I have one new question stemming from that. I have what
seems like a lot of overflows:
{'bfree': 81519646,
'big_bfree': 58940555,
'bigpages': 102577,
'buckets': 30477,
'dup': 0,
'dup_free': 0,
'ffactor': 0,
'free': 6973,
'magic': 398689,
'ndata': 83283,
'nkeys': 83283,
'overflows': 1429,
'ovfl_free': 4232315,
'pagesize': 4096,
'version': 8},

My data (serialized xml) is of surprisingly varying size, from under
1K to over 1M. If I increase pagesize, will the overflows automagically
be converted back to regular DB pages?


--
__________________________________________________ _____________________
Marco E. Nicosia | http://www.escape.org/~marco/ | marco (AT) escape (DOT) org

To change the pagesize you will need to do a db_dump and db_reload.
More info can be found here:

http://www.sleepycat.com/docs/utility/db_load.html

Ron