Has anyone found any issues pertaining to cygwin and cache size
allocation?

I built a native cygwin build (typical cd build_unix;
.../dist/configure; make) and am noticing that
after I set a cachesize over 1 GB and try to open the environment via
DB_ENV->open(), it's
returning error and db_strerror() is textually reporting that
DB_ENV->open() failed and I should run database recovery. This occurs
even without a database or environment being present (DB_CREATE
assumed). If I try to pass 1 to the gbyte arg or 1024 * 1024 * 1024 to
the nbytes arg of DB_ENV->set_cachesize() it will always fail upon the
initial open. However, if I reduce the cache size to 990 MB, things
will proceed as normal.

This is a modern system, dual-core Opteron 180, Windows Server 2003, 2
GB of RAM, latest 4.4.16 release of db, and aside from debugging the
actual open call itself or blaming Windows somehow I can't really think
of a reason (other than some kind of addressing limitation) that would
cause this.

There are no reported problems with using a 1GB cache on a cygwin
build.

Are you using DB_SYSTEM_MEM? If you, so may want to review the
information in "Windows Notes" section of the Reference Guide,
available in your download package or online at:
http://www.sleepycat.com/docs/ref/build_win/notes.html.

What does the code segment that performs the environment open look
like?

Regards,

Dave

I definitely am not using DB_SYSTEM_MEM. In fact, I tried to to see if
that would change anything and it just resulted in a hard error upon
trying to open the environment - i.e. it's not supported.

Here's my functions and backing headers that I use:
(db_coll is an abstract I use to pre-fill db attributes before I
create/open)

/* util_db.h: */

#ifndef __UTIL_DB_H
#define __UTIL_DB_H

#include <db.h>
#include <stdio.h>

#define db_perror(x,y,z) { fprintf(stderr, "%s: %s\n", y,
db_strerror(x)); return z; }

enum db_type {
DB_PRIMARY = 0x1 << 0,
DB_SECONDARY = 0x1 << 1,
DB_PRIMARY_DUP = 0x1 << 2
};

struct db_coll {
char dbe_n[256]; /* environment directory */
char db_n[256]; /* db name */
DB_ENV *dbe;
DB *db;
uint32_t flags; /* db_type flags */
struct db_coll *db_c_p; /* parent db_coll */
int32_t (*cb_dup_compare)(DB *, const DBT *, const DBT *);
int32_t (*cb_assoc)(DB *, const DBT *, const DBT *, DBT *);
};

int32_t db_open(struct db_coll *);
int32_t db_close(DB *);

#endif /* __UTIL_DB_H */


/* util_db.c: */
#include <stdlib.h>
#include <unistd.h>
#include <db.h>
#include "util_db.h"

static const size_t cache_max = 990 * 1024 * 1024;
static const size_t pagesize = 4 * 1024;
static const uint32_t fl_db = DB_CREATE;
static const uint32_t fl_dbe = DB_CREATE | DB_INIT_CDB | DB_INIT_MPOOL;

int32_t db_open(struct db_coll *db_c)
{
uint32_t fl_db;
uint32_t fl_dbe;
int32_t ret;

if ((db_c->flags & DB_PRIMARY) && db_c->dbe == NULL) {
if ((ret = db_env_create(&db_c->dbe, 0)) != 0)
db_perror(ret, "db_env_create", -1);

db_c->dbe->set_cachesize(db_c->dbe, 0, cache_max, 1);

if ((ret = db_c->dbe->open(db_c->dbe,
db_c->dbe_n,
fl_dbe, 0)) != 0)
{
db_perror(ret, "db_env_open", -1);
}
}

if (db_c->db == NULL) {
if ((ret = db_create(&db_c->db,
db_c->flags & DB_PRIMARY
? db_c->dbe
: db_c->db_c_p->dbe,
0)) != 0)
{
db_perror(ret, "db_create", -1);
}

db_c->db->set_pagesize(db_c->db, pagesize);

if (db_c->flags & (DB_SECONDARY | DB_PRIMARY_DUP)) {
db_c->db->set_flags(db_c->db, DB_DUP |
DB_DUPSORT);

if (db_c->cb_dup_compare != NULL)
db_c->db->set_dup_compare(db_c->db,
db_c->cb_dup_compare);
}

if ((ret = db_c->db->open(db_c->db,
NULL,
db_c->db_n,
NULL,
DB_BTREE,
fl_db,
0)) != 0)
{
db_perror(ret, "db_open", -1);
}

if (db_c->flags & DB_SECONDARY)
if ((ret =
db_c->db->associate(db_c->db_c_p->db,
NULL,
db_c->db,
db_c->cb_assoc,
DB_IMMUTABLE_KEY)) != 0)
{
db_perror(ret, "db_associate", -1);
}
}

return 0;
}

int32_t db_close(DB *db)
{
if (db == NULL) return -1;

db->close(db, 0);

return 0;
}

p.s. google group's formatting logic sucks!

I suspect that you are seeing a limitation of Cygwin's mmap
implementation (which is in turn limited by MapViewOfFileEx). A couple
of things to try:

* split the cache into multiple regions (by passing an integer greater
than 1 as the last parameter to set_cachesize).

* take Cygwin out of the equation by trying the same test with Berkeley
DB compiled for native Windows.

* try Windows x64, if possible (I don't know whether there is a 64-bit
aware version of Cygwin, though).

* open the environment with the DB_PRIVATE flag so that data is in the
heap rather than mapped into memory from a file. This prevents
multi-process access, but may shed some light on what is causing the
limitation.

Please let us know if you find out any more.

Michael.

1. I tried a setup like 3 * 512M, unfortunately the result was the
same.

2. Would compilation within cygwin of my application code referencing
native bdb libs be an issue?

3. No access to it yet, although I doubt cygwin takes much advantage of
it at this point.

4. Unfortunately I had already tried this as well :-/. I'll try
DB_PRIVATE with multi-region caches sometime this weekend, however.