I wanted to benchmark read access to a very large BerkeleyDB file for
an application I'm building (40 million documents of about 500 bytes
each). I figured the first step is to generate a file of appropriate
size, so I wrote a script that loops from 1 to 40m and inserts a record
where key = loop number and value is a string literal of about 500
bytes.

The problem is it's taking forever, which in retrospect I should have
expected. Writes take a long time, especially writes that are not
simple appends, like with a flat file.

I've searched through the BerkeleyDB docs and the web, but haven't
found any discussion of a method to bulk write to a BerkeleyDB file.
Does anyone know of one?

Alternatively, I could set up a huge RAM disk, create/populate the file
there, and then copy it to disk when done, but that requires more work
to set up, plus root access, and it won't scale beyond a few gigs (no
supercomputers in-house).

-ofer