I have a question about transactions, checkpoints, and recovery.
Little background: we use bsddb to store genealogical data.
The data usually is added either manually by the user, or by
importing from other data files. In this latter, batch mode,
we may be adding hundrends of thousands of records.
It seems impractical to use a single atomic txn for the whole
import, if for no other reason then simply because we may run
out of locks on a sufficiently large file.

If this is a reasonable thought (please correct me if this is
outright silly for some reason unknown to me!) then what I'm
left with is to use small txn, e.g. one for every db write.
All is fine, except when the import fails for some unforeseen reason.
In that case, we end up with half-the import done and half not done.
If we recover datatabase, only the transactions up to the failure
will be present.

This is not really acceptable, because what the user will want is
to undo the whole import, fix the problem, and then import again.
Half-import is not an option. As using a single atomic txn is
also not an option (or is it?) I thought of making a checkpoint
before starting the import. The hope was that then on failure
we can go back to the checkpoint and start from there.

Here's my question: how do I revert to the last checkpoint?
Is there any way to do that in bdb? If not, what other options
do I have? Would removing the logs and recovering get me back
to the checkpoint?

Any help is greatly appreciated!
Thanks in advance,
Alex

rshura (AT) gmail (DOT) com writes:

Starting with a snapshot of the database and all transaction
logs, you can perform catastrophic recovery to some point
back in time (option -t for db_recover), and then restart
your application on the result.

This will mean several minutes downtime, and you probably want
to do it on a _copy_ of the database (I'm not sure; I'm always
doing it to copies...)

Of course, you'll lose ANY committed made after the specified
recovery point.

If you have a no-downtime requirement, I can't see how anything
but regular transactions could solve your problem.

best regards
Patrick