Slow performance while scanning large database files

#11

* Patrick Schaaf:

Quote:

Oh, updates may happen in varying order on the seperate instances.

You need some commutative system.

Right. XOR is commutative.

Yes, of course it will do. But you have a^a = 0 for any a (this is
what I meant when I wrote that any element has order two). It means
that you can't detect some errors in the presence of duplicate rows.
If you use another commutative operation, say + (addition with
carries), you don't run into this problem.

Quote:

This leads to an important question: How do you check that the
databases are consistent? You'll need a consistent cut to answer this
question.

You can make a copy of the database and log files at any time,
and reconstruct any previous version at will. Lots of consistent
cuts to look at. This should be doable out-of-line on a backend
system.

Ah, I assumed that you wanted to perform on-line consistency checks.
Please ignore most of my comments. 8-P (My point about the inferiority
of XOR is still valid, though.)

#12

Florian Weimer <fw (AT) deneb (DOT) enyo.de> writes:

Quote:

Right. XOR is commutative.

Yes, of course it will do. But you have a^a = 0 for any a (this is
what I meant when I wrote that any element has order two). It means
that you can't detect some errors in the presence of duplicate rows.

I'm not concerned with fully duplicate rows, and, each 'a' being an md5sum,
the probability of different rows nulling each other is rather low. So that
won't hurt me...

Quote:

If you use another commutative operation, say + (addition with
carries), you don't run into this problem.

.... but now I understand what you were getting at. Thank you!

best regards
Patrick

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Slow performance while scanning large database files

comp.databases.berkeley-db comp.databases.berkeley-db

Re: database checksum - 09-14-2004 , 06:08 PM

Re: database checksum - 09-15-2004 , 01:31 AM