Hello Everyone,
I would like to compare a couple of free (as in Richard Stallman)
database engines for a study, evaluating their suitability for one of my
university projects.
The subjects are Firebird, SQLite, MySQL and PostgreSQL. All installed
on the same hardware and OS, but having only one of them active at a time.
One of the hypothetical scenarios is a Parliamentary Election's Day,
when millions of votes (think of the total population of a country) are
submitted during a fairly limited amount of time (e.g. 12 hours). Each
vote corresponds to a single INSERT statement in its own transaction,
many of which happens simultaneously.

My plan is to run some stress-tests, simulating such a scenario with a
random or changing load of concurrent transactions. And I am hoping to
measure the DB performance, hopefully feed the results to something like
gnuplot to have charts for illustration.
I have difficulties figuring out how to do this. Running such test on
a single database engine already seems quite a challenge, let alone
doing it on four different engines.

I am looking for suggestions or pointers on ways to perform concurrent
transactions on the above mentioned engines, and ways to measure DB
performance during the tests. If you know of a tool that can be used, or
a method to simulate such load, or sample source code that does such
thing, or an article discussing this topic, please let me know!

Regards,
--
Keve Nagy * Debrecen * Hungary
keve(at)mail(dot)poliod(dot)hu

On 2011-03-20, Keve Nagy <keve (AT) see (DOT) my.sig> wrote:
like a raincoat made of sack-cloth electronic voting is fundamentally
flawed.

multitasking would seem to be is the obvious way.
you probably want to do load generation on a different server.

number of active transactions

average time to complete transaction.

number of inserts per second.

you'll probably see an explosion of the first two when you reach some
critical level of the third and the server runs out of resources.

tweaking database configuration parameters may help.

--
⚂⚃ 100% natural

Maybe so. The concept itself however, is apparently believed to be a
great school example.

All noted.
And very much appreciated!

Regards,
--
Keve Nagy * Debrecen * Hungary
keve(at)mail(dot)poliod(dot)hu

On Wed, 23 Mar 2011 01:06:15 +0100, Keve Nagy <keve (AT) see (DOT) my.sig> wrote:

Why? One reason might be is that it appears to be quite doable,
but in actuality, it is very prone to problems. Read the RISKS-LIST
(USENET group comp.risks) for plenty of examples of them. The list is
archived at
http://catless.ncl.ac.uk/Risks/

[snip]

Sincerely,

Gene Wirchenko

On 20/03/2011 6:13 AM, Keve Nagy wrote:
Hard to imagine a single system (not to mention the communication
hardware) for that. 1) Do the math, say 50 million votes over 12 hours,
that would be over one thousand 'insert transactions' per second on
average. The peaks would likely be several or many times that. If
there are more elected positions per voter than just a single
parliamentary seat, multiply some more. 2) Look at all the plausible
requirements, not just the population. Eg., factors like how to size
recovery logs or not wanting to have to arrange a second election if the
single system fails, not to mention the arguments amongst the second set
of losers. Some things aren't naturally centralized, in most western
countries votes are counted on a local basis, eg. per riding. That
might result in an average load of only 100 transactions per second and
several hundreds peak. But in most (so-called) democracies the
candidates will have their own scrutineers and they'll want either paper
ballots or paper output from the voting systems. It's hard to imagine a
computer system that can do a judicial re-count.


I'm not up on current hardware but I'd say that besides doing the paper
analysis of the stuff above you could reasonably limit your stress-tests
to a much smaller scale.

Thanks for pointing these out, Paul!
Even though I was aware of these numbers, I didn't really think through
their total effect the way you just highlighted. I have already amended
the test plan accordingly, considering only a scaled-down version of the
task. Having the election broken down to regions is a perfect example.

In the meantime I have also made some progress in the implememtation of
the load test. I concluded that Perl DBI can be my best friend here,
allowing me to construct a tool that spawns and inserts data-records as
required. And it can be run on virtually any O/S, and against any of my
target database types.

Where I still don't really have an answer is the measurement part.
I don't see how to measure each database's performance of handling the
INSERTs. Can't quite grab what to measure, how I could measure that, and
how to feed it into something like gnuplot to get comparable graphs.

Thoughts and pointers on such technical details would be more than welcome!


Regards,
--
Keve Nagy * Debrecen * Hungary
keve(at)mail(dot)poliod(dot)hu

On Apr 27, 2:42*am, Keve Nagy <k... (AT) see (DOT) my.sig> wrote:
....
I don't know why you acknowledge Jasen B's suggestions (from more than
a month ago) and then ignore them.

Sounds like the assignment is late. From what was written, technical
advice seems beside the point. Ie., you have been given a measurement
problem but after all this time it seems you haven 't decided what to
measure. That would make tool technique moot. It would have been
better to try to run your tests without any measurements at all - that
experience might have prompted you to refine them or try Jasen's
metrics without further ado or even consider a more regimented field.