Processing Cluster

#11

Bram Wiekens wrote:

Quote:

Ok, I've checked it all again, so I'm a little more
aware of the situation now. Until now I had little
insight in the working of the algorithm. So I'm
slightly less confused now.. ;-)

The timing constraints are indeed important, as suggested.
The algorithm is a heuristic selfcalibrating algorithm for
a processing environment and tries to search the most ideal
situation (eliminating noise etc). What should be done is:
Send data to all nodes in the cluster, and the nodes get
a specific amount of time to compute this data. The only
difference between the nodes is the parameterset they get.
Then some node will decide (compute) which node has the best
result. This will be the calibrating parameters.

This is why I propose the three structure, it avoids
great message distribution in an all to all structure
(stronger knowledge of who is all isn't always available)

In general the systems do their work almost solitair.
And dataset/message exchange should be minimized, because
it gives overhead.

So again do you think think the tree structure would suffice,
or would another structure do a better job?
A Centralized database would probably congest
(alltough it's a pretty fast) network connection between the machines,
and a lot of versioning has to be done, to compare all different results.

Bram

This is obviously a heavy research project. While I don't have details of
papers or books you should examine the ideas from the SETI project.

Also the general requirements sound analogous to another system I encountered.
I interviewed at CAS (Chemical Abstracts Services) many years ago. The system
they had was a tree structure, though it was inverted from your design. All
the data was stored at the leaf nodes. The peak(root) of the tree received
client requests, farmed it out to the leaf machines and then gathered the results.

SO, I gather that the basic design might be headed in the right direction, but
you need a lot more research than what you'll get here in a newsgroup. Time to
search the ACM publications and such.

Good luck!
Ed

--
Ed Prochak
running http://www.faqs.org/faqs/running-faq/
netiquette http://www.psg.com/emily.html
--
"Two roads diverged in a wood and I
I took the one less travelled by
and that has made all the difference."
robert frost

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Processing Cluster

comp.databases comp.databases

Re: Processing Cluster - 06-19-2004 , 01:28 AM