dbTalk Databases Forums  

replication transport requirements

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss replication transport requirements in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Patrick Schaaf
 
Posts: n/a

Default replication transport requirements - 09-03-2003 , 06:24 AM






Hello all,

I have two fundamental questions regarding the requirements for
a Berkeley DB replication transport protocol:

- is out-of-order reception of messages permitted?
- is loss of messages permitted?

To be concrete: would it be sufficient to use plain UDP, maybe augmented
with a good message authenticator, without regard to retransmission and
sequencing?

best regards
Patrick

Reply With Quote
  #2  
Old   
Keith Bostic
 
Posts: n/a

Default Re: replication transport requirements - 09-03-2003 , 03:05 PM






mailer-daemon (AT) bof (DOT) de (Patrick Schaaf) wrote in message news:<3f55cf78$0$31866$9b622d9e (AT) news (DOT) freenet.de>...
Quote:
Hello all,

I have two fundamental questions regarding the requirements for
a Berkeley DB replication transport protocol:

- is out-of-order reception of messages permitted?
- is loss of messages permitted?
Yes, and yes.

Quote:
To be concrete: would it be sufficient to use plain UDP, maybe augmented
with a good message authenticator, without regard to retransmission and
sequencing?
Berkeley DB replication was carefully designed to be transport mechanism
agnostic, that is, we wanted to be able to run using TCP/IP, UDP, backplane
message passing protocol, or carrier pigeon.

Obviously, this has serious implications for the application's transactional
guarantees: if the message didn't arrive at the replica, should the master
return to the application that the transaction has committed? This means
applications requiring serious transactional guarantees will have to do more
work when using transport protocols that are lossy, for example.

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic bostic (AT) sleepycat (DOT) com
Sleepycat Software Inc. keithbosticim (ymsgid)
118 Tower Rd. +1-781-259-3139
Lincoln, MA 01773 http://www.sleepycat.com


Reply With Quote
  #3  
Old   
Patrick Schaaf
 
Posts: n/a

Default Re: replication transport requirements - 09-04-2003 , 12:51 AM



bostic (AT) sleepycat (DOT) com (Keith Bostic) writes:

Quote:
I have two fundamental questions regarding the requirements for
a Berkeley DB replication transport protocol:

- is out-of-order reception of messages permitted?
- is loss of messages permitted?

Yes, and yes.

To be concrete: would it be sufficient to use plain UDP, maybe augmented
with a good message authenticator, without regard to retransmission and
sequencing?

Berkeley DB replication was carefully designed to be transport mechanism
agnostic, that is, we wanted to be able to run using TCP/IP, UDP, backplane
message passing protocol, or carrier pigeon.
Great! Thanks, Keith. That's what I hoped for.

Quote:
Obviously, this has serious implications for the application's transactional
guarantees: if the message didn't arrive at the replica, should the master
return to the application that the transaction has committed? This means
applications requiring serious transactional guarantees will have to do more
work when using transport protocols that are lossy, for example.
I'm roughly aware of that; for my application, the most important thing
will be that master and replicas eventually catch up to a common state,
after network and node failures, and after possible master changes.
Right now, I checkpoint & copy over the whole database, then manually
feed data changed during the copying into the new machine. Works for the
rarely changed ~30000 entries now, won't work well when things get more
frequent.

As my application happens to be UDP based, a pure UDP transport seems
to fit me very well. Good to hear it should work.

best regards
Patrick


Reply With Quote
  #4  
Old   
Chris Newcombe
 
Posts: n/a

Default Re: replication transport requirements - 09-04-2003 , 12:41 PM



mailer-daemon (AT) bof (DOT) de (Patrick Schaaf) wrote in message news:<3f56d2de$0$1584$9b622d9e (AT) news (DOT) freenet.de>...
Quote:
I have two fundamental questions regarding the requirements for
a Berkeley DB replication transport protocol:

Patrick, just a side note. We have been building a high-load
replication app (we happen to use TCP/IP), and are extremely happy
with the library. However, if you are using replication you should
ask support (AT) sleepycat (DOT) com for the latest v4.2.x pre-release snapshot.
Replication in 4.1.24/25 had a number of bugs, but 4.2.x is looking
very solid indeed in our stress tests.

regards,

Chris


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.