dbTalk Databases Forums  

One way sync

comp.databases.mysql comp.databases.mysql


Discuss One way sync in the comp.databases.mysql forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Devin M
 
Posts: n/a

Default One way sync - 10-28-2010 , 03:20 PM






Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.

Any help would be appreciated,
Devin Morin

Reply With Quote
  #2  
Old   
The Natural Philosopher
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 03:25 PM






Devin M wrote:
Quote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.

Any help would be appreciated,
Devin Morin
Real time? Or some sort of backup?

as far as backup goes, you can, as I have discovered, rsync ISAM files.
Ugly but it works, especially if you stop the mysqld process while you
do it.

Real time syncing is another matter.

Reply With Quote
  #3  
Old   
Devin M
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 03:28 PM



On Oct 28, 1:25*pm, The Natural Philosopher <t... (AT) invalid (DOT) invalid>
wrote:
Quote:
Devin M wrote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.

Any help would be appreciated,
Devin Morin

Real time? Or some sort of backup?

as far as backup goes, you can, as I have discovered, rsync ISAM files.
Ugly but it works, especially if you stop the mysqld process while you
do it.

Real time syncing is another matter.
Real time, would probably like to have it compare every 5 min with a
cron job and then sync if it needs to.

Reply With Quote
  #4  
Old   
Jerry Stuckle
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 03:56 PM



On 10/28/2010 4:28 PM, Devin M wrote:
Quote:
On Oct 28, 1:25 pm, The Natural Philosopher<t... (AT) invalid (DOT) invalid
wrote:
Devin M wrote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.

Any help would be appreciated,
Devin Morin

Real time? Or some sort of backup?

as far as backup goes, you can, as I have discovered, rsync ISAM files.
Ugly but it works, especially if you stop the mysqld process while you
do it.

Real time syncing is another matter.

Real time, would probably like to have it compare every 5 min with a
cron job and then sync if it needs to.
TNP is incorrect (as usual). You can't just rsync the files without
stopping MySQL. A lot of data may still be held in the buffers and not
written to disk yet. The result will be an inconsistent database.

Have you looked into replication?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex (AT) attglobal (DOT) net
==================

Reply With Quote
  #5  
Old   
Devin M
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 04:17 PM



On Oct 28, 1:56*pm, Jerry Stuckle <jstuck... (AT) attglobal (DOT) net> wrote:
Quote:
On 10/28/2010 4:28 PM, Devin M wrote:









On Oct 28, 1:25 pm, The Natural Philosopher<t... (AT) invalid (DOT) invalid
wrote:
Devin M wrote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.

Any help would be appreciated,
Devin Morin

Real time? Or some sort of backup?

as far as backup goes, you can, as I have discovered, rsync ISAM files..
Ugly but it works, especially if you stop the mysqld process while you
do it.

Real time syncing is another matter.

Real time, would probably like to have it compare every 5 min with a
cron job and then sync if it needs to.

TNP is incorrect (as usual). You can't just rsync the files without
stopping MySQL. *A lot of data may still be held in the buffers and not
written to disk yet. *The result will be an inconsistent database.

Have you looked into replication?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstuck... (AT) attglobal (DOT) net
==================
No I have not looked into replication yet, however I was thinking
that an easier solution might be to push the data to the slave servers
whenever a record is created or updated. What I have right now are a
few DNS Servers that use MySql for the data store. Maybe I can write a
C daemon to monitor the primary database and if there are any changes
it can send them over the network to a client running on the slave
servers? Would I be basically duplicating any replication solutions
that are already out there?
Regards,
Devin

Reply With Quote
  #6  
Old   
Devin M
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 04:25 PM



On Oct 28, 1:56*pm, Jerry Stuckle <jstuck... (AT) attglobal (DOT) net> wrote:
Quote:
On 10/28/2010 4:28 PM, Devin M wrote:









On Oct 28, 1:25 pm, The Natural Philosopher<t... (AT) invalid (DOT) invalid
wrote:
Devin M wrote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.

Any help would be appreciated,
Devin Morin

Real time? Or some sort of backup?

as far as backup goes, you can, as I have discovered, rsync ISAM files..
Ugly but it works, especially if you stop the mysqld process while you
do it.

Real time syncing is another matter.

Real time, would probably like to have it compare every 5 min with a
cron job and then sync if it needs to.

TNP is incorrect (as usual). You can't just rsync the files without
stopping MySQL. *A lot of data may still be held in the buffers and not
written to disk yet. *The result will be an inconsistent database.

Have you looked into replication?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstuck... (AT) attglobal (DOT) net
==================
Looks like my reply got lost, Basically I have a few DNS servers that
need to pull the records from a master database server and keep them
on a local mysql server. I have not looked into replication but
basically what I am thinking about doing is writing a daemon in C that
can run on the two different servers, check for any missing or
incorrect data on the slave side and use the master server as the data
source. The only problem I have with this is a lack of experience with
the C library. Can you recommend a efficient way to compare and send
data?

Regards,
Devin

Reply With Quote
  #7  
Old   
Axel Schwenke
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 04:26 PM



Devin M <devinmrn (AT) gmail (DOT) com> wrote:

Quote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database..
Any reason why you use the terms "master" and "slave" here?
Do you mean that in the sense of MySQL replication?

Quote:
Basically I need an efficient way to find if a database is
out of sync and a way to sync it back to the master.
"out of sync" is actually hard to check. But MySQL can write
a log that contains all modifications that were done locally.
This log can be shipped and applied to another MySQL instance.
This is how MySQL implements replication. The "master" writes
this log and the "slave" fetches the log and applies it.

Standard replication allows each slave to have only one master.
But if you implement log-shipping and -application on your own
(this is next to trivial with the available tools) and if the
changes done on the two(!) masters are compatible, then this
approach might work for you.

Another idea would be to form a ring of the 3 instances. This
is a standard setup, but it's harder to setup (and maintain!).

RTFM!


XL

Reply With Quote
  #8  
Old   
Axel Schwenke
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 04:38 PM



Devin M <devinmrn (AT) gmail (DOT) com> wrote:
Quote:
No I have not looked into replication yet, however I was thinking
that an easier solution might be to push the data to the slave servers
whenever a record is created or updated.
LOL.

This is exactly how MySQL implements replication.


XL

Reply With Quote
  #9  
Old   
Devin M
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 05:50 PM



On Oct 28, 2:38*pm, Axel Schwenke <axel.schwe... (AT) gmx (DOT) de> wrote:
Quote:
Devin M <devin... (AT) gmail (DOT) com> wrote:

No *I have not looked into replication yet, however I was thinking
that an easier solution might be to push the data to the slave servers
whenever a record is created or updated.

LOL.

This is exactly how MySQL implements replication.

XL
Haha yeah im most likely reinventing the wheel here, Im going to take
a look at replication in a few. Hopefully it fits my needs.

Reply With Quote
  #10  
Old   
The Natural Philosopher
 
Posts: n/a

Default Re: One way sync - 10-28-2010 , 07:36 PM



Devin M wrote:
Quote:
On Oct 28, 1:56 pm, Jerry Stuckle <jstuck... (AT) attglobal (DOT) net> wrote:
On 10/28/2010 4:28 PM, Devin M wrote:









On Oct 28, 1:25 pm, The Natural Philosopher<t... (AT) invalid (DOT) invalid
wrote:
Devin M wrote:
Hello, I am wondering what would be the most efficient way to
synchronize two slave databases to a master database.. Basically I
need an efficient way to find if a database is out of sync and a way
to sync it back to the master.
Any help would be appreciated,
Devin Morin
Real time? Or some sort of backup?
as far as backup goes, you can, as I have discovered, rsync ISAM files.
Ugly but it works, especially if you stop the mysqld process while you
do it.
Real time syncing is another matter.
Real time, would probably like to have it compare every 5 min with a
cron job and then sync if it needs to.
TNP is incorrect (as usual). You can't just rsync the files without
stopping MySQL. A lot of data may still be held in the buffers and not
written to disk yet. The result will be an inconsistent database.

which is more or less what I said, of course....



Quote:
Have you looked into replication?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstuck... (AT) attglobal (DOT) net
==================

No I have not looked into replication yet, however I was thinking
that an easier solution might be to push the data to the slave servers
whenever a record is created or updated. What I have right now are a
few DNS Servers that use MySql for the data store. Maybe I can write a
C daemon to monitor the primary database and if there are any changes
it can send them over the network to a client running on the slave
servers? Would I be basically duplicating any replication solutions
that are already out there?
Regards,
Devin

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.