dbTalk Databases Forums  

Implementing Incremental Rollforwards

comp.databases.ingres comp.databases.ingres


Discuss Implementing Incremental Rollforwards in the comp.databases.ingres forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
jm
 
Posts: n/a

Default Implementing Incremental Rollforwards - 09-08-2011 , 05:51 AM






I've just started looking at setting up incremental rollforwards for a
client and I have a question:

The documentation says that all journals since the first rollforwarddb
-incremental have to be present before running the final rollforwarddb
-rollback. If you take daily checkpoints, then presumably only those
journals created since the checkpoint are necessary (as a new journal
is created with a checkpoint). Does anyone know if this is the case?

I'm running Ingres 9.2.

Thanks

Reply With Quote
  #2  
Old   
Martin Bowes
 
Posts: n/a

Default Re: [Info-Ingres] Implementing Incremental Rollforwards - 09-08-2011 , 06:30 AM






Hi Jackie,

Only those journals taken since the checkpoint you started the incremental recovery from are required. So if you have checkpoint N on your recovery host but are at checkpoint N+5 on your source host then the recovery host still needs the journals taken since N.

Note that there have been bugs in incremental recovery which will prevent you recovering through a checkpoint boundary...these have been fixed but you will need to check your patch level.

There have also been bugs which make the recovered database single user.

Otherwise I have found it to be extremely useful, very easy to set up and script around plus very robust.

Have fun.

Martin Bowes

-----Original Message-----
From: jm [mailto:jackie_mclellen (AT) talk21 (DOT) com]
Sent: 08 September 2011 11:51
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: [Info-Ingres] Implementing Incremental Rollforwards

I've just started looking at setting up incremental rollforwards for a
client and I have a question:

The documentation says that all journals since the first rollforwarddb
-incremental have to be present before running the final rollforwarddb
-rollback. If you take daily checkpoints, then presumably only those
journals created since the checkpoint are necessary (as a new journal
is created with a checkpoint). Does anyone know if this is the case?

I'm running Ingres 9.2.

Thanks
_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Reply With Quote
  #3  
Old   
jm
 
Posts: n/a

Default Re: Implementing Incremental Rollforwards - 09-08-2011 , 07:21 AM



On Sep 8, 12:30*pm, Martin Bowes <martin.bo... (AT) ctsu (DOT) ox.ac.uk> wrote:
Quote:
Hi Jackie,

Only those journals taken since the checkpoint you started the incremental recovery from are required. So if you have checkpoint N on your recovery host but are at checkpoint N+5 on your source host then the recovery host still needs the journals taken since N.

Note that there have been bugs in incremental recovery which will preventyou recovering through a checkpoint boundary...these have been fixed but you will need to check your patch level.

There have also been bugs which make the recovered database single user.

Otherwise I have found it to be extremely useful, very easy to set up andscript around plus very robust.

Have fun.

Martin Bowes

-----Original Message-----
From: jm [mailto:jackie_mclel... (AT) talk21 (DOT) com]
Sent: 08 September 2011 11:51
To: info-ing... (AT) kettleriverconsulting (DOT) com
Subject: [Info-Ingres] Implementing Incremental Rollforwards

I've just started looking at setting up incremental rollforwards for a
client and I have a question:

The documentation says that all journals since the first rollforwarddb
-incremental have to be present before running the final rollforwarddb
-rollback. If you take daily checkpoints, then presumably only those
journals created since the checkpoint are necessary (as a new journal
is created with a checkpoint). Does anyone know if this is the case?

I'm running Ingres 9.2.

Thanks
_______________________________________________
Info-Ingres mailing list
Info-Ing... (AT) kettleriverconsulting (DOT) comhttp://ext-cando.kettleriverconsulting.com/mailman/listinfo/info-ingres


Hi Martin,

Thanks for the swift reply, but can you explain why the rollfrward
needs the earlier journals. Once it's applied them, surely it won't
look at them again when we run the final rollforward. I know that all
the transactions have been committed because we are running an offline
checkpoint.

Thanks

Jackie

Reply With Quote
  #4  
Old   
Paul White
 
Posts: n/a

Default Re: [Info-Ingres] Implementing Incremental Rollforwards - 09-08-2011 , 07:33 AM



If you have correctly copied checkpoint N+5 along with associated dump, cnf
and all journals after N+5, then you can disregard journals for checkpoints
prior to N+5.

You just need to re-initialise the by clearing data, copy the latest cnf to
the data directory then re-start the rollforward sequence. I do this each
day to get around the occasional problem to which Marty refers.

Paul

-----Original Message-----
From: info-ingres-bounces (AT) kettleriver...ting (DOT) com
[mailto:info-ingres-bounces (AT) kettleriverconsulting (DOT) com] On Behalf Of Martin
Bowes
Sent: Thursday, 8 September 2011 9:30 PM
To: Ingres and related product discussion forum
Cc: jm
Subject: Re: [Info-Ingres] Implementing Incremental Rollforwards

Hi Jackie,

Only those journals taken since the checkpoint you started the incremental
recovery from are required. So if you have checkpoint N on your recovery
host but are at checkpoint N+5 on your source host then the recovery host
still needs the journals taken since N.

Note that there have been bugs in incremental recovery which will prevent
you recovering through a checkpoint boundary...these have been fixed but you
will need to check your patch level.

There have also been bugs which make the recovered database single user.

Otherwise I have found it to be extremely useful, very easy to set up and
script around plus very robust.

Have fun.

Martin Bowes

-----Original Message-----
From: jm [mailto:jackie_mclellen (AT) talk21 (DOT) com]
Sent: 08 September 2011 11:51
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: [Info-Ingres] Implementing Incremental Rollforwards

I've just started looking at setting up incremental rollforwards for a
client and I have a question:

The documentation says that all journals since the first rollforwarddb
-incremental have to be present before running the final rollforwarddb
-rollback. If you take daily checkpoints, then presumably only those
journals created since the checkpoint are necessary (as a new journal
is created with a checkpoint). Does anyone know if this is the case?

I'm running Ingres 9.2.

Thanks
_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres


_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Reply With Quote
  #5  
Old   
jm
 
Posts: n/a

Default Re: Implementing Incremental Rollforwards - 09-08-2011 , 08:39 AM



On Sep 8, 1:33*pm, "Paul White" <shift7soluti... (AT) gmail (DOT) com> wrote:
Quote:
If you have correctly copied checkpoint N+5 along with associated dump, cnf
and all journals after N+5, then you can disregard journals for checkpoints
prior to N+5.

You just need to re-initialise the by clearing data, copy the latest cnf to
the data directory then re-start the rollforward sequence. I do this each
day to get around the occasional problem to which Marty refers.

Paul

-----Original Message-----
From: info-ingres-boun... (AT) kettleriverconsulting (DOT) com

[mailto:info-ingres-boun... (AT) kettleriverconsulting (DOT) com] On Behalf Of Martin
Bowes
Sent: Thursday, 8 September 2011 9:30 PM
To: Ingres and related product discussion forum
Cc: jm
Subject: Re: [Info-Ingres] Implementing Incremental Rollforwards

Hi Jackie,

Only those journals taken since the checkpoint you started the incremental
recovery from are required. So if you have checkpoint N on your recovery
host but are at checkpoint N+5 on your source host then the recovery host
still needs the journals taken since N.

Note that there have been bugs in incremental recovery which will prevent
you recovering through a checkpoint boundary...these have been fixed but you
will need to check your patch level.

There have also been bugs which make the recovered database single user.

Otherwise I have found it to be extremely useful, very easy to set up and
script around plus very robust.

Have fun.

Martin Bowes

-----Original Message-----
From: jm [mailto:jackie_mclel... (AT) talk21 (DOT) com]
Sent: 08 September 2011 11:51
To: info-ing... (AT) kettleriverconsulting (DOT) com
Subject: [Info-Ingres] Implementing Incremental Rollforwards

I've just started looking at setting up incremental rollforwards for a
client and I have a question:

The documentation says that all journals since the first rollforwarddb
-incremental have to be present before running the final rollforwarddb
-rollback. If you take daily checkpoints, then presumably only those
journals created since the checkpoint are necessary (as a new journal
is created with a checkpoint). Does anyone know if this is the case?

I'm running Ingres 9.2.

Thanks
_______________________________________________
Info-Ingres mailing list
Info-Ing... (AT) kettleriverconsulting (DOT) comhttp://ext-cando.kettleriverconsulting.com/mailman/listinfo/info-ingres

_______________________________________________
Info-Ingres mailing list
Info-Ing... (AT) kettleriverconsulting (DOT) comhttp://ext-cando.kettleriverconsulting.com/mailman/listinfo/info-ingres


Hi Paul,

But if I copy the cnf file to the default database location any time
after starting the first incremental rollforward, won't the database
status be incorrect - not INCR_RFP - which would prevent it processing
the journals incrementally?

Jackie

Reply With Quote
  #6  
Old   
Armand Pirvu (gmail)
 
Posts: n/a

Default Re: [Info-Ingres] Implementing Incremental Rollforwards - 09-08-2011 , 10:32 AM



There was a bug (which is fixed) which caused the need to have all journals from ckp 1 onwards.

Rollforwarddb changes the cnf in several areas not just the INCR_RFP. Look at the last log address which got zeroed. Why it got zeroed ? LSN's, LA's and the like combined of course with epoch time stamp etc etc. There is a KB how these relate (at least one IIRC)




Before

Status : VALID,JOURNAL,CKP,DUMP,ROLL_FORWARD,CFG_BACKUP

----Journal information---------------------------------------------------------
Last Log Address Journaled : <1309131331:67397:104>


After

Status : VALID,JOURNAL,DUMP,ROLL_FORWARD,CFG_BACKUP,INCR_RF P

----Journal information---------------------------------------------------------
Last Log Address Journaled : <0:0:0>



Copying the cnf as mentioned is for restarting rollforwarddb only. Copying the cnf and expecting rollforwarddb incremental to continue happy will not happen. It will prevent indeed the incremental processing of the journals

Most likely the error will be


E_DM1356_RFP_INVALID_ARGUMENT An invalid rollforwarddb argument has been specified.
E_DM1301_RFP_ROLL_FORWARD An error occurred that prevents further processing.



On Sep 8, 2011, at 8:39 AM, jm wrote:

Quote:
On Sep 8, 1:33 pm, "Paul White" <shift7soluti... (AT) gmail (DOT) com> wrote:
If you have correctly copied checkpoint N+5 along with associated dump, cnf
and all journals after N+5, then you can disregard journals for checkpoints
prior to N+5.

You just need to re-initialise the by clearing data, copy the latest cnf to
the data directory then re-start the rollforward sequence. I do this each
day to get around the occasional problem to which Marty refers.

Paul

-----Original Message-----
From: info-ingres-boun... (AT) kettleriverconsulting (DOT) com

[mailto:info-ingres-boun... (AT) kettleriverconsulting (DOT) com] On Behalf Of Martin
Bowes
Sent: Thursday, 8 September 2011 9:30 PM
To: Ingres and related product discussion forum
Cc: jm
Subject: Re: [Info-Ingres] Implementing Incremental Rollforwards

Hi Jackie,

Only those journals taken since the checkpoint you started the incremental
recovery from are required. So if you have checkpoint N on your recovery
host but are at checkpoint N+5 on your source host then the recovery host
still needs the journals taken since N.

Note that there have been bugs in incremental recovery which will prevent
you recovering through a checkpoint boundary...these have been fixed but you
will need to check your patch level.

There have also been bugs which make the recovered database single user.

Otherwise I have found it to be extremely useful, very easy to set up and
script around plus very robust.

Have fun.

Martin Bowes

-----Original Message-----
From: jm [mailto:jackie_mclel... (AT) talk21 (DOT) com]
Sent: 08 September 2011 11:51
To: info-ing... (AT) kettleriverconsulting (DOT) com
Subject: [Info-Ingres] Implementing Incremental Rollforwards

I've just started looking at setting up incremental rollforwards for a
client and I have a question:

The documentation says that all journals since the first rollforwarddb
-incremental have to be present before running the final rollforwarddb
-rollback. If you take daily checkpoints, then presumably only those
journals created since the checkpoint are necessary (as a new journal
is created with a checkpoint). Does anyone know if this is the case?

I'm running Ingres 9.2.

Thanks
_______________________________________________
Info-Ingres mailing list
Info-Ing... (AT) kettleriverconsulting (DOT) comhttp://ext-cando.kettleriverconsulting.com/mailman/listinfo/info-ingres

_______________________________________________
Info-Ingres mailing list
Info-Ing... (AT) kettleriverconsulting (DOT) comhttp://ext-cando.kettleriverconsulting.com/mailman/listinfo/info-ingres



Hi Paul,

But if I copy the cnf file to the default database location any time
after starting the first incremental rollforward, won't the database
status be incorrect - not INCR_RFP - which would prevent it processing
the journals incrementally?

Jackie
_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Reply With Quote
  #7  
Old   
Martin Bowes
 
Posts: n/a

Default Re: [Info-Ingres] Implementing Incremental Rollforwards - 09-09-2011 , 02:26 AM



Quote:
Thanks for the swift reply, but can you explain why the rollfrward
needs the earlier journals. Once it's applied them, surely it won't
look at them again when we run the final rollforward. I know that all
the transactions have been committed because we are running an offline
checkpoint.
I suspect that when the final rollforwarddb is done on the database (with: -c +j -incremental -rollback) you need to rollback uncommitted transactions. The system may need to scan the journals to work this out.

Marty

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.