dbTalk Databases Forums  

Incremental export best practice?

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Incremental export best practice? in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Johnny Orson
 
Posts: n/a

Default Incremental export best practice? - 11-22-2004 , 07:33 AM






Hi,

I have several dts packages which I want to set from "export all
records" to incremental export. I need to export these records once an
hour, so my first idea is to modify the select statement to only
include the records from the last hour, but, when the server misses a
turn for some reason (rebooting), that means the integrity is broken.

Now, in the tables I export, I do have an autoincemental ID-Field. Is
it a good practice to set up a table, where, with every export, I
update a field (i.e. MAXID) with the current maximum ID I exported,
and modify the export DTS package to first fetch the current MAXID and
then select only the records with ID > that MAXID ?

I never did this, but I want to do it right and foolproof, so, I would
like some advise, especially because I am not sure how to put the two
steps needed into one dts package.

Best Regards!
John

Reply With Quote
  #2  
Old   
Darren Green
 
Posts: n/a

Default Re: Incremental export best practice? - 11-23-2004 , 07:49 AM






"Johnny Orson" <obesityrules (AT) gmx (DOT) li> wrote

Quote:
Hi,

I have several dts packages which I want to set from "export all
records" to incremental export. I need to export these records once an
hour, so my first idea is to modify the select statement to only
include the records from the last hour, but, when the server misses a
turn for some reason (rebooting), that means the integrity is broken.

Now, in the tables I export, I do have an autoincemental ID-Field. Is
it a good practice to set up a table, where, with every export, I
update a field (i.e. MAXID) with the current maximum ID I exported,
and modify the export DTS package to first fetch the current MAXID and
then select only the records with ID > that MAXID ?
Yes, that would be my preferred method. Just make sure you only store this
once you have successfully extracted the data, and you rollback the extract
if "store" part fails as well. A transaction may be useful, or just logicaly
order steps to minimise risk.


Quote:
I never did this, but I want to do it right and foolproof, so, I would
like some advise, especially because I am not sure how to put the two
steps needed into one dts package.

Best Regards!
John



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.