dbTalk Databases Forums  

Copying 69 million records using DTS package

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Copying 69 million records using DTS package in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Rosie
 
Posts: n/a

Default Copying 69 million records using DTS package - 04-13-2006 , 07:52 PM






Hi,

Can I copy a table with 69 million records from one database to another in
the same server using DTS package? Or I have to create a stored procedure to
read and insert, for example 50000 records each time (in this case we will
have a slow process)?

Does DTS use transaction log like regular insert statements?

The table has about 30 columns. The datatype of most of them is CHAR or
VARCHAR.

Thank you,
Rosie






Reply With Quote
  #2  
Old   
Allan Mitchell
 
Posts: n/a

Default Re: Copying 69 million records using DTS package - 04-13-2006 , 08:50 PM






Hello rosie,

Yes you can certainly copy a table with 69 million rows using DTS

You can also use a proc to do this. (you could even put the proc inside DTS)

You can read the whole table in one go but remember that it will be kept
in the logspace until the whole lot commits. Batches will allow you to commit
the rows earlier and let the log breathe

It can use TXNs yes.

1. Enable transactions at the package level (Package Properties)
2. On Success between each task
3. In the workflow properties for each task (check "Join transaction if
present","Rollback Transaction on Failure")
4. Make sure MSDTC is started


NOTE: Not everything supports joining a transaction




Allan Mitchell
www.SQLDTS.com
www.SQLIS.com
www.Konesans.com

Quote:
Hi,

Can I copy a table with 69 million records from one database to
another in the same server using DTS package? Or I have to create a
stored procedure to read and insert, for example 50000 records each
time (in this case we will have a slow process)?

Does DTS use transaction log like regular insert statements?

The table has about 30 columns. The datatype of most of them is CHAR
or VARCHAR.

Thank you,
Rosie



Reply With Quote
  #3  
Old   
Jéjé
 
Posts: n/a

Default Re: Copying 69 million records using DTS package - 04-14-2006 , 07:36 AM



Hi,

if you use the bulk insert to copy the rows from 1 table to another, the log
generated depends on the database log setup.
if the log setup of target database is FULL, then EVERYTHING is logged.
if its set to Bulk Log or simple, then bulk insert commands are logged.

the log is linked to the command executed to the database NOT the program
used to load the data, you can use another tool to load your data, until its
a bulkinsert command, there is no log

so, if you do the copy using a storeprocedure, you'll generate a log. Using
DTS or not.

"Rosie" <Rosie (AT) discussions (DOT) microsoft.com> wrote

Quote:
Hi,

Can I copy a table with 69 million records from one database to another in
the same server using DTS package? Or I have to create a stored procedure
to
read and insert, for example 50000 records each time (in this case we will
have a slow process)?

Does DTS use transaction log like regular insert statements?

The table has about 30 columns. The datatype of most of them is CHAR or
VARCHAR.

Thank you,
Rosie








Reply With Quote
  #4  
Old   
Rosie
 
Posts: n/a

Default Re: Copying 69 million records using DTS package - 04-20-2006 , 05:21 PM



Thank you.

"Jéjé" wrote:

Quote:
Hi,

if you use the bulk insert to copy the rows from 1 table to another, the log
generated depends on the database log setup.
if the log setup of target database is FULL, then EVERYTHING is logged.
if its set to Bulk Log or simple, then bulk insert commands are logged.

the log is linked to the command executed to the database NOT the program
used to load the data, you can use another tool to load your data, until its
a bulkinsert command, there is no log

so, if you do the copy using a storeprocedure, you'll generate a log. Using
DTS or not.

"Rosie" <Rosie (AT) discussions (DOT) microsoft.com> wrote in message
news:35F039C7-E8E1-4241-97AC-A712B78B2688 (AT) microsoft (DOT) com...
Hi,

Can I copy a table with 69 million records from one database to another in
the same server using DTS package? Or I have to create a stored procedure
to
read and insert, for example 50000 records each time (in this case we will
have a slow process)?

Does DTS use transaction log like regular insert statements?

The table has about 30 columns. The datatype of most of them is CHAR or
VARCHAR.

Thank you,
Rosie









Reply With Quote
  #5  
Old   
vipinjosea
 
Posts: n/a

Default Re: Copying 69 million records using DTS package - 04-21-2006 , 08:41 AM



http://www.codeproject.com/useritems/DTS__VBNET_.asp


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.