dbTalk Databases Forums  

How to execute a group of tasks in parallel

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss How to execute a group of tasks in parallel in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Calvin KD
 
Posts: n/a

Default How to execute a group of tasks in parallel - 10-04-2005 , 12:34 AM






Hi everyone,
Can someone tell me how to design my DTS to import Reference Tables (those
with Primary Keys) to run in parallel. Since each Ref. table is independent
of each other, i figured that it's much more efficient to import them in
parallel as appose to sequential. However, the question is, how do i design
the DTS to do this, then depending on success/failure, it continues to import
the rest of the tables (those with Foreign Keys).
Source : Oracle
Destination: SQL Server 2000
As a summary, what i would ideally like is as follows:
Get data from Oracle -> Create SQL Temp tables -> Import Ref. tables into
temp tables -> Import other tables (with foreign keys) -> Insert/Update
master tables (SQL)

Any suggestion is greatly appreciated.
Calvin


Reply With Quote
  #2  
Old   
Darren Green
 
Posts: n/a

Default Re: How to execute a group of tasks in parallel - 10-04-2005 , 07:01 AM






Draw the package with multiple tasks in parallel, then lead them all to the
single point of convergence, through Exec SQL Tasks.

To ensure you get parallel running, use separate connections for each task,
do not re-use.
Set the Execution parameters, package properties which has a thread limit.

Bandwidth is often the biggest restriction, so ensure you have enough to
make parallel running worthwhile. Also set the batch sizes to a suitable
value - test it.


--
Darren Green
http://www.sqldts.com
http://www.sqlis.com

"Calvin KD" <CalvinKD (AT) discussions (DOT) microsoft.com> wrote

Quote:
Hi everyone,
Can someone tell me how to design my DTS to import Reference Tables (those
with Primary Keys) to run in parallel. Since each Ref. table is
independent
of each other, i figured that it's much more efficient to import them in
parallel as appose to sequential. However, the question is, how do i
design
the DTS to do this, then depending on success/failure, it continues to
import
the rest of the tables (those with Foreign Keys).
Source : Oracle
Destination: SQL Server 2000
As a summary, what i would ideally like is as follows:
Get data from Oracle -> Create SQL Temp tables -> Import Ref. tables into
temp tables -> Import other tables (with foreign keys) -> Insert/Update
master tables (SQL)

Any suggestion is greatly appreciated.
Calvin




Reply With Quote
  #3  
Old   
Calvin KD
 
Posts: n/a

Default Re: How to execute a group of tasks in parallel - 10-04-2005 , 09:35 PM



Thanks so much Darren. It helped.

Cheers,
Calvin.

"Darren Green" wrote:

Quote:
Draw the package with multiple tasks in parallel, then lead them all to the
single point of convergence, through Exec SQL Tasks.

To ensure you get parallel running, use separate connections for each task,
do not re-use.
Set the Execution parameters, package properties which has a thread limit.

Bandwidth is often the biggest restriction, so ensure you have enough to
make parallel running worthwhile. Also set the batch sizes to a suitable
value - test it.


--
Darren Green
http://www.sqldts.com
http://www.sqlis.com

"Calvin KD" <CalvinKD (AT) discussions (DOT) microsoft.com> wrote in message
news:2B3B40F8-8A80-4048-B6FE-7F3F591163AC (AT) microsoft (DOT) com...
Hi everyone,
Can someone tell me how to design my DTS to import Reference Tables (those
with Primary Keys) to run in parallel. Since each Ref. table is
independent
of each other, i figured that it's much more efficient to import them in
parallel as appose to sequential. However, the question is, how do i
design
the DTS to do this, then depending on success/failure, it continues to
import
the rest of the tables (those with Foreign Keys).
Source : Oracle
Destination: SQL Server 2000
As a summary, what i would ideally like is as follows:
Get data from Oracle -> Create SQL Temp tables -> Import Ref. tables into
temp tables -> Import other tables (with foreign keys) -> Insert/Update
master tables (SQL)

Any suggestion is greatly appreciated.
Calvin





Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.