dbTalk Databases Forums  

SSIS Performace/Design

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss SSIS Performace/Design in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
brian_harris
 
Posts: n/a

Default SSIS Performace/Design - 06-16-2006 , 02:07 PM






I have a question about correct design/performace for the data source in SSIS.
I am moving data from some oracle tables to sqlserver database on a regular
basis.
Some tables only have a few thousand records but 1 has 2 million another has
4 million records. Based on my current understanding I have placed all of
these data moves into 1 data source so that they can be executed in parallel.
I am presuming that this will give me best performance. After all of data
is copied I then execute some execute sql task on individual tables. I do
not care what order the data is copied from remote oracle database.

So my question is this the best way to design this part of application? Did
I make any bad assumptions? Is their any reason that I should put each copy
data into its own data source?

Reply With Quote
  #2  
Old   
Allan Mitchell
 
Posts: n/a

Default Re: SSIS Performace/Design - 06-17-2006 , 09:08 AM






Hello brian_harris,

If you do not care about the order of the data being loaded then I would
be tempted on a multi proc box to divide my source data into ranges. Let's
take time for instance. Divide the source data up into equalish chunks.
How many chunks? I generally start with CPU Count - 1.

Now in either a single Data Flow Task with CPU Count - 1 Pipelines or CPU
Count - 1 Data flow tasks with 1 pipeline in each you move that data in parallel.

Donald farmer gave a presentation the pther night on Perf. You should be
able to download it over at MSDN Webcasts


Allan



Quote:
I have a question about correct design/performace for the data source
in SSIS.
I am moving data from some oracle tables to sqlserver database on a
regular
basis.
Some tables only have a few thousand records but 1 has 2 million
another has
4 million records. Based on my current understanding I have placed
all of
these data moves into 1 data source so that they can be executed in
parallel.
I am presuming that this will give me best performance. After all of
data
is copied I then execute some execute sql task on individual tables.
I do
not care what order the data is copied from remote oracle database.

So my question is this the best way to design this part of
application? Did I make any bad assumptions? Is their any reason that
I should put each copy data into its own data source?




Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.