dbTalk Databases Forums  

Quick data transformation using SSIS

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Quick data transformation using SSIS in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Ahamed Faridh
 
Posts: n/a

Default Quick data transformation using SSIS - 09-24-2008 , 11:39 AM








I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager-->
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***

Reply With Quote
  #2  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM






Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #3  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #4  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #5  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #6  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #7  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #8  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
  #9  
Old   
Todd C
 
Posts: n/a

Default RE: Quick data transformation using SSIS - 09-24-2008 , 12:28 PM



Take a look at the DefaultBufferMaxRows property for the Data Flow task in
question. (3rd in the list under Misc.) The default is 10,000. Since the data
flow 'pipeline' happens in-memory, you will be somewhat restricted by
available memory on the machine.

HTH
--
Todd C

[If this response was helpful, please indicate by clicking the appropriate
answer at the bottom]


"Ahamed Faridh" wrote:

Quote:

I am having 80 millions record in a table. I want to move all the
records to a destination table using SSIS. I am using OLEDB Source and
OLEDB Destination in Dataflow Tasks for moving the records.

I want to move 10000 records per batch. I have updated 10000 in the
OLEDB Destination Editor-->Connection Manager--
Rows per Batch = 10000. The Default value was 1.

My Question is :

1)Does this update, really boost up my performance of the
transformation?
2) What is the maximum value i can give which does not affect my
performance.
3) Do we have any other Data Flow Task which transfers the data quickly
between tables in SSIS.

*** Sent via Developersdex http://www.developersdex.com ***


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.