dbTalk Databases Forums  

Load if not exists

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Load if not exists in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Chris
 
Posts: n/a

Default Load if not exists - 07-14-2006 , 01:08 PM







In SQL 2005 SSIS - how would I be able to load data into a oledb destination
table only if the record does not exist?

I could load the data into a staging table then call a sproc to do an insert
using a left join where dest_table.pk is null, however I prefer not to manage
a staging table and want SSIS to first verify not exist, then load...

any help much appreciated,
chris

Reply With Quote
  #2  
Old   
Chris
 
Posts: n/a

Default RE: Load if not exists - 07-14-2006 , 04:25 PM







Ok - it appears the lookup transformation will work, I can ignore the error
that will stop the whole package when the lookup is not found. Fine - now I
can use a conditional split and pull all my null records to be sent to ole
destination.

Serious performance problem w/ lookup, before package even starts it goes to
lookup table, does a select * and caches entire table for lookup. Well, my 8
million record table brings the ssis server to it's knees. New problem now -
I will post new thread regarding this. Looks like all is solved w/ this
thread.



"Chris" wrote:

Quote:
In SQL 2005 SSIS - how would I be able to load data into a oledb destination
table only if the record does not exist?

I could load the data into a staging table then call a sproc to do an insert
using a left join where dest_table.pk is null, however I prefer not to manage
a staging table and want SSIS to first verify not exist, then load...

any help much appreciated,
chris

Reply With Quote
  #3  
Old   
Allan Mitchell
 
Posts: n/a

Default RE: Load if not exists - 07-15-2006 , 04:36 AM



Hello Chris,


Chris see my earlier post re the memory and caching.

As for what to do with "no match was found"


You have a couple of options


In the error setup you can

Fail Component - Default - No match found and the component fails
Redirect Row - Any source rows not matched go dow the error flow
Ignore Error - Pump down the normal path all rows and will contain NULLs
where no match possible.


Allan



Quote:
Ok - it appears the lookup transformation will work, I can ignore the
error that will stop the whole package when the lookup is not found.
Fine - now I can use a conditional split and pull all my null records
to be sent to ole destination.

Serious performance problem w/ lookup, before package even starts it
goes to lookup table, does a select * and caches entire table for
lookup. Well, my 8 million record table brings the ssis server to it's
knees. New problem now - I will post new thread regarding this. Looks
like all is solved w/ this thread.

"Chris" wrote:

In SQL 2005 SSIS - how would I be able to load data into a oledb
destination table only if the record does not exist?

I could load the data into a staging table then call a sproc to do an
insert using a left join where dest_table.pk is null, however I
prefer not to manage a staging table and want SSIS to first verify
not exist, then load...

any help much appreciated,
chris



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.