dbTalk Databases Forums  

Preventing import of duplicate records

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Preventing import of duplicate records in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Ingar
 
Posts: n/a

Default Preventing import of duplicate records - 01-17-2006 , 08:02 AM






I'm trying to import a text file into a SQL 2000 database. But my challenge
is to make sure my customer doesn't import the same text file twice. The file
does not have any unique ID. I may be able to make an unique key combined by
2 or 3 of the fields in the file. But how should I check against this when
importing the file. I was hoping to use a DTS package ti import the file.

Hope someone understands what I'm after, and have some suggestions on what
to do.

Reply With Quote
  #2  
Old   
Ryan
 
Posts: n/a

Default Re: Preventing import of duplicate records - 01-17-2006 , 09:01 AM






Assuming you are able to make some uniqueness about the records by combining
columns this would work :-

Rather than loading the text file into it's final destination do this in
your DTS :-

Create Holding_Table
Insert Rows into Holding_Table
Remove any dupes from Holding_Table
INSERT Destination_Table SELECT * FROM Holding_Table WHERE UniqueID NOT
IN (Select UniqueID FROM Destination_Table)
Drop Holding_Table

Depending on the volume of data it may be quicker if you add an index to the
Holding_Table and rahter than using my NOT IN syntax try WHERE NOT EXISTS..

--
HTH. Ryan
"Ingar" <Ingar (AT) discussions (DOT) microsoft.com> wrote

Quote:
I'm trying to import a text file into a SQL 2000 database. But my
challenge
is to make sure my customer doesn't import the same text file twice. The
file
does not have any unique ID. I may be able to make an unique key combined
by
2 or 3 of the fields in the file. But how should I check against this when
importing the file. I was hoping to use a DTS package ti import the file.

Hope someone understands what I'm after, and have some suggestions on what
to do.



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.