dbTalk Databases Forums  

Import csv Updte and delete

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Import csv Updte and delete in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Gustavo
 
Posts: n/a

Default Import csv Updte and delete - 07-15-2004 , 01:08 PM






Hello All,
Access recently kicked me out of its domain (data
warehouse too big too handle) and I landed in the
beautiful SQL 2000.
This is probably very basic but I need some guidance.

Every week I have a csv file with over 7,500,000 records
that I need to:
1-Import it to an existing SQL table and at the same
time...
2-Update the existing records and append the new ones.

The csv file drops a few thousands of old records every
so often and I need to keep them forever in the SQL table
once they are there, otherwise I would probably just try
to delete and append all records every week, which
probably is not the best approach anyway.

The number of fields, length and order is always the same
in the csv file.
I have been reading about DTS and ActivX scripts, Bulk
Insert,Triggers and other methods but they are all new to
me and I am not sure which one is the best to accomplish
this.
I tried using the wizard but I was only able to create a
new table but not even append them to an existing one.

The question is what would be the best approach to
achieve this task?..or at least part of it? Where do I
look for examples or tutorials? I bought a book called
Microsoft SQL 2000 Bible by Paul Nielsen but I has not
been much help for me.

Any advice or guidance will be greatly appreciated
Thanks
Gustavo


Reply With Quote
  #2  
Old   
Allan Mitchell
 
Posts: n/a

Default Re: Import csv Updte and delete - 07-15-2004 , 01:36 PM






In article <2dd5201c46a96$c37db290$a501280a (AT) phx (DOT) gbl>, Gustavo wrote:
Quote:
Hello All,
Access recently kicked me out of its domain (data
warehouse too big too handle) and I landed in the
beautiful SQL 2000.
This is probably very basic but I need some guidance.

Every week I have a csv file with over 7,500,000 records
that I need to:
1-Import it to an existing SQL table and at the same
time...
2-Update the existing records and append the new ones.

The csv file drops a few thousands of old records every
so often and I need to keep them forever in the SQL table
once they are there, otherwise I would probably just try
to delete and append all records every week, which
probably is not the best approach anyway.

The number of fields, length and order is always the same
in the csv file.
I have been reading about DTS and ActivX scripts, Bulk
Insert,Triggers and other methods but they are all new to
me and I am not sure which one is the best to accomplish
this.
I tried using the wizard but I was only able to create a
new table but not even append them to an existing one.

The question is what would be the best approach to
achieve this task?..or at least part of it? Where do I
look for examples or tutorials? I bought a book called
Microsoft SQL 2000 Bible by Paul Nielsen but I has not
been much help for me.

Any advice or guidance will be greatly appreciated
Thanks
Gustavo

So what I personally would do is have a staging table of the same
format as your text file.

You load that up using BULK INSERT (either the task or the TSQL
command).
You then use TSQL to do your UPDATEs and INSERTs in that Order.

You will need to join on a key for the updates and you will need to do
the rows that are in the staging table but not in the real table by
outer joining the two.




Allan Mitchell (Microsoft SQL Server MVP)
MCSE,MCDBA
www.SQLDTS.com
www.konesans.com - for all your consultancy needs





Reply With Quote
  #3  
Old   
Gustavo
 
Posts: n/a

Default Re: Import csv Updte and delete - 07-15-2004 , 01:40 PM



Thanks for the advice Allan, I will look into that
Gustavo
Quote:
-----Original Message-----
In article <2dd5201c46a96$c37db290$a501280a (AT) phx (DOT) gbl>,
Gustavo wrote:
Hello All,
Access recently kicked me out of its domain (data
warehouse too big too handle) and I landed in the
beautiful SQL 2000.
This is probably very basic but I need some guidance.

Every week I have a csv file with over 7,500,000
records
that I need to:
1-Import it to an existing SQL table and at the same
time...
2-Update the existing records and append the new ones.

The csv file drops a few thousands of old records
every
so often and I need to keep them forever in the SQL
table
once they are there, otherwise I would probably just
try
to delete and append all records every week, which
probably is not the best approach anyway.

The number of fields, length and order is always the
same
in the csv file.
I have been reading about DTS and ActivX scripts, Bulk
Insert,Triggers and other methods but they are all new
to
me and I am not sure which one is the best to
accomplish
this.
I tried using the wizard but I was only able to create
a
new table but not even append them to an existing one.

The question is what would be the best approach to
achieve this task?..or at least part of it? Where do I
look for examples or tutorials? I bought a book called
Microsoft SQL 2000 Bible by Paul Nielsen but I has not
been much help for me.

Any advice or guidance will be greatly appreciated
Thanks
Gustavo


So what I personally would do is have a staging table of
the same
format as your text file.

You load that up using BULK INSERT (either the task or
the TSQL
command).
You then use TSQL to do your UPDATEs and INSERTs in that
Order.

You will need to join on a key for the updates and you
will need to do
the rows that are in the staging table but not in the
real table by
outer joining the two.




Allan Mitchell (Microsoft SQL Server MVP)
MCSE,MCDBA
www.SQLDTS.com
www.konesans.com - for all your consultancy needs



.


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.