![]() | |
![]() |
| | Thread Tools | Display Modes |
#1
| |||
| |||
|
#2
| |||
| |||
|
|
Hello All, Access recently kicked me out of its domain (data warehouse too big too handle) and I landed in the beautiful SQL 2000. This is probably very basic but I need some guidance. Every week I have a csv file with over 7,500,000 records that I need to: 1-Import it to an existing SQL table and at the same time... 2-Update the existing records and append the new ones. The csv file drops a few thousands of old records every so often and I need to keep them forever in the SQL table once they are there, otherwise I would probably just try to delete and append all records every week, which probably is not the best approach anyway. The number of fields, length and order is always the same in the csv file. I have been reading about DTS and ActivX scripts, Bulk Insert,Triggers and other methods but they are all new to me and I am not sure which one is the best to accomplish this. I tried using the wizard but I was only able to create a new table but not even append them to an existing one. The question is what would be the best approach to achieve this task?..or at least part of it? Where do I look for examples or tutorials? I bought a book called Microsoft SQL 2000 Bible by Paul Nielsen but I has not been much help for me. Any advice or guidance will be greatly appreciated Thanks Gustavo |
#3
| |||
| |||
|
|
-----Original Message----- In article <2dd5201c46a96$c37db290$a501280a (AT) phx (DOT) gbl>, Gustavo wrote: Hello All, Access recently kicked me out of its domain (data warehouse too big too handle) and I landed in the beautiful SQL 2000. This is probably very basic but I need some guidance. Every week I have a csv file with over 7,500,000 records that I need to: 1-Import it to an existing SQL table and at the same time... 2-Update the existing records and append the new ones. The csv file drops a few thousands of old records every so often and I need to keep them forever in the SQL table once they are there, otherwise I would probably just try to delete and append all records every week, which probably is not the best approach anyway. The number of fields, length and order is always the same in the csv file. I have been reading about DTS and ActivX scripts, Bulk Insert,Triggers and other methods but they are all new to me and I am not sure which one is the best to accomplish this. I tried using the wizard but I was only able to create a new table but not even append them to an existing one. The question is what would be the best approach to achieve this task?..or at least part of it? Where do I look for examples or tutorials? I bought a book called Microsoft SQL 2000 Bible by Paul Nielsen but I has not been much help for me. Any advice or guidance will be greatly appreciated Thanks Gustavo So what I personally would do is have a staging table of the same format as your text file. You load that up using BULK INSERT (either the task or the TSQL command). You then use TSQL to do your UPDATEs and INSERTs in that Order. You will need to join on a key for the updates and you will need to do the rows that are in the staging table but not in the real table by outer joining the two. Allan Mitchell (Microsoft SQL Server MVP) MCSE,MCDBA www.SQLDTS.com www.konesans.com - for all your consultancy needs . |
![]() |
| Thread Tools | |
| Display Modes | |
| |