Bulk import data - advice sought on options -
08-15-2006
, 02:29 PM
I have a classic ASP application that uses Access as the db. I'm in the
process of attempting to migrate it to SQL Server, and am learning about SQL
Server as I go.
One part of the app imports data to the db. This is not the most used part
of the app - possibly once every month or two. The way I am doing this at
the moment is probably very inefficient, and I am looking at options to
improve the process. Here's a description of the current process:
1. Prepare data to be imported on templated Excel spreadsheets, with column
heads matching the field names in Access.
2. Import spreadsheets into temp db. Upload it to Web server.
3. Script fetches data to be imported into a recordset, then an array.
4. Disconnects from the temp db.
5. Script then looks at the import data, row by row, for matches in the
existing db on company name and first line of address.
6. If no match is found, the company record is added to the existing data as
a new record, then associated contacts are added as new records.
7. If a match is found on the company record, script checks associated
Contact data for matches on initial and surname in the exisitng data.
8. If a match is found, row is skipped. Otherwise contact data is added to
the main db.
The new db will be SQL Server 2005 Express (to start with), and I can see
that it may be possible to create a stored proc to handle the import task
more efficiently. I have also read a bit about DTS, which I understand is
now called Integration Services in 2005, but is not supported as such in the
Express version. But BOL says that 2005 Express provides "Legacy support
for DTS packages". I have the full version of SQL Server 2000, so I can
have a go at writing a DTS package.
I am happy to get my hands dirty - do the research and create the code, but
I could do with some advice on which direction I should go: Stored proc?
DTS? Something else?
Thanks
Mike |