dbTalk Databases Forums  

DTS import - all data not imported

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss DTS import - all data not imported in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
William Patterson
 
Posts: n/a

Default DTS import - all data not imported - 12-02-2004 , 02:18 PM






I am working with some financial data that has many, many observations, but
failrly small records. I an creating a comma separated variable (CSV) file
using a VB.NET program and then I use DTS Import to import the CSV file into
SQL Server 2000 via a file DSN.

This all works fine for the smaller data sets, but larger datasets with 24
million or 37 million records do not import properly. The DTS import runs,
but it stops, with no error messages, after about 18 million records.

Is there something I can do to track down this problem or is there a
workaround? The system is running on a fully patched Windows 2003 server
system and SQL is fully patched

Thanks,

Bill Patterson
UW-Madison School of Business



Reply With Quote
  #2  
Old   
Allan Mitchell
 
Posts: n/a

Default Re: DTS import - all data not imported - 12-02-2004 , 02:31 PM






Why are you using a File DSN?

Why not use the File (Source) adapter in DTS itself?

Could it be that the connection is getting lost?
Your log blowing?
How are you loading? I would use the ExecuteSQL task if using DTS.

If this is the only thing for which you want to use DTS then consider the
BULK INSERT TSQL statement



--
--

Allan Mitchell MCSE,MCDBA, (Microsoft SQL Server MVP)
www.SQLDTS.com - The site for all your DTS needs.
www.konesans.com - Consultancy from the people who know


"William Patterson" <wpatterson (AT) bus (DOT) wisc.edu> wrote

Quote:
I am working with some financial data that has many, many observations, but
failrly small records. I an creating a comma separated variable (CSV)
file
using a VB.NET program and then I use DTS Import to import the CSV file
into
SQL Server 2000 via a file DSN.

This all works fine for the smaller data sets, but larger datasets with 24
million or 37 million records do not import properly. The DTS import
runs,
but it stops, with no error messages, after about 18 million records.

Is there something I can do to track down this problem or is there a
workaround? The system is running on a fully patched Windows 2003 server
system and SQL is fully patched

Thanks,

Bill Patterson
UW-Madison School of Business





Reply With Quote
  #3  
Old   
William Patterson
 
Posts: n/a

Default Re: DTS import - all data not imported - 12-02-2004 , 02:59 PM



I should add that I am using the File (source) adapter in DTS, not a file
DSN.


"William Patterson" <wpatterson (AT) bus (DOT) wisc.edu> wrote

Quote:
I am working with some financial data that has many, many observations,
but
failrly small records. I an creating a comma separated variable (CSV)
file
using a VB.NET program and then I use DTS Import to import the CSV file
into
SQL Server 2000 via a file DSN.

This all works fine for the smaller data sets, but larger datasets with 24
million or 37 million records do not import properly. The DTS import
runs,
but it stops, with no error messages, after about 18 million records.

Is there something I can do to track down this problem or is there a
workaround? The system is running on a fully patched Windows 2003 server
system and SQL is fully patched

Thanks,

Bill Patterson
UW-Madison School of Business





Reply With Quote
  #4  
Old   
Ed
 
Posts: n/a

Default Re: DTS import - all data not imported - 12-02-2004 , 04:03 PM



Are you using Transformation Task?
If so, you may want to check on "Always Commit the Final Batch" and then
llok at the next record to see what's wrong
Ed

"William Patterson" wrote:

Quote:
I should add that I am using the File (source) adapter in DTS, not a file
DSN.


"William Patterson" <wpatterson (AT) bus (DOT) wisc.edu> wrote in message
news:udbjowK2EHA.3500 (AT) TK2MSFTNGP09 (DOT) phx.gbl...
I am working with some financial data that has many, many observations,
but
failrly small records. I an creating a comma separated variable (CSV)
file
using a VB.NET program and then I use DTS Import to import the CSV file
into
SQL Server 2000 via a file DSN.

This all works fine for the smaller data sets, but larger datasets with 24
million or 37 million records do not import properly. The DTS import
runs,
but it stops, with no error messages, after about 18 million records.

Is there something I can do to track down this problem or is there a
workaround? The system is running on a fully patched Windows 2003 server
system and SQL is fully patched

Thanks,

Bill Patterson
UW-Madison School of Business






Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.