dbTalk Databases Forums  

Importing a large flat file through DTS

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Importing a large flat file through DTS in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
dverma
 
Posts: n/a

Default Importing a large flat file through DTS - 05-27-2005 , 04:10 PM






I have a large text file which has columns name in first row (column
separated by "|" char). The file contains 125,000 records when I use DTS to
import this file the DTS hangs after importing 61000 rows (as per progress
bar showing). Is there is any limitation on maximum number of rows the "text
source" in DTS can access.
To prove that file has no problem I created a ODBC connection to file and
imported successfully all the rows. What's wrong with "Text Source" type
connection in DTS? Is there any way to set maximum number of rows to be
imported? I also tried Import\Export wizard to import file but the wizard is
also using the "Text Source" connection in DTS as a result hangs after 61000
records as per progress bar.

--
dpk verma

Reply With Quote
  #2  
Old   
Mike Gercevich via SQLMonster.com
 
Posts: n/a

Default Re: Importing a large flat file through DTS - 05-28-2005 , 01:03 AM






You are most likely inserting in bulk as one large transaction. Your log
file is full. On the Options tab of the transform, insert on every 1000 to
10000 records. I have performed this type of operation on table copies
between server with over 3 million records.

-Mike Gercevich

--
Message posted via http://www.sqlmonster.com

Reply With Quote
  #3  
Old   
dverma
 
Posts: n/a

Default Re: Importing a large flat file through DTS - 05-31-2005 , 08:46 AM



Mike,
The text file size is 44 MB and I have increased the Transaction Log
file to 100MB but still does not work.
--
dpk verma


"Mike Gercevich via SQLMonster.com" wrote:

Quote:
You are most likely inserting in bulk as one large transaction. Your log
file is full. On the Options tab of the transform, insert on every 1000 to
10000 records. I have performed this type of operation on table copies
between server with over 3 million records.

-Mike Gercevich

--
Message posted via http://www.sqlmonster.com


Reply With Quote
  #4  
Old   
dverma
 
Posts: n/a

Default Re: Importing a large flat file through DTS - 05-31-2005 , 09:01 AM



Thanks Mike, I did change the insert on every 10,000 records but the DTS
still hangs at magic number 61000. Any other suggestion to fix this. If this
is log file issue then why it is working with Microsoft Text-Treiber driver
(sorry, I called this ODBC driver while posting the problem) and not with
connection type of "DTSFlatFile" ?
--
dpk verma


"Mike Gercevich via SQLMonster.com" wrote:

Quote:
You are most likely inserting in bulk as one large transaction. Your log
file is full. On the Options tab of the transform, insert on every 1000 to
10000 records. I have performed this type of operation on table copies
between server with over 3 million records.

-Mike Gercevich

--
Message posted via http://www.sqlmonster.com


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.