dbTalk Databases Forums  

import subset of text files in a directory

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss import subset of text files in a directory in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Klein-4
 
Posts: n/a

Default import subset of text files in a directory - 07-04-2008 , 08:08 PM







I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K

Reply With Quote
  #2  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM






On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #3  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #4  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #5  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #6  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #7  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #8  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #9  
Old   
matteus
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 04:38 AM



On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:
Quote:
I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K
Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:
Quote:
1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.
1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon...



Reply With Quote
  #10  
Old   
Klein-4
 
Posts: n/a

Default Re: import subset of text files in a directory - 07-06-2008 , 08:44 AM



On Jul 6, 4:38*am, matteus <matteog... (AT) gmail (DOT) com> wrote:
Quote:
On Jul 5, 3:08 am, Klein-4 <soreeyeh... (AT) gmail (DOT) com> wrote:





I've just begun working with SQL Server (2000) recently and have some
familiarity with setting up DTS packages. *I also have a lot of
experience with VBA, so ActiveX scripting should come easily to me.
Basically, I have a folder, let's say "V:\MyFolder\", and I would like
to

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table,
5) go to step 1.

The files have the form 's005%.txt', where % is a wildcard and I would
like to loop this procedure until all files of this form in the given
folder are exhausted.

I'm looking for some detailed assistance here, if possible, and it
would be greatly appreciated. *I've been waiting for a month to get
permission from the DBAs for BULK INSERT and I believe I may be
waiting until sometime after hell freezes over... ;-)

K

Hi Klein,
for what i know, DTS don't allow you to loop through group of tasks,
unless you use activex scripting. in this case you must script it
entirely. In SQL Server 2005 ther're some new objects that let you do
this quite easily.
Depending on your specific needs, here are some workarounds:
- execute the package every... minute and importing one file (eg,
alphabetical order) every execution, taking care of deleting that file
at the end of the dts.
- at the beginning of the package, merge all txt files into one
(assuming they've all the same structure) and then import it, with
activex shouldn't be difficult for you.

the one-file-package shuld appear like this:

1) truncate a holding table in SQL Server,
2) import a file in the given folder into this holding table,
3) run a couple of queries against the table, and then
4) append the rows in the holding table to a master table
5) go to step 1.

1) execute sql task
2) file connetion to your .txt
grey arrow connectin (transform data task in which you choose field,
field names, ecc)
3) oledb connection to your hodling table (by 'holding' you mean a
sort of temp or staging table, don't you?)
4) execute sql task - insert into master select * from holding

Bye,
M.

PS: Here it's VERY hot and wet and i don't believe hell is going to
freeze over soon... - Hide quoted text -

- Show quoted text -
Thanks much. I'll try this out. :-)

K



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.