dbTalk Databases Forums  

SSIS package - how to determine if a file is in use.

microsoft.public.sqlserver.datawarehouse microsoft.public.sqlserver.datawarehouse


Discuss SSIS package - how to determine if a file is in use. in the microsoft.public.sqlserver.datawarehouse forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Everett Music
 
Posts: n/a

Default SSIS package - how to determine if a file is in use. - 03-03-2010 , 04:44 PM






I have several SSIS packages that import data from files. I am having
trouble reliably determining whether a file is in use. I am working in
Visual Studio 2005. I am creating SSIS packages for SQL2005. I am importing
data from text files. These text files are written to network directories
from local, remote (WAN) and dialup sources. I need to be able to determine
if a file is still being written so that I don't try and import only part of
a file.

Reply With Quote
  #2  
Old   
Aeterna
 
Posts: n/a

Default RE: SSIS package - how to determine if a file is in use. - 03-04-2010 , 05:51 AM






Several options spring to mind.

One would be to monitor the modified datetime of the files and start the
SSIS packages when a certain amount of time has elapsed since the last
update.

Another would be to add a separate termination file for each of the files
which is produced, so that you only run a SSIS package on files where a
termination file is present.

Would either of these options work for you?


"Everett Music" wrote:

Quote:
I have several SSIS packages that import data from files. I am having
trouble reliably determining whether a file is in use. I am working in
Visual Studio 2005. I am creating SSIS packages for SQL2005. I am importing
data from text files. These text files are written to network directories
from local, remote (WAN) and dialup sources. I need to be able to determine
if a file is still being written so that I don't try and import only part of
a file.

Reply With Quote
  #3  
Old   
Everett Music
 
Posts: n/a

Default RE: SSIS package - how to determine if a file is in use. - 03-05-2010 , 09:32 AM



Unfortunately neither of these options will work for me.
As for monitoring the timestamp of the file and waiting for a specific
amount of time to pass before trying to read the file; Several of the files I
am interfacing with are uploaded via dialup and the timestamp is not
modified, so the file could have been created yesterday and uploaded today
with yesterdays timestamp.
As for creating a seperate termination file for each file; that would
require all processes that I receive files from to be re-written to generate
a termination file.
I currently test the availability of the file by trying to open the file
in 'non sharing' mode. This works most of the time, but I am still plagued
by some of the files not being locked while they are being written to. The
most grevious offender appears to be java.

"Aeterna" wrote:

Quote:
Several options spring to mind.

One would be to monitor the modified datetime of the files and start the
SSIS packages when a certain amount of time has elapsed since the last
update.

Another would be to add a separate termination file for each of the files
which is produced, so that you only run a SSIS package on files where a
termination file is present.

Would either of these options work for you?


"Everett Music" wrote:

I have several SSIS packages that import data from files. I am having
trouble reliably determining whether a file is in use. I am working in
Visual Studio 2005. I am creating SSIS packages for SQL2005. I am importing
data from text files. These text files are written to network directories
from local, remote (WAN) and dialup sources. I need to be able to determine
if a file is still being written so that I don't try and import only part of
a file.

Reply With Quote
  #4  
Old   
Todd C
 
Posts: n/a

Default RE: SSIS package - how to determine if a file is in use. - 03-24-2010 , 07:24 AM



I know there is no native 'Task' in SSIS to make this determination, but can
you use a Script task to poke at the file to see if it's fully baked?

In the script task you can leverage all of the System.IO library. I'm no
expert in this area and only guessing here, but it is where I would start.

HTH
--
Todd C
MCTS SQL Server 2005


"Everett Music" wrote:

Quote:
Unfortunately neither of these options will work for me.
As for monitoring the timestamp of the file and waiting for a specific
amount of time to pass before trying to read the file; Several of the files I
am interfacing with are uploaded via dialup and the timestamp is not
modified, so the file could have been created yesterday and uploaded today
with yesterdays timestamp.
As for creating a seperate termination file for each file; that would
require all processes that I receive files from to be re-written to generate
a termination file.
I currently test the availability of the file by trying to open the file
in 'non sharing' mode. This works most of the time, but I am still plagued
by some of the files not being locked while they are being written to. The
most grevious offender appears to be java.

"Aeterna" wrote:

Several options spring to mind.

One would be to monitor the modified datetime of the files and start the
SSIS packages when a certain amount of time has elapsed since the last
update.

Another would be to add a separate termination file for each of the files
which is produced, so that you only run a SSIS package on files where a
termination file is present.

Would either of these options work for you?


"Everett Music" wrote:

I have several SSIS packages that import data from files. I am having
trouble reliably determining whether a file is in use. I am working in
Visual Studio 2005. I am creating SSIS packages for SQL2005. I am importing
data from text files. These text files are written to network directories
from local, remote (WAN) and dialup sources. I need to be able to determine
if a file is still being written so that I don't try and import only part of
a file.

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.