dbTalk Databases Forums  

Pain in... Loop through work flow with a recursive FSO loop

microsoft.public.sqlserver.dts microsoft.public.sqlserver.dts


Discuss Pain in... Loop through work flow with a recursive FSO loop in the microsoft.public.sqlserver.dts forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Todd
 
Posts: n/a

Default Pain in... Loop through work flow with a recursive FSO loop - 11-04-2004 , 06:20 PM






Ok, now that I lost you, here is my problem... I have a folder
structure like
A
1
1.xml
1.htm
1a.htm
2
2.xml
2.htm
2a.htm
3
3.xml
3.htm
3.html

And I need to loop through all the files and every xml file I need to
continue the workflow and insert values into the database. Looking at
the example I can do pretty much everything but deal with the
recursive loop

Set Params --> Start Loop --> XML Parsing --> SQL Task (call Sproc)
--> Loop

Here is the Start Loop step

Function Main()
dim pkg, conTextFile, stpEnterLoop, stpFinished

set pkg = DTSGlobalVariables.Parent
set stpEnterLoop = pkg.Steps("DTSStep_DTSActiveScriptTask_5")
set stpFinished = pkg.Steps("DTSStep_DTSActiveScriptTask_6")


if ShouldILoop = True then
msgbox DTSGlobalVariables("File").Value
stpEnterLoop.DisableStep = False
stpFinished.DisableStep = True
stpEnterLoop.ExecutionStatus = DTSStepExecStat_Waiting
ELSE
msgbox "none"
stpEnterLoop.DisableStep = True
stpFinished.DisableStep = False
stpFinished.ExecutionStatus = DTSStepExecStat_Waiting
End If

Main = DTSTaskExecResult_Success
End Function

Function ShouldILoop
dim fso, fil, fold, pkg, counter
dim fsoSubFold, strPath

strPath = DTSGlobalVariables("ImportFolder").Value

set pkg = DTSGlobalVariables.Parent


set fso = CreateObject("Scripting.FileSystemObject")

set fold = fso.GetFolder(strPath)

' counter = fold.files.count
' msgbox counter

' if counter >=1 THEN
for each fil in fold.files
If right(fil.path,3) = "xml" Then
msgbox "xml"
'DTSGlobalVariables("File").Value = fil.path
DTSGlobalVariables("File_Name").Value = fil.path
ShouldILoop = CBool(True)
Else
ShouldILoop = CBool(False)
End If
next

' Else
' ShouldILoop = CBool(False)
' End If
' Next
' else
' ShouldILoop = CBool(False)
' End If

For each fsoSubFold in fold.SubFolders
DTSGlobalVariables("ImportFolder").Value = fsoSubFold.Path
' ShouldILoop(fsoSubFold.Path)
ShouldILoop
Next
End Function


Need some help, burning too many cycles here

TIA

Reply With Quote
  #2  
Old   
Allan Mitchell
 
Posts: n/a

Default Re: Pain in... Loop through work flow with a recursive FSO loop - 11-05-2004 , 12:15 AM






So what does happen?
Are you looping through the folders also?
Errors?


--
--

Allan Mitchell MCSE,MCDBA, (Microsoft SQL Server MVP)
www.SQLDTS.com - The site for all your DTS needs.
www.konesans.com - Consultancy from the people who know


"Todd" <toddack (AT) hotmail (DOT) com> wrote

Quote:
Ok, now that I lost you, here is my problem... I have a folder
structure like
A
1
1.xml
1.htm
1a.htm
2
2.xml
2.htm
2a.htm
3
3.xml
3.htm
3.html

And I need to loop through all the files and every xml file I need to
continue the workflow and insert values into the database. Looking at
the example I can do pretty much everything but deal with the
recursive loop

Set Params --> Start Loop --> XML Parsing --> SQL Task (call Sproc)
--> Loop

Here is the Start Loop step

Function Main()
dim pkg, conTextFile, stpEnterLoop, stpFinished

set pkg = DTSGlobalVariables.Parent
set stpEnterLoop = pkg.Steps("DTSStep_DTSActiveScriptTask_5")
set stpFinished = pkg.Steps("DTSStep_DTSActiveScriptTask_6")


if ShouldILoop = True then
msgbox DTSGlobalVariables("File").Value
stpEnterLoop.DisableStep = False
stpFinished.DisableStep = True
stpEnterLoop.ExecutionStatus = DTSStepExecStat_Waiting
ELSE
msgbox "none"
stpEnterLoop.DisableStep = True
stpFinished.DisableStep = False
stpFinished.ExecutionStatus = DTSStepExecStat_Waiting
End If

Main = DTSTaskExecResult_Success
End Function

Function ShouldILoop
dim fso, fil, fold, pkg, counter
dim fsoSubFold, strPath

strPath = DTSGlobalVariables("ImportFolder").Value

set pkg = DTSGlobalVariables.Parent


set fso = CreateObject("Scripting.FileSystemObject")

set fold = fso.GetFolder(strPath)

' counter = fold.files.count
' msgbox counter

' if counter >=1 THEN
for each fil in fold.files
If right(fil.path,3) = "xml" Then
msgbox "xml"
'DTSGlobalVariables("File").Value = fil.path
DTSGlobalVariables("File_Name").Value = fil.path
ShouldILoop = CBool(True)
Else
ShouldILoop = CBool(False)
End If
next

' Else
' ShouldILoop = CBool(False)
' End If
' Next
' else
' ShouldILoop = CBool(False)
' End If

For each fsoSubFold in fold.SubFolders
DTSGlobalVariables("ImportFolder").Value = fsoSubFold.Path
' ShouldILoop(fsoSubFold.Path)
ShouldILoop
Next
End Function


Need some help, burning too many cycles here

TIA



Reply With Quote
  #3  
Old   
Toldd Y
 
Posts: n/a

Default Re: Pain in... Loop through work flow with a recursive FSO loop - 11-05-2004 , 05:20 AM



I have made some modifications

Function Main()
dim pkg, conTextFile, stpEnterLoop, stpFinished
dim fso, folder, subFolder

Set fso = createObject("scripting.FileSystemObject")
set folder = fso.GetFolder(DTSGlobalVariables("ImportFolder").V alue)


set pkg = DTSGlobalVariables.Parent
set stpEnterLoop = pkg.Steps("DTSStep_DTSActiveScriptTask_5")
set stpFinished = pkg.Steps("DTSStep_DTSActiveScriptTask_6")

If LoopFolders = True Then
if ShouldILoop = True then
stpEnterLoop.DisableStep = False
stpFinished.DisableStep = True
stpEnterLoop.ExecutionStatus = DTSStepExecStat_Waiting
ELSE
stpEnterLoop.DisableStep = True
stpFinished.DisableStep = False
stpFinished.ExecutionStatus = DTSStepExecStat_Waiting
End If
Else
stpEnterLoop.DisableStep = True
stpFinished.DisableStep = False
stpFinished.ExecutionStatus = DTSStepExecStat_Waiting
End IF

Main = DTSTaskExecResult_Success
End Function

Function LoopFolders
dim fso, folder, subFolder, folderCounter

Set fso = createObject("scripting.FileSystemObject")
set folder = fso.GetFolder(DTSGlobalVariables("ImportFolder").V alue)

folderCounter = folder.SubFolders.count

If folderCounter >=1 Then
For Each subFolder in Folder.SubFolders
DTSGlobalVariables("SubFolder").Value = subfolder.path
LoopFolders = CBool(True)
Next
Else
LoopFolders = CBool(False)
End If
End Function

Function ShouldILoop
dim fso, fil, fold, pkg, counter
dim fsoSubFold, strPath
dim stpEnterLoop, stpFinished

strPath = DTSGlobalVariables("SubFolder").Value
set pkg = DTSGlobalVariables.Parent

set fso = CreateObject("Scripting.FileSystemObject")
set fold = fso.GetFolder(strPath)

counter = fold.files.count

if counter >=1 THEN
for each fil in fold.files
If right(fil.path,3) = "xml" Then
DTSGlobalVariables("File_Name").Value = fil.path
ShouldILoop = CBool(True)
End IF
next
Else
ShouldILoop = CBool(False)
End If
End Function

I can loop through all the folders, subfolders, find if it is an XML doc
and even return true for ShouldILoop.

The problem appears to be that if ShouldILoop is true, it is not
entering the workflow and looping back. This task keeps looping until
the very last XML file is found and then it enters the workflow and
processes the XML file.

What I need to happen is, if an XML file is found, about 6000 XML files
out of 85,000 files, it should enter the workflow for each one. This is
not currently happening.

Thanks for the help

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

Reply With Quote
  #4  
Old   
Allan Mitchell
 
Posts: n/a

Default Re: Pain in... Loop through work flow with a recursive FSO loop - 11-06-2004 , 01:32 AM



What happens if you Exit Function after getting your return value

--
--

Allan Mitchell MCSE,MCDBA, (Microsoft SQL Server MVP)
www.SQLDTS.com - The site for all your DTS needs.
www.konesans.com - Consultancy from the people who know


"Toldd Y" <toddack (AT) hotmail (DOT) com> wrote

Quote:
I have made some modifications

Function Main()
dim pkg, conTextFile, stpEnterLoop, stpFinished
dim fso, folder, subFolder

Set fso = createObject("scripting.FileSystemObject")
set folder = fso.GetFolder(DTSGlobalVariables("ImportFolder").V alue)


set pkg = DTSGlobalVariables.Parent
set stpEnterLoop = pkg.Steps("DTSStep_DTSActiveScriptTask_5")
set stpFinished = pkg.Steps("DTSStep_DTSActiveScriptTask_6")

If LoopFolders = True Then
if ShouldILoop = True then
stpEnterLoop.DisableStep = False
stpFinished.DisableStep = True
stpEnterLoop.ExecutionStatus = DTSStepExecStat_Waiting
ELSE
stpEnterLoop.DisableStep = True
stpFinished.DisableStep = False
stpFinished.ExecutionStatus = DTSStepExecStat_Waiting
End If
Else
stpEnterLoop.DisableStep = True
stpFinished.DisableStep = False
stpFinished.ExecutionStatus = DTSStepExecStat_Waiting
End IF

Main = DTSTaskExecResult_Success
End Function

Function LoopFolders
dim fso, folder, subFolder, folderCounter

Set fso = createObject("scripting.FileSystemObject")
set folder = fso.GetFolder(DTSGlobalVariables("ImportFolder").V alue)

folderCounter = folder.SubFolders.count

If folderCounter >=1 Then
For Each subFolder in Folder.SubFolders
DTSGlobalVariables("SubFolder").Value = subfolder.path
LoopFolders = CBool(True)
Next
Else
LoopFolders = CBool(False)
End If
End Function

Function ShouldILoop
dim fso, fil, fold, pkg, counter
dim fsoSubFold, strPath
dim stpEnterLoop, stpFinished

strPath = DTSGlobalVariables("SubFolder").Value
set pkg = DTSGlobalVariables.Parent

set fso = CreateObject("Scripting.FileSystemObject")
set fold = fso.GetFolder(strPath)

counter = fold.files.count

if counter >=1 THEN
for each fil in fold.files
If right(fil.path,3) = "xml" Then
DTSGlobalVariables("File_Name").Value = fil.path
ShouldILoop = CBool(True)
End IF
next
Else
ShouldILoop = CBool(False)
End If
End Function

I can loop through all the folders, subfolders, find if it is an XML doc
and even return true for ShouldILoop.

The problem appears to be that if ShouldILoop is true, it is not
entering the workflow and looping back. This task keeps looping until
the very last XML file is found and then it enters the workflow and
processes the XML file.

What I need to happen is, if an XML file is found, about 6000 XML files
out of 85,000 files, it should enter the workflow for each one. This is
not currently happening.

Thanks for the help

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.