Wiki Home

Recursive Directory Processing


Namespace: Wiki

Below is a method of performing file processing throughout an entire directory tree. The function, RECURSE(), is called with the full path of a directory. RECURSE() will then read all files and directories in the path. A function can be called to process files that it finds. Plus, the function calls itself to process any sub-directories.

This method is extremely handy for me for global directory processing, particularly with web sites. For example, conditional processing can be run on all files to add, remove, or change content.

Enjoy!
Michael Reynolds

Very nice recursive routine, Michael, I really enjoyed it! (in my fast(!) PII 350 it processed 685 dirs, 11,277 files in 3 secs. Very usefull. I retired my old routine! Thanks) -- Fernando Alvares

Thanks! I'm glad it worked so well for you. It's proven to be an important tool for me. I was asked to add meta tags and update copyright notices in thousands of web pages of a corporate web site. This routine did it all in seconds! -- Michael Reynolds

This is an awesome routine and I use it. However, for directories that have embedded spaces: "\program files\microsoft visual foxpro", you need to put " " around for use with the CHDIR command:

CHDIR "&pcDir"
CHDIR "&lcCurDrive.&lcCurDir."

I just ran into this today.

Peter Diotte

[2003.05.15] Nice one!...I've just used it to scan our development servers looking for DBC's, when it finds one, it runs GenDBC to save the DBC structure.
Very useful
Cheers

John Taylor

Nice UDF. To make this more re-usable, you might consider wrapping as a method of a class. Then add a hook at the "Process File (lcFile) here..." location to call another method like this.Process( m.lcFile ) for example. Then, you could:
1. Subclass and just implement different Process() methods for certain tasks.
2. Extend even more with swappable strategy objects or chains thereof. See Hook Operation.
3. Employ BINDEVENT() in the Process() method to allow loosely coupled binding to all sorts of delegates with BINDEVENT()
- ?lc

See also: Recursion In VFP
and a similar recursive routine by Ed Rauh in Windows Scripting Host
lcCurDrive = JUSTDRIVE(FULLPATH(CURDIR()))
lcCurDir = CURDIR()

Recurse('n:\path')
CHDIR &lcCurDrive.&lcCurDir.

****

FUNCTION Recurse
LPARAMETERS pcDir
LOCAL lnPtr, lnFileCount, laFileList, lcDir, lcFile

CHDIR (pcDir)
? 'Dir -> ' + FULLPATH(CURDIR())

DIMENSION laFileList[1]

*--- Read the chosen directory.
lnFileCount = ADIR(laFileList, '*.*', 'D')
FOR lnPtr = 1 TO lnFileCount

  IF 'D' $ laFileList[lnPtr, 5]

    *--- Get directory name.
    lcDir = laFileList[lnPtr, 1]

    *--- Ignore current and parent directory pointers.
    IF lcDir != '.' AND lcDir != '..'
      *--- Call this routine again.
      Recurse(lcDir)
    ENDIF

  ELSE

    *--- Get the file name.
    lcFile = LOWER(laFileList[lnPtr, 1])
    IF [Condition is True]
      Process File (lcFile) here...
    ENDIF

  ENDIF

ENDFOR

*--- Move back to parent directory.
CHDIR ..


Nice Routine, Micheal! Here's a form of it as a class (applying some suggestions made above):
* Test Source Code
x=Create('TestMyRecurse')
set textmerge to TestRecurse NOSHOW
x.Recurse("C:\")
set textmerge to
modi comm TestRecurse.txt


DEFINE CLASS TestMyRecurse AS Recurse
PROCEDURE PROCESS( pcFile )
  \<>
ENDPROC
ENDDEFINE

DEFINE CLASS Recurse AS Session

PROCEDURE Process( pcFile )
  * Abstract Method to be overridden in subclasses that actually do something
ENDPROC

FUNCTION Recurse
LPARAMETERS pcDir
LOCAL lnPtr, lnFileCount, laFileList, lcDir, lcFile, lcCurDir
lcCurDir = FULLPATH(CURDIR())
CHDIR (pcDir)
*? 'Dir -> ' + FULLPATH(CURDIR())

DIMENSION laFileList[1]

*--- Read the chosen directory.
lnFileCount = ADIR(laFileList, '*.*', 'D')
FOR lnPtr = 1 TO lnFileCount

  IF 'D' $ laFileList[lnPtr, 5]
    *--- Get directory name.
    lcDir = laFileList[lnPtr, 1]

    *--- Ignore current and parent directory pointers.
    IF lcDir != '.' AND lcDir != '..'
      *--- Call this routine again.
      THIS.Recurse(addbs(pcDir)+lcDir)
    ENDIF

  ELSE
    *--- Get the Long file name and process it:
    THIS.Process( THIS.Fn2lfn(addbs(pcDir)+laFileList[lnPtr, 1]) )
  ENDIF
ENDFOR

*--- Move back to parent directory.
CHDIR (lcCurDir)
RETURN
ENDFUNC

FUNCTION Fn2Lfn
LPARAMETERS pcFileName
* FN2LFN = "FileName" To "LongFileName"
*
*    AUTHOR: Wgcs.. Function derived from sample code posted by
*             AirCon (IS/IT--Manageme)  on TekTips.com on May 12, 2003
*      DATE: added LASv10.40
*   PURPOSE: converts an 8.3 or an all-uppercase filename into
*             it's proper-case long filename.
* REFERENCE: Used by aasApplication.GetFile
*     NOTES: If the file isn't found, then the original passed
*            filename is returned.

Declare Long FindFirstFile in Kernel32 as fnFindFirst ;
   String cFileName, String @WIN32_DATA
Declare FindClose in Kernel32 as fnFindClose Long hFile
LOCAL lcFile, lcBuffer, lcPath
lcPath = addbs(JustPath(pcFileName))
lcFile = JustFName(pcFileName)
lcBuffer = replicate(chr(0), 318)
hFile = fnFindFirst(pcFileName, @lcBuffer)
If (hFile > 0)
   lcFile = substr(lcBuffer, 45)
   lcFile = left(lcFile, at(chr(0), lcFile)-1)
   fnFindClose(hFile)
endif
Clear Dlls fnFindFirst, fnFindClose
RETURN lcPath+lcFile
ENDFUNC

ENDDEFINE

Very Helpful! I converted it into a class. Also, check out the new arguments to the ADIR function. You no longer need the fn2lfn library.
Makes it much faster. I also added code to PROCESS to create and populate a cursor with the ADIR info as it goes along. Then I can process the cursor afterward any way I want to.
Thanks! Saved me quite a bit of noodling. Recursive stuff is always a bit mind bending. Ray Kirk
You're assuming that I'm not still using VFP6.... ;) I still need fn2lfn. - ?wgcs

Ray - can you post the code you put in PROCESS here? I understand that it might be VFP7-specific if you're using the new ADIR() columns, but it would still be useful.

Also, I would change one more thing to make the class even more extensible. I would replace the line
THIS.Recurse(addbs(pcDir)+lcDir)
with
THIS.ProcessDir(lcDir, pcDir)
and then add the ProcessDir shell method with some default code:
PROCEDURE ProcessDir( pcDir, pcParentDir )
  * Override this method as desired.
  THIS.Recurse(addbs(pcParentDir)+pcDir)
ENDPROC
This allows the developer to perform other work on directories, and handle empty directories. It also allows them to put a DODEFAULT() call in their code to continue recursing downward. -- Zahid Ali

In appreciation for the time you guys saved me today, here's a version of Michael Reynolds's code to sum the filesizes within a directory, recursively. Feel free to improve it, this is a wiki after all. I had to write this because the dos DIR command was giving me Access is Denied messages on UNC paths that I had access to.

PROCEDURE RecursiveFilesizeSum
LPARAMETERS pcDir
LOCAL lnPtr, lnFileCount, laFileList, lcDir, lcFile, workingsum
*?"			", pcdir
	
	workingsum = 0
	DIMENSION laFileList[1]

	lnFileCount = ADIR(laFileList, ADDBS(pcDir) + "*.*", 'D')	&& Read the chosen directory.
	FOR lnPtr = 1 TO lnFileCount
	  IF 'D' $ laFileList[lnPtr, 5]
	    lcDir = laFileList[lnPtr, 1]  && Get directory name.

	    IF lcDir != '.' AND lcDir != '..'		&& Ignore current and parent directory pointers.
	      workingsum = workingsum + RecursiveFilesizeSum(ADDBS(pcDir) + lcDir)		&& Call this routine again.
	    ENDIF

	  ELSE
	    workingsum = workingsum + laFileList[lnPtr ,2]
	  ENDIF

	ENDFOR
	*? pcDir, workingsum
RETURN workingsum







Check the following sample code. Starting from a given point, say "C:\Program Files", it creates a cursor with a tree of all subfolders, which includes totals for file count, file size, and last write time:

Using Find First FileOffsite link to http://www.news2news.com/vfp/?function=378
and Find Next FileOffsite link to http://www.news2news.com/vfp/?function=379
functions to build a tree of subdirectories for a given path
at http://www.news2news.com/vfp/?example=236&function=378

Anatoliy Mogylevets

See Also Recursion In VFP
Category VFP Tips And Tricks Category Code Samples
( Topic last updated: 2015.09.16 11:18:55 AM )