How to get around batch file processing limit

Posted by Patrick Cuff on Stack Overflow See other posts from Stack Overflow or by Patrick Cuff
Published on 2010-03-11T14:44:57Z Indexed on 2010/03/11 21:09 UTC
Read the original article Hit count: 583

Filed under:
|

I have a Windows batch file that processes all the files in a given directory. I have 206,783 files I need to process:

for %%f in (*.xml) do call :PROCESS %%f
goto :STOP

:PROCESS
:: do something with the file
program.exe %1 > %1.new
set /a COUNTER=%COUNTER%+1
goto :EOF

:STOP
@echo %COUNTER% files processed

When I run the batch file, the following output is written:

65535 files processed

As part of the processing, an output file is created for each file procesed, with a .new extension. When I do a dir *.new it reports 65,535 files exist.

So, it appears my command environment has a hard limit on the number of files it can recognize, and that limit is 64K - 1.

  1. Is there a way to extend the command environment to manage more than 64K - 1 files?
  2. If not, would a VBScript or JavaScript be able to process all 206,783 files?

I'm running on Windows 2003 server, Enterprise Edition, 32-bit.


UPDATE

It looks like the root cause of my issue was with the built-in Windows "extract" command for ZIP files.

The files I have to process were copied from another system via a ZIP file. My server doesn't have a ZIP utility installed, just the native Windows commands. I right-clicked on the ZIP file, and did an "Extract all...", which apparently just extracted the first 65,535 files.

I downloaded and installed 7-zip onto my server, unzipped all the files, and my batch script worked as intended.

© Stack Overflow or respective owner

Related posts about batch-file

Related posts about Windows