How to get around batch file processing limit
Posted
by Patrick Cuff
on Stack Overflow
See other posts from Stack Overflow
or by Patrick Cuff
Published on 2010-03-11T14:44:57Z
Indexed on
2010/03/11
21:09 UTC
Read the original article
Hit count: 583
batch-file
|Windows
I have a Windows batch file that processes all the files in a given directory. I have 206,783 files I need to process:
for %%f in (*.xml) do call :PROCESS %%f
goto :STOP
:PROCESS
:: do something with the file
program.exe %1 > %1.new
set /a COUNTER=%COUNTER%+1
goto :EOF
:STOP
@echo %COUNTER% files processed
When I run the batch file, the following output is written:
65535 files processed
As part of the processing, an output file is created for each file procesed, with a .new
extension. When I do a dir *.new
it reports 65,535 files exist.
So, it appears my command environment has a hard limit on the number of files it can recognize, and that limit is 64K - 1.
- Is there a way to extend the command environment to manage more than 64K - 1 files?
- If not, would a VBScript or JavaScript be able to process all 206,783 files?
I'm running on Windows 2003 server, Enterprise Edition, 32-bit.
UPDATE
It looks like the root cause of my issue was with the built-in Windows "extract" command for ZIP files.
The files I have to process were copied from another system via a ZIP file. My server doesn't have a ZIP utility installed, just the native Windows commands. I right-clicked on the ZIP file, and did an "Extract all...", which apparently just extracted the first 65,535 files.
I downloaded and installed 7-zip onto my server, unzipped all the files, and my batch script worked as intended.
© Stack Overflow or respective owner