Downloading a large site with wget
- by Evan Gill
Hi,
I'm trying to mirror a very large site but wget never seems to finish properly. I am using the command:
wget -r -l inf -nc -w 0.5 {the-site}
I have downloaded a good portion of the site, but not the whole thing. The content does not change fast enough to bother using time-stamping.
After running overnight, this message appears:
File `{filename}.html' already there; not retrieving.
File `{filename}.html' already there; not retrieving.
File `{filename}.html' already there; not retrieving.
File `{filename}.html' already there; not retrieving.
Killed
does anyone know what is happening and how I can fix it?