I've setup a Varnish cache mirror to sit in front of a file server, but it seems to be endlessly re-downloading data from my file server.
There's about 100GB of data in total, but so far Varnish has downloaded 800GB from my file server.
I'm using the default VCL file that comes with Varnish and the response headers for files served by the file server are similar to the following:
HTTP/1.1 200 OK
Cache-Control: max-age=290304000, public
Content-Type: image/jpeg
Expires: Wed, 29 Dec 2010 21:38:33 GMT
Server: Microsoft-IIS/7.0
E-Tag: "8b4723296ab697530768f18b1378b269"
Content-Disposition: inline; filename=image046.jpg;
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 23 Dec 2010 05:38:33 GMT
Content-Length: 100592
I'm starting varnishd with the following options:
varnish/sbin/varnishd -a 0.0.0.0:80 -f varnish/etc/varnish/default.vcl -s file,varnish/var/lib/varnish/varnish_storage.bin,100G