Squid not caching files (Randomly)
- by Heinrich
I want to use an intercepting squid server to cache specific large zip files that users in my network download frequently.
I have configured squid on a gateway machine and caching is working for "static" zip files that are served from an Apache web server outside our network.
The files that I want to have cached by squid are zip files 100MB which are served from a heroku-hosted Rails application. I set an ETag header (SHA hash of the zip file on the server) and Cache-Control: public header. However, these files are not cached by squid. This, for example, is a request that is not cached:
$ curl --no-keepalive -v -o test.zip --header "X-Access-Key: 20767ed397afdea90601fda4513ceb042fe6ab4e51578da63d3bc9b024ed538a" --header "X-Customer: 5" "http://MY_APP.herokuapp.com/api/device/v1/media/download?version=latest"
* Adding handle: conn: 0x7ffd4a804400
* Adding handle: send: 0
* Adding handle: recv: 0
...
> GET /api/device/v1/media/download?version=latest HTTP/1.1
> User-Agent: curl/7.30.0
> Host: MY_APP.herokuapp.com
> Accept: */*
> X-Access-Key: 20767ed397afdea90601fda4513ceb042fe6ab4e51578da63d3bc9b024ed538a
> X-Customer: 5
>
0 0 0 0 0 0 0 0 --:--:-- 0:00:09 --:--:-- 0< HTTP/1.1 200 OK
* Server Cowboy is not blacklisted
< Server: Cowboy
< Date: Mon, 18 Aug 2014 14:13:27 GMT
< Status: 200 OK
< X-Frame-Options: SAMEORIGIN
< X-Xss-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< ETag: "95e888938c0d539b8dd74139beace67f"
< Content-Disposition: attachment; filename="e7cce850ae728b81fe3f315d21a560af.zip"
< Content-Transfer-Encoding: binary
< Content-Length: 125727431
< Content-Type: application/zip
< Cache-Control: public
< X-Request-Id: 7ce6edb0-013a-4003-a331-94d2b8fae8ad
< X-Runtime: 1.244251
< X-Cache: MISS from AAA.fritz.box
< Via: 1.1 vegur, 1.1 AAA.fritz.box (squid/3.3.11)
< Connection: keep-alive
In the logs squid is reporting a TCP_MISS.
This is the relevant excerpt from my squid file:
# Squid normally listens to port 3128
http_port 3128
http_port 3129 intercept
# Uncomment and adjust the following to add a disk cache directory.
maximum_object_size 1000 MB
maximum_object_size_in_memory 1000 MB
cache_dir ufs /usr/local/var/cache/squid 10000 16 256
cache_mem 2000 MB
# Leave coredumps in the first cache dir
coredump_dir /usr/local/var/cache/squid
cache_store_log daemon:/usr/local/var/logs/cache_store.log
#refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern -i .(zip) 525600 100% 525600 override-expire ignore-no-cache ignore-no-store
refresh_pattern . 0 20% 4320
## DNS Configuration
dns_nameservers 8.8.8.8 8.8.4.4
After trying around for some time I realized that squid is sometimes deciding that my file is cacheable, sometimes not, depending on whether and when I enable/disable the dns_nameservers directive.
What could be wrong here?
