How mod_cache working with "must-revalidate" and "max-age"?

Posted by Dmitriy Sosunov on Server Fault See other posts from Server Fault or by Dmitriy Sosunov
Published on 2012-11-04T10:50:59Z Indexed on 2012/11/04 11:05 UTC
Read the original article Hit count: 426

Quick question before I will explain my flow: ?an mod_cache perform revalidate with if-none-match only if max-age is expired in case if it configured in reverse proxy mode?

My goal is to reduce a number of revalidation requests to our the origin server.

For instance: The first request goes to the origin server and then mod_cache save a response in to the cache according to header cache-control: max-age. And only when max-age is expired then mod_cache will revalidate with if-none-match.

Currently, mod_cache revalidate each request, regardless that max-age is defined or not.

My configuration of Apache 2.4.3 (Windows), on linux I see the same behavior that I will show below.

    
   ServerName proxy.lo
   ProxyRequests Off    
   ProxyPreserveHost Off

   Header set Vary "Accept, Content-Type, Content-Encoding, Accept-Language"

   RequestHeader set X-Forwarded-Proto "http"

   # modify header for user agent's
   Header set Cache-Control "private, no-cache, no-store, no-transform"

   CacheQuickHandler off

   CacheDefaultExpire 300

   # the origin server do not provide last-modified
   CacheIgnoreNoLastMod On
   CacheIgnoreCacheControl On

   # the origin server define cache-control: private, no-store only for user agents
   # Therefore, I would like ignore those headers on the proxy server.
   CacheStorePrivate On
   CacheStoreNoStore On

   CacheEnable disk /
   CacheRoot "C:/Apache.Cache" 
   CacheDirLevels 5
   CacheDirLength 4 

   CacheMinExpire 15

   CacheDetailHeader on
   CacheHeader on

   KeepAlive Off

   ProxyPass / http://origin.lo/
   ProxyPassReverse / http://origin.lo/

Also, I have turned on debug log level to see how mod_cache handles a content for caching: I provided this to show that mod_proxy always decides that a content isn't fresh. Why?I provided this to show that mod_proxy always decide that a content isn't fresh. Why? max-age was provided (see below).

[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] cache_storage.c(624): [client 192.168.1.100:63741] AH00698: cache: Key for entity /testpage?(null) is http://proxy.lo/testpage?
[Sun Nov 04 11:58:42.899890 2012] [cache_disk:debug] [pid 6492:tid 1400] mod_cache_disk.c(569): [client 192.168.1.100:63741] AH00709: Recalled cached URL info header http://proxy.lo/testpage?
[Sun Nov 04 11:58:42.899890 2012] [cache_disk:debug] [pid 6492:tid 1400] mod_cache_disk.c(865): [client 192.168.1.100:63741] AH00720: Recalled headers for URL http://proxy.lo/testpage?
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] cache_storage.c(320): [client 192.168.1.100:63741] AH00695: Cached response for /testpage isn't fresh.  Adding/replacing conditional request headers.
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(414): [client 192.168.1.100:63741] AH00757: Adding CACHE_SAVE filter for /testpage
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(448): [client 192.168.1.100:63741] AH00759: Adding CACHE_REMOVE_URL filter for /testpage
[Sun Nov 04 11:58:42.899890 2012] [proxy:debug] [pid 6492:tid 1400] mod_proxy.c(1068): [client 192.168.1.100:63741] AH01143: Running scheme http handler (attempt 0)
[Sun Nov 04 11:58:42.899890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(1976): AH00942: HTTP: has acquired connection for (origin.lo)
[Sun Nov 04 11:58:42.899890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(2029): [client 192.168.1.100:63741] AH00944: connecting http://origin.lo/testpage to origin.lo:80
[Sun Nov 04 11:58:42.901890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(2151): [client 192.168.1.100:63741] AH00947: connected /testpage to origin.lo:80
[Sun Nov 04 11:58:42.901890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(2554): AH00962: HTTP: connection complete to 192.168.1.100:80 (origin.lo)
[Sun Nov 04 11:58:42.903890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(1991): AH00943: http: has released connection for (origin.lo)
[Sun Nov 04 11:58:42.903890 2012] [headers:debug] [pid 6492:tid 1400] mod_headers.c(800): AH01502: headers: ap_headers_output_filter()
[Sun Nov 04 11:58:42.903890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(1190): [client 192.168.1.100:63741] AH00769: cache: Caching url: /testpage
[Sun Nov 04 11:58:42.903890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(1196): [client 192.168.1.100:63741] AH00770: cache: Removing CACHE_REMOVE_URL filter.
[Sun Nov 04 11:58:42.904890 2012] [cache_disk:debug] [pid 6492:tid 1400] mod_cache_disk.c(1318): [client 192.168.1.100:63741] AH00737: commit_entity: Headers and body for URL http://proxy.lo/testpage? cached.

The first request to the origin server without mod_proxy to http://origin.lo/

GET http://origin.lo/testpage HTTP/1.1
Host: origin.lo
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

The first response from the origin without mod_proxy

HTTP/1.1 200 OK
Cache-Control: must-revalidate, proxy-revalidate, max-age=30
Content-Type: application/json; charset=utf-8
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sun, 04 Nov 2012 10:11:01 GMT
Content-Length: 1877

So, I assumed that revalidation must be occur only in 30 seconds after the success response. Is't right?

Let's check it:)

Within 30 sec, the Google Chrome didn't perform any requests to the origin server to revalidate a request and has return the response from local cache.

When max-age is expired, the Google Chrome perform a request to revalidate:

GET http://origin.lo/testpage HTTP/1.1
Host: origin.lo
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/xml
If-None-Match: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

and response:

HTTP/1.1 304 Not Modified
Cache-Control: must-revalidate, proxy-revalidate, max-age=30
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sun, 04 Nov 2012 10:16:20 GMT

As you can see, all works as expected. User agent revalidates request only when max-age is expired.

Let's now try perform the folling flow though mod_proxy (see configuration above).

The first request:

GET http://proxy.lo/testpage HTTP/1.1
Host: proxy.lo
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

and the response was:

HTTP/1.1 200 OK
Date: Sun, 04 Nov 2012 10:23:36 GMT
Server: Apache
Cache-Control: private, no-cache, no-store, no-transform
Content-Type: application/json; charset=utf-8
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Content-Length: 1932
Vary: Accept,Content-Type,Content-Encoding,Accept-Language
X-Cache: MISS from proxy.lo
X-Cache-Detail: "cache miss: attempting entity save" from proxy.lo
Connection: close

Ok, let's see to the disk cache and try to see how request and response was stored. (I cut binary data)

http://proxy.lo/testpage?
Cache-Control: private, no-cache, no-store, no-transform
Content-Type: application/json; charset=utf-8
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Date: Sun, 04 Nov 2012 10:27:15 GMT
Content-Length: 1932
Vary: Accept, Content-Type, Content-Encoding, Accept-Language

Host: proxy.lo
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
X-Forwarded-Proto: http
Cache-Control: max-age=300, must-revalidate
X-Forwarded-For: 192.168.1.100
X-Forwarded-Host: proxy.lo
X-Forwarded-Server: origin.lo

Ok, what we see? We see that the first request was performed with max-age=300 & must-revalidate

Ok, looks good, as for me, lets perform the next call:

GET http://proxy.lo/testpage HTTP/1.1
Host: proxy.lo
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3

and the second response from mod_proxy:

HTTP/1.1 200 OK
Date: Sun, 04 Nov 2012 10:31:58 GMT
Server: Apache
Cache-Control: private, no-cache, no-store, no-transform
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Content-Length: 1932
Vary: Accept,Content-Type,Content-Encoding,Accept-Language
X-Cache: REVALIDATE from proxy.lo
X-Cache-Detail: "conditional cache hit: entity refreshed" from proxy.lo
Connection: close
Content-Type: application/json; charset=utf-8

SO, MY QUESTION IS: WHY mod_proxy perform revalidation on each request regardless that max-age is defined?

N.B. Apache 2.4.3

Thanks, I would be grateful for any help.

© Server Fault or respective owner

Related posts about apache2

Related posts about reverse-proxy