To avoid having to parse X-Forwarded-For in Varnish, I'm trying to just set a header on the SSL terminator (currently Apache) that stores the direct client IP in a header.
On our development machine, this works:
RequestHeader set X-Foo %{REMOTE_ADDR}e
However, in staging it doesn't. Specifically, the header is empty, as illustrated by both varnishlog:
13 TxHeader b X-Foo: (null)
(On the development machine, this shows the IP address as expected.)
Similarly, logging REMOTE_ADDR shows that it only appears to be populated on the dev machine:
# Config
LogFormat "%{X-Forwarded-For}i %{REMOTE_ADDR}e" combined
CustomLog "/var/log/httpd/access_log" combined
# Log file, staging
<my ip> -
# Log file, development
<my ip> <my ip>
Since the dev machine is, well, a dev machine, it is different in a number of ways; however, I can't track down which difference is causing this. The versions of Apache are the same (2.2.22), and I don't see anything relevant in any of the standard config files or /etc/sysconfig/httpd. And the rest of the system is reasonably similar, since they're built off the same CentOS 5 base image.
I can't even tell from the Apache documentation whether REMOTE_ADDR is expected to exist or not as an environment variable, but it clearly works on one machine, whether by fluke or design, and the inconsistency is driving me mad.