I want to install matplotlib on OS X. If possible, using homebrew.

I installed Python 2.7.1 using brew install python, I modified my path to use it
I installed pip using brew install pip
I installed numpy 1.5.1 using pip install numpy
I installed scipy 0.8.0 using pip install scipy

This is where it gets hairy. pip install matplotlib will fetch the wrong version of matplotlib, which is incompatible with the recent version of numpy.

The solution is to fetch the correct version of matplotlib manually:

pip install -f http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0.1/matplotlib-1.0.1.tar.gz matplotlib

But, that version fails to compile since it can't find the freetype headers:

In file included from src/ft2font.cpp:1:
src/ft2font.h:14:22: error: ft2build.h: No such file or directory

These headers are actually installed in /usr/X11/include as part of the X11 developer tools.

So, how can I make matplotlib use these headers?