The following script:

import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as mpl

mpl.rc('font', family='sans-serif')
mpl.rc('text', usetex=True)

fig = mpl.figure()
ax = fig.add_subplot(1,1,1)
ax.text(0.2,0.5,r"Math font: $451^\circ$")
ax.text(0.2,0.7,r"Normal font (except for degree symbol): 451$^\circ$")

fig.savefig('test.png')

is an attempt to use a sans-serif font in matplotlib with LaTeX. The issue is that the math font is still a serif font (as indicated by the axis numbers, and as demonstrated by the labels in the center). Is there a way to set the math font to also be sans-serif?