I have a web service endpoint that I am building where people will post an xml file to, and it will really get pounded with over 1K requests per second.

Now they are sending in these xml files via http post, but a good majority of them will be rate limited.

The problem is, the rate limiting will be done by the web application by looking up the source_id in the xml, and if it is over x requests per minute, it will not be processed further.

I was wondering if I could do rate limit checking earlier in the processing somehow and thus save the 50K file going threw the pipeline to my web servers and eating up resources.

Could a load balancer make a call out to verify rate usage somehow?

If this is possible, I could maybe put the source_id in a host header so even the XML file doesn't have to be parsed and loaded into memory.

Is it possible to just look at host headers and not load up the entire 50K xml file into memory?

I really appreciate your insights as this takes more knowledge of the entire tcp/ip stack etc.