A business partner has asked to set up a site-to-site VPN just so that a few servers can communicate with each other over HTTPS. I'm convinced this isn't necessary, or even desirable. To be fair it must be part of a wider policy, potentially even a legal requirement. However I'd like to convince them to simply offer an IP to us (and us only) and a port of their choosing for HTTPS.

Has anyone had a similar experience, or had to come up with a cast-iron argument against a VPN?

Allow me to expand a little - we have a web service that initiates a connection to the partner's corresponding service using an encrypted HTTP connection. The connection uses a client certificate to authenticate. The connection is firewalled so only our IPs can contact the service. So why is a VPN necessary?