We have two internet connections which are load balanced to share the load between the two. We set this up after one of the internet provider proved to be less than reliable but great speed and latency wise when it is working. We'd rather utilise both connections as much as possible rather than leave one idle until the other drops out.

We have a number of remote workers who occasionally need to connect via VPN from their laptops or iPads, we also have a small number of permanent LAN to LAN tunnels running from smaller branches.

Originally we only had one internet connection and used one of our static IP addresses for all VPN users. Now that we have two internet connections running all of the time I am trying to make sure that the VPN is available to our team regardless of which connection drops. So my solution is to create two A records for our domain name with a value of vpn. and the two static IP addresses from each peer.

Is this a sensible way of achieving this? Should I expect higher latency due to packets being lost if one peer fails and some packets still get routed to it anyway?

A brief mockup of the setup I have: