I have discovered through trial and error that the MATLAB engine function is not completely thread safe.

Does anyone know the rules?

Discovered through trial and error:

On Windows, the connection to MATLAB is via COM, so the COM Apartment threading rules apply. All calls must occur in the same thread, but multiple connections can occur in multiple threads as long as each connection is isolated.

From the answers below, it seems that this is not the case on UNIX, where calls can be made from multiple threads as long as the calls are made serially.