Hi Bill, I suppose that you are using a "robust" Proxy Adapter; otherwise, the failure of the Remote Data Adapter would have caused the Server to fail at once.
As far as I know, this error message can be issued by the .NET runtime barely if the targeted port is not open for listening.
As long as a Remote Adapter is connected, the connection ports are indeed not open by the Proxy Adapter;
only when the Remote Adapter disconnects will the "robust" Proxy Adapter open the ports for listening again.
So, what you see is not supposed to happen, but if something goes wrong in the recovery process, then it may.

What do you see in the Server log?
With factory setting, in a normal case, you should see log related with the previous Data Adapter failure, then the reopening of the listening port and eventually the new connection.