I have managed to replicate the case, but only in very unusual conditions.

According to the previous thread dump (I would like to avoid delving into the memory dump in the first place) the client has just issued a rebind request and it is waiting for the answer, which is not coming.
This can be replicated by blocking the Server, but not by closing the Server (in the latter case, HttpWebRequest.BeginGetResponse would invoke its listener).
Do you notice any issues in the Server or in the network, such that a request reaches the Server but gets no answer?

As long as the client is waiting for a connection to be established, it is correct that it keeps the process alive, as the underlying session is still to be considered alive, until the reconnection timeout expires.
By the way, the reconnection timeout is the ReconnectionTimeoutMillis property of ConnectionInfo, but if a session is in smart polling, the whole PollingIdleMillis is added.
After the timeout expires, the connection attempt should be abandoned.

However, I acknowledge that there is a bug.
In fact, after the timeout expires, the underlying connection attempt is not interrupted;
hence the thread (which is still not a background thread) keeps the process alive until something happens on the network and this may last for unboundedly long.
We'll try to fix that behavior for the next release.
Sorry for the inconvenience.

For now, the application should cooperate.
When the reconnection timeout expires, the application receives an invocation of OnFailure on the IConnectionListener.
There, it should evaluate if the process must terminate and, in that case, force the termination through Environment.Exit.
On the other hand, if the process issues CloseConnection while the blocked reconnection attempt is in place, OnFailure will not be invoked and the session will close normally;
however, the case is also affected by the bug, hence, the process will still be kept alive.
So, also after invoking CloseConnection, the application should evaluate if the process must terminate and, in that case, force the termination through Environment.Exit.
Alternatively, you could handle both cases by acting in the OnClose callback, which is always invoked.