-
Update: The number of connections are roughly equivalent to the number of sessions. If I take the server offline for a short period and restart it, the number of connections looks roughly double the number of sessions.....but appear to fall slightly as time goes on.
No scientific proof - but a speculative observation: The longer I leave the server offline the more connections I get for the same number of sessions. Also the more time I take the server offline, the more connections I get:
On startup: 32 sessions and 32 connections
Stop and restart: 32 sessions and approx 50 connections (falling to about 40)
Stop and restart: 32 sessions and approx 90 connections (falling to about 60)
and so on
So if I have 1000 sessions, and the server goes offline for a long period of time and then gets restarted, I think I am going to see some big numbers in the connections statistics compared to the sessions.
I think this only happens with clients that had a good connection, then lost it. They go into an endless connection/fail/retry cycle - and when the server comes on they seem to have more connections than they need.
This may be completely normal and expected of course.
-
I have provided the logs from Saturday and Sunday to support@lightstreamer.com using https://www.wetransfer.com/#
-
Hi Kevin,
I confirm we have received the logs, thank you. Now , we will check them.
-
Update:
Some users have now been given access through the firewall.
They currently have 152 sessions and 411 connections (maximum 751). Some errors and warnings in the logs too. We will continue to collect the log information for the time being.
-
1 Attachment(s)
While the server is offline, I observe more and more iframe elements being created in my browsers DOM. See screenshot below, Is this normal?
Attachment 187
-
Hi Kevin,
That's a known bug that was fixed in the latest version.
The bug affects the 6.1.4 and 6.2.2 versions.
The bug is triggered by the client being unable to reach the server
http://www.lightstreamer.com/distros...TML#web_client
We considered the bug while investigating the issue, but it does not seem likely it to be related to your issue, as the browser would not permit so many open sockets to the server in the first place (the number of open sockets is way too high with respect of the active sessions).
Anyhow, you can monitor the impact of the issue on the server by monitoring the log for lines like the following:
xxxxxxxx |INFO |LightstreamerLogger.webServer |SERVER POOLED THREAD 2 |Serving request: /lightstreamer/xhr.html?id=3&domain=lightstreamer.com& on "Lightstreamer HTTP Server" from xxxxxxx
My colleagues will get back to you about the issue in general ASAP
-
OK thanks for the update.
I will get today's log to you as soon as I can. While it was running, the load was apparently normal on the firewall so that is both good news and bad news (from a diagnosis point of view). However, the users complained that the application was performing poorly while it was running. I don't know if this is a coincidence or not - but I do know that the LS server was using a fairly high level of CPU while it was active (about 16%).
Is there any way to throttle the CPU used? I already have <selector_pool_size>1</selector_pool_size>
-
Hi Kevin,
From the logs we noticed that the issue starts after 2 restarts of the server executed at 23:27 and 23:35 on 21 February.
Before these events, all day long, the server seems to run regularly, and the relationship between connections and sessions remains within acceptable limits.
Soon after the restarts, indeed, the connections count begins to be very high compared to the number of sessions, but only about 10 minutes after the second restart that this bursts exceeding 1000 and continuing to grow.
Also the logs starts to show exceptions like this:
Code:
21-Feb-15 23:58:59,316|ERROR|LightstreamerLogger.connections | Lightstreamer HTTPS Server|Accept error on socket Lightstreamer HTTPS Server:
java.net.SocketException: Invalid argument
...
that are really strange and suggest a heavy problem with the network.
Could you confirm that the restarts of the server was something planned or have been performed as a result of a problem?
The first resart takes a very long time, in the meantime have been carried out some updates or changes to the systems or network?
-
OK thanks - I will ask what they were up to
-
@Mone - I am now on client version Version 6.2.5 build 1669 but I am still seeing the DOM elements building up. Eventually my browser dies as a result.
At the moment the customer has the LS server switched off - so the same thing is happening on their browsers. Eventually they die. So it is quite a bad situation although not connected directly to this issue.