Results 1 to 7 of 7
  1. #1
    Senior Member
    Join Date
    Aug 2014
    Posts
    71

    too much time during GC process

    hello,
    we have a problem during the lightstreamer garbage collection (GC). The total heap of our LS is 6 GB and when the free heap reach a bout 4 GB it takes too much time in GC process about 5:15 seconds that may lead to the lightstreamer be disconnected at this time of GC and then lead to some adapters be disconnected because of no data received.
    this problem of GC was started from 2 days ago only but befor there was no problem and the GC was not take any time.

    this is the LS log file of today : http://www.mediafire.com/download/3a...+12-1-2016.rar

  2. #2
    Senior Member
    Join Date
    Aug 2014
    Posts
    71
    hello,
    remind, i am wait for reply

  3. #3
    Administrator
    Join Date
    Feb 2012
    Location
    Milano
    Posts
    716
    Hi Ahmed,

    Please could you confirm the Lightstreamer server version in use and if you have added some GC tuning to your Lightstreamer instance?
    The garbage collection can be tuned in the launch script of Lighstreamer server, LS.bat or LS.sh depending your os.

    Please note that since Lightstreamer 6 we officially adds configuration for MaxGCPauseMillis that try to set a target for the maximum GC pause time. This is a soft goal, and the JVM will make its best effort to achieve it.
    This option is valid definitely for Lighstreamer 5.x too; furthermore in case of large heap we also suggest the G1 algorithm. G1 is an incremental parallel compacting GC that provides more predictable pause times compared to standard GC in particular with large heap.

    Hence, if you have not yet set any specific GC tuning you might try something like this:

    Code:
    	set JAVA_OPTS=-server -XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -XX:G1ReservePercent=20
    Regards,
    Giuseppe

  4. #4
    Senior Member
    Join Date
    Aug 2014
    Posts
    71
    we are using the standard GC for java
    for long time now we not do any changes to the GC thats why we attach the log file so you may something suspicious
    my LS is of virsion 5.1.2
    but i remind you that this problem was started from 2 days ago only but befor there was no problem and the GC was not take any time.

    this is the LS.bat file portion

    rem === CHECK THIS OUT
    rem ================================================== =====================
    rem JAVA_OPTS should contain any Java Virtual Machine options. Here are some tips:
    rem 1) Always use the "-server" option.
    rem 2) Give more RAM to the server process, especially with heavy traffic, by specifying a min and max "heap"
    rem E.g.: If you have 4 GB and the box is dedicated to Lightstreamer, you might set 1 GB min heap and 3 GB max
    rem heap with these options: "-Xms1G -Xmx3G"
    rem 3) Choose a better "garbage collector" if you want to reduce latency. An option that often gives good results is:
    rem "-XX:+UseConcMarkSweepGC". If you are using Java 7 (apart from early versions) or later, you might prefer to try:
    rem "-XX:+UseG1GC". Many other tuning options are available (please see Oracle docs).
    set JAVA_OPTS=-server -Xms6G

  5. #5
    Administrator
    Join Date
    Feb 2012
    Location
    Milano
    Posts
    716
    Hi Ahmed,

    From the log is evident the behavior of memory, which always repeats the same pattern, drops to approximately 4GB and then starts a major GC to recover approximately 2 GB; this takes several seconds (from 15 to about 1 minute).
    This is behavior is not acceptable.

    With a heap of 6GB we definitely recommends G1, and so I confirm to try the settings suggested above (rely on the default settings also makes you subject to possible changes of behavior).

    That said, it is difficult say why the problem started just two days ago. Some updates at system level (SO updates, JVM change, ...)?
    Some changes in memory usage (increase number of sessions, or the same number of sessions but with a increase in the frequency of updates, or larger size of the updates)?
    The changes might be very small but have triggered a threshold effect.

    Regards,
    Giuseppe

  6. #6
    Senior Member
    Join Date
    Aug 2014
    Posts
    71
    this is the LS log file of today.
    http://www.mediafire.com/download/dd...+13-1-2016.rar

    there is no problem in LS occured today and all thing works fine.
    you can compare this log file with the log of yesterday i attached in the previous post that you may find difference lead to the problem

  7. #7
    Administrator
    Join Date
    Feb 2012
    Location
    Milano
    Posts
    716
    Hi Ahmed,

    From the new log it is clear that the GC behaves much better.
    However I found a couple of cases where we have a GC pause of a few seconds (for example at 11:57:05,996 and at 14:57:11,976).
    These pauses are smaller than the connection timeout of clients and therefore have been harmless.

    Since the JVM have to handle with a quite large heap, you will always have a potential risk using the default GC configuration.
    What triggers the issue is hard to say, it does not seem the server load since it seems that for the Jan 13 the load was greater than Jan 12, do you confirm?
    But you should also consider the general state of the macchine apart Lightstreamer JVM process.

    But ultimately configuring the GC with the recommended settings you should shirk the issue.

    Regards,
    Giuseppe

 

 

Similar Threads

  1. connection to LS disconnect much
    By engcoder in forum Adapter SDKs
    Replies: 3
    Last Post: October 16th, 2014, 10:24 AM
  2. get and process data from server
    By emmajyu in forum Client SDKs
    Replies: 2
    Last Post: June 24th, 2013, 10:01 AM
  3. LS thread hangs on process exit
    By bartol82 in forum Adapter SDKs
    Replies: 3
    Last Post: October 20th, 2008, 03:01 PM
  4. Getting through IIS "Basic" authorization process
    By wmolde in forum Client SDKs
    Replies: 14
    Last Post: February 12th, 2008, 10:10 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT +1. The time now is 10:26 AM.