Results 1 to 10 of 11

Hybrid View

  1. #1
    Member
    Join Date
    Apr 2007
    Location
    Kerry
    Posts
    17

    Bandwidth and event pushing

    Hi all,

    I have implemented my own adapter and my own client using the lighstreamer server and client APIs. So far, so good, everything works.

    In the client API I used the proxy approach to see the bandwidth usage. I've seen that the average usage is 1.5Kbit/sec. Currently I'm pushing Java objects from the server (ok, the attributes, I build the objects afterwards) and currently I'm getting an average of 3 objects/second. That seems quite slow, so probably I'm quite sure that I'm doing something wrong or that there is a way to tune this.

    I tried to change the max_bandwidth, max_frequency and buffer_size parameters in the adapters.xml file but with no apparent effect.

    So the question is, is there a way to improve the bandwidth/throughput? would it be a solution to implement more items and instead of doing a single subscription do many subscriptions?

    Thanks,
    Martin

  2. #2
    Member
    Join Date
    Apr 2007
    Location
    Kerry
    Posts
    17
    I forgot to say that I'm running the Vivace edition, quite important due to bandwidth restrictions I guess.

  3. #3
    Power Member
    Join Date
    Jul 2006
    Location
    Cesano Maderno, Italy
    Posts
    784
    Hi,

    please, could you modify your lightstreamer_log_conf.xml file to set the "subscriptions" and "pump" categories to DEBUG level and then attach a server log showing the issue?

  4. #4
    Member
    Join Date
    Apr 2007
    Location
    Kerry
    Posts
    17

    Logs attached

    Sure.

    I'm currently trying to simulate a trading environment. I've deployed an adapter that contains an embedded rate generator. This generator creates rates at variable speeds, i.e. Some rates are generated more frequently than others. My Lighstreamer adaptor listens to that generator and each time it receives a rate pushes it to the client side.

    I have been doing some tests and also looking at the log files and now the problem seems clear to me (of course, probably it's not a problem). Basically I'm pushing all the rates objects through the same channel ("rates"). So, in the log file you can see how some currency pair rates are being discarded. See attached file "first-try.txt". In the client, I only receive the data that is pumped in the event.

    So I guess I should create a channel per currency pair. Does this have any consequence in the communication? (i.e. does this mean more sockets open? does this mean more bandwidth usage? etc.)

    Thanks for the useful help and regards,
    Martin

    PS. Note that I have stripped down the log for being able to upload it.
    Attached Files Attached Files

  5. #5
    Member
    Join Date
    Apr 2007
    Location
    Kerry
    Posts
    17

    Puzzled about second test

    Hi again.

    I have upgraded my test to create six different subscription channels one for each currency pair I have.

    However, I'm now puzzled because the adapter seems to successfully send all the rates in the currency pairs some times, but there are other times that it performs throttling and only sends some of the rates. In the attached file "second-try.txt" see how the first time only one rate is sent, but in the second time the four rates are sent, the next time only 2 of five rates are sent, the next time 4/7, etc.

    I'm sure there is some logic behind of this behaviour. Any tips?

    Thanks,
    Martin
    Attached Files Attached Files

  6. #6
    Power Member
    Join Date
    Jul 2006
    Location
    Cesano Maderno, Italy
    Posts
    784

    max_delay_millis

    Hello,

    Are you using the MERGE subscription mode right?

    The problem is that since the updates are all published on the same item, Lightstreamer merges the updates if needed:

    Code:
    08.May.07 09:02:05,934 <TRACE> INCOMING DATA for rates --> {bidValue=0.07943651574153077, sellCCY=USD, bidCCY=EUR, sellValue=0.19836981819406774}
    08.May.07 09:02:05,935 <TRACE> INCOMING DATA for rates --> {bidValue=0.04185744998798879, sellCCY=JPY, bidCCY=EUR, sellValue=0.002155323147410959}
    08.May.07 09:02:05,936 <TRACE> INCOMING DATA for rates --> {bidValue=0.9033005599346077, sellCCY=USD, bidCCY=JPY, sellValue=0.3281485803285823}
    08.May.07 09:02:05,937 <TRACE> INCOMING DATA for rates --> {bidValue=0.947976333651295, sellCCY=GBP, bidCCY=EUR, sellValue=0.38439026792729647}
    08.May.07 09:02:06,044 <TRACE> INCOMING DATA for rates --> {bidValue=0.9040453255654483, sellCCY=USD, bidCCY=EUR, sellValue=0.9458336664422791}
    08.May.07 09:02:06,045 <TRACE> INCOMING DATA for rates --> {bidValue=0.9708380297991672, sellCCY=GBP, bidCCY=EUR, sellValue=0.215475738803115}
    08.May.07 09:02:06,134 <TRACE> Pumping event in session S1321751661N1: 1,1,1||0.9708380297991672||0.215475738803115
    In the example 6 updates arrives to the server but only one is forwarded to the clients.

    If yoo take a look at the timings of the "Pumping in Session" messages you will see a 200ms pattern:

    09:02:05,537
    09:02:05,734
    09:02:05,934
    09:02:06,134
    and so on...

    Those 200ms are set in the Lightstreamer_conf.xml file:
    Code xml:
    1.  <max_delay_millis>200</max_delay_millis>

    What appens is that Lightstreamer is enabled to send just one packet each 200ms (this is a session related delay) unless there is too much data to be sent in one single packet (due to frequency limits bandwidth limits etc this is just a simple explanation of the delivering algorithm). Btw since you publish every update to the same item, after 200ms from the last "Pumping event in session" - from Lightstreamer point of view - there is only one relevant update that must be sent (the latest).


    Quote Originally Posted by churrusco
    So I guess I should create a channel per currency pair.
    This is indeed a good idea.

    Quote Originally Posted by churrusco
    does this mean more sockets open?
    The socket is always the same as Lightstreamer protocol take care of handling more items on the same data flow

    Quote Originally Posted by churrusco
    does this mean more bandwidth usage?
    Obviously more bandwidth will be used if more updates reach the client, btw you can limit the bandwidth both client and server side.

  7. #7
    Power Member
    Join Date
    Jul 2006
    Location
    Cesano Maderno, Italy
    Posts
    784
    Hi,

    You wrote another post while I was writing the response for the previous one

    In the new log you can see how the merge algorithm works when different items are used:
    Code:
    08.May.07 09:56:57,239 <TRACE> INCOMING DATA for EUR/GBP --> {bidValue=0.3952132682553846, sellCCY=GBP, bidCCY=EUR, sellValue=0.1582634326389245}
    08.May.07 09:56:57,338 <TRACE> INCOMING DATA for EUR/USD --> {bidValue=0.9052316638725739, sellCCY=USD, bidCCY=EUR, sellValue=0.47263148734258653}
    08.May.07 09:56:57,339 <TRACE> INCOMING DATA for EUR/GBP --> {bidValue=0.7117531450899011, sellCCY=GBP, bidCCY=EUR, sellValue=0.5475364773823609}
    08.May.07 09:56:57,340 <TRACE> INCOMING DATA for USD/NZD --> {bidValue=0.3461941736979528, sellCCY=NZD, bidCCY=USD, sellValue=0.19362361881184487}
    08.May.07 09:56:57,439 <TRACE> INCOMING DATA for EUR/USD --> {bidValue=0.7626206088735283, sellCCY=USD, bidCCY=EUR, sellValue=0.5145519477153373}
    08.May.07 09:56:57,440 <TRACE> INCOMING DATA for JPY/USD --> {bidValue=0.204416519895124, sellCCY=USD, bidCCY=JPY, sellValue=0.2764089416286716}
    08.May.07 09:56:57,440 <TRACE> INCOMING DATA for EUR/GBP --> {bidValue=0.5949231423651182, sellCCY=GBP, bidCCY=EUR, sellValue=0.5972830556785825}
    08.May.07 09:56:57,450 <TRACE> Pumping event in session S-725498679N1: 4,1,1||0.7626206088735283||0.5145519477153373
    3,1,1||0.5949231423651182||0.5972830556785825
    6,1,1|USD|0.3461941736979528|NZD|0.19362361881184487
    2,1,1||0.204416519895124||0.2764089416286716
    Inside this 200ms you can see that the server accumulated 7 events but 4 are sent to the client. If you look further you can see that those 7 updates pertains to only 4 items because there are 3 EUR/GBP and 2 EUR/USD events so that the server to send the latest update per each item have to send only 4 updates.

    Hope that helps.

 

 

Similar Threads

  1. Bandwidth in iOS
    By bor306 in forum Client SDKs
    Replies: 7
    Last Post: July 11th, 2012, 03:28 PM
  2. AMF and Consuming Network Bandwidth?
    By ernivan in forum Client SDKs
    Replies: 1
    Last Post: March 11th, 2011, 10:35 AM
  3. Replies: 1
    Last Post: March 11th, 2010, 04:53 PM
  4. Replies: 4
    Last Post: August 14th, 2009, 04:28 AM
  5. Pushing complex data structure
    By shreyaspurohit in forum Adapter SDKs
    Replies: 4
    Last Post: March 30th, 2009, 12:06 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT +1. The time now is 12:08 PM.