Results 1 to 2 of 2
  1. #1

    Python: lightstream data and have another process access last 15minutes histo

    Hello,
    I followed your github example for Python and it works great with gainCapital (thank you).
    That said, I am trying to keep the last 15 minutes of historics (or other timeperiod), in a shared memory class, which i would be able to access from another processor to compute statistics. Ideally I would use a third processor to manage orders sent/book positions.
    So, I would ideally use mulitprocessing, where 1 processor would be used to handle the lightstreamer and fill histo of 15 minutes period, another processor to check values and calculated statistics and a 3rd one to handle the orders sent, and check positions, as well as P&L.
    Any help would be greatly appreciated.

    So far i modified the "_on_item_update" class method to add any new data to a database but I realised that calls to database takes too much time, hence looking at other alternative.
    Modified code (using pymysql):

    def _on_item_update(self, item_update):

    self.insert_to_mdb_with_pymysql(
    (
    item_update["values"]["MarketId"],
    _Datetime.utcfromtimestamp(
    int(
    str(
    str(item_update["values"]["TickDate"]).split(sep="(")[1]
    ).split(")")[0]
    )
    / 1000
    ),
    item_update["values"]["Bid"],
    item_update["values"]["Price"],
    item_update["values"]["Offer"],
    )
    )

    That bit of code works but it won't work for what i am trying to achieve.

    I would greatly appreciate help regarding, program structure, libraries, and if possible an example.

    Thank you for you help.
    NB: I was thinking to keep live in memory the last 15 minutes of data from my subscribed stocks but it might be better to save data to harddrive in csv or other best practise and make a call for the desired time line: I am exploring here, so would apreciate the help.

    Best
    Nono
    Last edited by nono_london; May 12th, 2020 at 10:59 PM. Reason: adding more details

  2. #2
    Administrator
    Join Date
    Feb 2008
    Location
    Siracusa
    Posts
    154
    Hi Nono,

    unfortunately, I'm not an expert in Python development, but it looks like a multithreading approach is the best way to achieve your targets.
    If inserting data into the database takes too much time, then probably you should switch to a sort of asynchronous task that will be in charge of only operating on the DB. Otherwise, you risk blocking too often the thread calling the _on_item_update method.
    For example, you could enqueue items onto a buffer which will be processed by another thread, whose task is to update the DB.

    Best,
    Gianluca

 

 

Similar Threads

  1. php/perl/python lightstream client
    By Eli Shabtay in forum Client SDKs
    Replies: 1
    Last Post: May 11th, 2015, 08:48 AM
  2. php/perl/python lightstream client
    By Eli Shabtay in forum General
    Replies: 1
    Last Post: May 11th, 2015, 08:48 AM
  3. get and process data from server
    By emmajyu in forum Client SDKs
    Replies: 2
    Last Post: June 24th, 2013, 09:01 AM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT +1. The time now is 08:09 AM.