Page 1 of 2 12 LastLast
Results 1 to 10 of 17
  1. #1
    Power Member
    Join Date
    Sep 2013
    Location
    Coimbatore
    Posts
    121

    Issue on deleting stocks from adapter

    I have some issue on my adapter. But my server code working properly. Some times adapter showing old data in user page. Some times it comes and disappear, and in some case now my server shows only 2 rows but adapter having 3 stock rows and displaying 3 rows in user page. But extra one row is too old data and deleted from server also. Even though it showing in adapter. If I restart the LS it working fine, but after some times again this issue comes and I need to restart the server. I couldn't find the reason and issue for this.
    I'm using the following code,


    private final HashMap<String, RateRow> liveratesnew = new HashMap<String, RateRow>();
    private final HashMap<String, RateRow> liveratesold = new HashMap<String, RateRow>();


    synchronized (liveratesnew) {
    if ((liveratesnew.size() != liveratesold.size())) {
    if (liveratesnew.size() > liveratesold.size()) {
    synchronized (liveratesold) {
    liveratesnew.forEach((k, v) -> {
    if (!liveratesold.containsKey(liveratesnew.get(k).get ItemName())) {
    liveratesold.put(liveratesnew.get(k).getItemName() , v);
    final HashMap<String, String> liverateevent = new HashMap<String, String>();
    liverateevent.put("key", liveratesnew.get(k).getItemName());
    liverateevent.put("desc", liveratesnew.get(k).getInstrumentName());
    liverateevent.put("bid", liveratesnew.get(k).getBidRate());
    liverateevent.put("ask", liveratesnew.get(k).getAskRate());
    listener.onActualStatus(liveratesnew.get(k).getIte mName(), liverateevent,
    false, 1);
    }
    });
    }
    } else if (liveratesnew.size() < liveratesold.size()) {
    final HashMap<String, String> removeoldkey = new HashMap<String, String>();
    synchronized (liveratesold) {
    for (Entry<String, RateRow> e : liveratesold.entrySet()) {
    String key = e.getKey();
    if (!liveratesnew.containsKey(liveratesold.get(key))) {
    listener.onDeleteStatus(liveratesold.get(key).getI temName());
    removeoldkey.put(key, liveratesold.get(key).getItemName());
    // liveratesold.remove(liveratesold.get(key).getItemN ame());


    }
    }
    }


    for (Entry<String, String> e : removeoldkey.entrySet()) {
    String key = e.getKey();
    // listener.onDeleteStatus(key);
    liveratesold.remove(key);
    }
    removeoldkey.clear();


    }
    }
    }

  2. #2
    Power Member
    Join Date
    Feb 2008
    Location
    Siracusa
    Posts
    161
    Hi rvkvino,

    it's very hard to understand what is going on from the code you shown, as it its not clear where it is located inside your adapter.
    That said, at first glance it seems that you are running against some race conditions, as you are checking liveratesold.size() outside the synchronized blocks on which it could be modified.

    Try to move size checks inside synchronized blocks and get back to us.

    Gianluca

  3. #3
    Power Member
    Join Date
    Sep 2013
    Location
    Coimbatore
    Posts
    121
    public class ExternalFeedSimulator {
    private static final Timer dispatcher = new Timer();
    private final String REMOTE_RATE_FEED_URL = "serverurl";
    private final ArrayList<RateRow> liverates = new ArrayList<RateRow>();
    private final HashMap<String, RateRow> liveratesnew = new HashMap<String, RateRow>();
    private final HashMap<String, RateRow> liveratesold = new HashMap<String, RateRow>();
    private ExternalFeedListener listener;


    public void start() {
    sendGet();
    // long waitTime = displayrates.computeNextWaitTime();
    long waitTime = 500;
    liveratescheduleGenerator(liverates, waitTime);
    }


    // HTTP GET request
    public void sendGet() {
    String url = REMOTE_RATE_FEED_URL;
    URL obj;
    try {
    obj = new URL(url);
    HttpURLConnection con = (HttpURLConnection) obj.openConnection();


    // optional default is GET
    con.setRequestMethod("GET");


    // add request header
    con.setRequestProperty("User-Agent", USER_AGENT);


    // int responseCode = con.getResponseCode();


    // System.out.println("Response Code : " + responseCode);


    BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
    String inputLine;
    StringBuffer response = new StringBuffer();


    while ((inputLine = in.readLine()) != null) {
    response.append(inputLine);
    }
    in.close();


    if (response.toString() == null || response.toString().equals(""))
    return;
    String strlowerresponse = response.toString();


    String trade_type = getItemValue(strlowerresponse, "trade_type");
    String rate_display = getItemValue(strlowerresponse, "rate_display");
    String update_time = getItemValue(strlowerresponse, "gold_updatetime");
    String market_closed_msg = "";
    String commodityupdatetime = getItemValue(strlowerresponse, "commodityupdatetime");


    trade_type = trade_type.trim();
    int startIndex = 0;
    liveratesnew.clear();
    while (startIndex != -1) {
    int item_start_pos = strlowerresponse.indexOf("<Commodity>", startIndex) + 11;
    if (item_start_pos < 11)
    break;
    int item_end_pos = strlowerresponse.indexOf("</Commodity>", item_start_pos + 1);
    if (item_end_pos <= -1)
    break;
    String strnode = strlowerresponse.substring(item_start_pos, item_end_pos);
    String instrid = getItemValue(strnode, "id");
    String instrname = getItemValue(strnode, "name");
    String strsellrate = getItemValue(strnode, "selling_rate");
    String strbuyrate = getItemValue(strnode, "buying_rate");
    String strhighrate = getItemValue(strnode, "selling_high");
    String strlowrate = getItemValue(strnode, "selling_low");


    int ordernumber = Integer.parseInt(getItemValue(strnode, "ordernumber"));
    if (strbuyrate.equalsIgnoreCase("") || strbuyrate.equalsIgnoreCase("0"))
    strbuyrate = "-";
    if (strsellrate.equalsIgnoreCase("") || strsellrate.equalsIgnoreCase("0"))
    strsellrate = "-";
    RateRow commrate = new RateRow("Item" + instrid, instrname, strbuyrate, strsellrate, strlowrate,
    strhighrate, ordernumber);


    liverates.add(commrate);
    liveratesnew.put("Item" + instrid, commrate);
    startIndex = item_end_pos;
    }


    if (trade_type.equalsIgnoreCase("3") || rate_display.equalsIgnoreCase("1")) {
    rate_display = "0";
    } else {
    rate_display = "1";
    }
    RateRow marketstatus = new RateRow("Marketstatus", rate_display, commodityupdatetime, update_time,
    market_closed_msg);
    curmarketstatus.add(marketstatus);
    } catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
    } catch (NumberFormatException e) {
    System.out.println("not a number");
    }
    }


    public synchronized void removeListener() {
    // remove the listener
    this.listener = null;
    }


    public String getItemValue(String strdata, String query) {
    String strlowerdata = strdata.toLowerCase();
    query = query.toLowerCase();
    int item_start_pos = strlowerdata.indexOf("<" + query + ">") + query.length() + 2;
    if (item_start_pos < query.length() + 2)
    return "";
    int item_end_pos = strlowerdata.indexOf("</" + query + ">");
    if (item_end_pos < query.length() + 2)
    return "";


    String item_value = strdata.substring(item_start_pos, item_end_pos);
    item_value = item_value.trim();
    return item_value;
    }


    private void currentmarketstatusscheduleGenerator(final ArrayList<RateRow> nmarketstatus, long waitTime) {
    dispatcher.schedule(new TimerTask() {
    @Override
    public void run() {
    long nextWaitTime;
    synchronized (nmarketstatus) {
    if (listener != null) {
    for (RateRow marketstatus : nmarketstatus) {
    final HashMap<String, String> markstatus = new HashMap<String, String>();
    markstatus.put("desc", marketstatus.getInstrumentName());
    markstatus.put("ratedisplay", marketstatus.getBidRate());
    markstatus.put("comupdatetime", marketstatus.getAskRate());
    markstatus.put("updatetime", marketstatus.getLow_price());
    markstatus.put("msg", marketstatus.getHigh_price());
    listener.onEvent(marketstatus.getInstrumentName(), markstatus, false);
    }
    }
    nmarketstatus.clear();
    nextWaitTime = 500;
    }


    currentmarketstatusscheduleGenerator(curmarketstat us, nextWaitTime);
    }
    }, waitTime);
    }
    private void liveratescheduleGenerator(final ArrayList<RateRow> displayrates, long waitTime) {
    dispatcher.schedule(new TimerTask() {
    @Override
    public void run() {
    long nextWaitTime;
    synchronized (displayrates) {
    if (listener != null) {
    for (RateRow curbidaskrates : displayrates) {
    final HashMap<String, String> bidaskevent = new HashMap<String, String>();
    bidaskevent.put("key", curbidaskrates.getItemName());
    bidaskevent.put("desc", curbidaskrates.getInstrumentName());
    bidaskevent.put("bid", curbidaskrates.getBidRate());
    bidaskevent.put("ask", curbidaskrates.getAskRate());
    bidaskevent.put("low", curbidaskrates.getLow_price());
    bidaskevent.put("high", curbidaskrates.getHigh_price());
    bidaskevent.put("order", Integer.toString(curbidaskrates.getDisplayorder()) );
    listener.onActualStatus(curbidaskrates.getItemName (), bidaskevent, false, 2);
    // listener.onEvent(curbidaskrates.getItemName(),
    // bidaskevent, false);
    }


    }
    displayrates.clear();
    nextWaitTime = 500;


    synchronized (liveratesnew) {
    if ((liveratesnew.size() != liveratesold.size())) {
    if (liveratesnew.size() > liveratesold.size()) {
    synchronized (liveratesold) {
    liveratesnew.forEach((k, v) -> {
    if (!liveratesold.containsKey(liveratesnew.get(k).get ItemName())) {
    liveratesold.put(liveratesnew.get(k).getItemName() , v);
    final HashMap<String, String> liverateevent = new HashMap<String, String>();
    liverateevent.put("key", liveratesnew.get(k).getItemName());
    liverateevent.put("desc", liveratesnew.get(k).getInstrumentName());
    liverateevent.put("bid", liveratesnew.get(k).getBidRate());
    liverateevent.put("ask", liveratesnew.get(k).getAskRate());
    liverateevent.put("low", liveratesnew.get(k).getLow_price());
    liverateevent.put("high", liveratesnew.get(k).getHigh_price());
    liverateevent.put("order",
    Integer.toString(liveratesnew.get(k).getDisplayord er()));
    listener.onActualStatus(liveratesnew.get(k).getIte mName(), liverateevent,
    false, 1);
    }
    });
    }
    } else if (liveratesnew.size() < liveratesold.size()) {

    final HashMap<String, String> removeoldkey = new HashMap<String, String>();
    synchronized (liveratesold) {
    for (Entry<String, RateRow> e : liveratesold.entrySet()) {
    String key = e.getKey();
    if (!liveratesnew.containsKey(liveratesold.get(key))) {
    listener.onDeleteStatus(liveratesold.get(key).getI temName());
    removeoldkey.put(key, liveratesold.get(key).getItemName());
    // liveratesold.remove(liveratesold.get(key).getItemN ame());


    }
    }
    }


    for (Entry<String, String> e : removeoldkey.entrySet()) {
    String key = e.getKey();
    // listener.onDeleteStatus(key);
    liveratesold.remove(key);
    }
    removeoldkey.clear();


    }
    }
    }


    }
    sendGet();
    liveratescheduleGenerator(liverates, nextWaitTime);
    }
    }, waitTime);
    }


    public void setFeedListener(ExternalFeedListener listener) {
    this.listener = listener;
    }


    public void sendRates(String itemName) {
    liveratesnew.forEach((k, v) -> {
    final RateRow liverate = liveratesnew.get(k);
    if (liverate.getItemName().contains(itemName)) {
    dispatcher.schedule(new TimerTask() {
    @Override
    public void run() {
    synchronized (liverate) {
    final HashMap<String, String> crates = new HashMap<String, String>();
    crates.put("key", liverate.getItemName());
    crates.put("desc", liverate.getInstrumentName());
    crates.put("bid", liverate.getBidRate());
    crates.put("ask", liverate.getAskRate());
    crates.put("low", liverate.getLow_price());
    crates.put("high", liverate.getHigh_price());
    crates.put("order", Integer.toString(liverate.getDisplayorder()));
    listener.onActualStatus(liverate.getItemName(), crates, true, 1);
    }
    }
    }, 0);
    }


    });
    }
    }

  4. #4
    Power Member
    Join Date
    Feb 2008
    Location
    Siracusa
    Posts
    161
    Hi rvkvino,

    please try to remove possible race conditions as pointed out earlier; after that, we will evaluate whether to proceed with code inspection.

    Thanks and Regards,
    Gianluca

  5. #5
    Power Member
    Join Date
    Sep 2013
    Location
    Coimbatore
    Posts
    121
    As You have mentioned I have checked the code many times and I have return the size checking code inside the synchronized blocks only. This issue not happening all the times. Some of the times only this issue happening and I need to restart the LS server then only get it work properly.

  6. #6
    Power Member
    Join Date
    Feb 2008
    Location
    Siracusa
    Posts
    161
    Ok rkvino,

    we need the LS server log to investigate the issue. Before that, please update lighstreamer_log_conf.xml as follows:

    <logger name="LightstreamerLogger.subscriptions" level="DEBUG"/>
    <logger name="LightstreamerLogger.pump" level="DEBUG"/>

    and try to reproduce the problem.

    Then, send the log file to support@lightstreamer.com.

    Most important, please precisely describe what you are expecting in terms of rows to be displayed, providing us with a simple example of real data.

    Thanks
    Gianluca

  7. #7
    Power Member
    Join Date
    Sep 2013
    Location
    Coimbatore
    Posts
    121
    We are facing this issue often. My remote adapter produces only 2 rows of stocks. Example Stock-A bid-123 ask-124 and Stock-B bid 122 ask 125 like this. But I'm receiving some times Stock-X with this 2 stocks and totally showing 3 stocks. This Stock-X was created previously and deleted. Even now not in a database also. While facing this issue I need to restart the LS server. If I restart LS server then it disappears Stock-X but after some times it again shows in the stock list. But this Stock-X now deleted from the server. Now I feel like this stock stored in LS server cookies or cache. If it stores in LS server in-memory means how do I delete all the in memory data and reproduce again.

  8. #8
    Administrator
    Join Date
    Jul 2006
    Location
    Milan
    Posts
    1,090
    Since you talk about "rows" I suppose that you are referring to an item in COMMAND mode which contains the stocks as keys.
    Can you please confirm that?

    If you have an item in COMMAND mode and you send a DELETE event for a key, and then you see the key appearing again upon new subscriptions, this is unexpected.
    However, only by watching at a concrete log, like instructed by Gianluca, can we find what is wrong.

    One possibility is that you send a DELETE event for the key but immediately later, because of lack of synchronization, you send an UPDATE event for the same key. In this case, the Server would fix the inconsistency by converting the UPDATE into a new ADD. You can detect the case in the log, by finding a WARN like this:
    Unexpected UPDATE event for key XXX. Event propagated as an ADD command.

    Anyway, if this happens and you need to clean up the whole item, you can also try invoking clearSnapshot from your Data Adapter.

  9. #9
    Power Member
    Join Date
    Sep 2013
    Location
    Coimbatore
    Posts
    121
    I'm using the Command mode only in my adapter, and also I have used delete command when removing stock from server. Now we are having only 2 stocks but it shows 3 stocks unfortunately. When I restart the LS server then it shows 2 stocks. How to clean up the whole item from the server.

  10. #10
    Administrator
    Join Date
    Jul 2006
    Location
    Milan
    Posts
    1,090
    It's not clear to me what you mean by "using the Command mode only in my adapter"; I suppose that the clients also subscribe to your item by specifying COMMAND mode, am I right?
    Hence, the second part of my previous answer applies.

 

 

Similar Threads

  1. Replies: 4
    Last Post: October 24th, 2011, 10:33 AM
  2. TIBCO AMS-GI data adapter issue
    By ganeshk in forum Adapter SDKs
    Replies: 1
    Last Post: September 22nd, 2010, 07:52 PM
  3. Replies: 3
    Last Post: January 7th, 2010, 09:57 AM
  4. Replies: 4
    Last Post: January 3rd, 2010, 09:03 AM
  5. Problem running JMS stocks demo
    By mnenchev in forum General
    Replies: 3
    Last Post: August 10th, 2009, 04:22 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
All times are GMT +1. The time now is 10:27 AM.