-
September 1st, 2017, 08:22 AM
#1
Issue on deleting stocks from adapter
I have some issue on my adapter. But my server code working properly. Some times adapter showing old data in user page. Some times it comes and disappear, and in some case now my server shows only 2 rows but adapter having 3 stock rows and displaying 3 rows in user page. But extra one row is too old data and deleted from server also. Even though it showing in adapter. If I restart the LS it working fine, but after some times again this issue comes and I need to restart the server. I couldn't find the reason and issue for this.
I'm using the following code,
private final HashMap<String, RateRow> liveratesnew = new HashMap<String, RateRow>();
private final HashMap<String, RateRow> liveratesold = new HashMap<String, RateRow>();
synchronized (liveratesnew) {
if ((liveratesnew.size() != liveratesold.size())) {
if (liveratesnew.size() > liveratesold.size()) {
synchronized (liveratesold) {
liveratesnew.forEach((k, v) -> {
if (!liveratesold.containsKey(liveratesnew.get(k).get ItemName())) {
liveratesold.put(liveratesnew.get(k).getItemName() , v);
final HashMap<String, String> liverateevent = new HashMap<String, String>();
liverateevent.put("key", liveratesnew.get(k).getItemName());
liverateevent.put("desc", liveratesnew.get(k).getInstrumentName());
liverateevent.put("bid", liveratesnew.get(k).getBidRate());
liverateevent.put("ask", liveratesnew.get(k).getAskRate());
listener.onActualStatus(liveratesnew.get(k).getIte mName(), liverateevent,
false, 1);
}
});
}
} else if (liveratesnew.size() < liveratesold.size()) {
final HashMap<String, String> removeoldkey = new HashMap<String, String>();
synchronized (liveratesold) {
for (Entry<String, RateRow> e : liveratesold.entrySet()) {
String key = e.getKey();
if (!liveratesnew.containsKey(liveratesold.get(key))) {
listener.onDeleteStatus(liveratesold.get(key).getI temName());
removeoldkey.put(key, liveratesold.get(key).getItemName());
// liveratesold.remove(liveratesold.get(key).getItemN ame());
}
}
}
for (Entry<String, String> e : removeoldkey.entrySet()) {
String key = e.getKey();
// listener.onDeleteStatus(key);
liveratesold.remove(key);
}
removeoldkey.clear();
}
}
}
-
September 1st, 2017, 11:19 AM
#2
Hi rvkvino,
it's very hard to understand what is going on from the code you shown, as it its not clear where it is located inside your adapter.
That said, at first glance it seems that you are running against some race conditions, as you are checking liveratesold.size() outside the synchronized blocks on which it could be modified.
Try to move size checks inside synchronized blocks and get back to us.
Gianluca
-
September 1st, 2017, 01:22 PM
#3
public class ExternalFeedSimulator {
private static final Timer dispatcher = new Timer();
private final String REMOTE_RATE_FEED_URL = "serverurl";
private final ArrayList<RateRow> liverates = new ArrayList<RateRow>();
private final HashMap<String, RateRow> liveratesnew = new HashMap<String, RateRow>();
private final HashMap<String, RateRow> liveratesold = new HashMap<String, RateRow>();
private ExternalFeedListener listener;
public void start() {
sendGet();
// long waitTime = displayrates.computeNextWaitTime();
long waitTime = 500;
liveratescheduleGenerator(liverates, waitTime);
}
// HTTP GET request
public void sendGet() {
String url = REMOTE_RATE_FEED_URL;
URL obj;
try {
obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
// add request header
con.setRequestProperty("User-Agent", USER_AGENT);
// int responseCode = con.getResponseCode();
// System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
if (response.toString() == null || response.toString().equals(""))
return;
String strlowerresponse = response.toString();
String trade_type = getItemValue(strlowerresponse, "trade_type");
String rate_display = getItemValue(strlowerresponse, "rate_display");
String update_time = getItemValue(strlowerresponse, "gold_updatetime");
String market_closed_msg = "";
String commodityupdatetime = getItemValue(strlowerresponse, "commodityupdatetime");
trade_type = trade_type.trim();
int startIndex = 0;
liveratesnew.clear();
while (startIndex != -1) {
int item_start_pos = strlowerresponse.indexOf("<Commodity>", startIndex) + 11;
if (item_start_pos < 11)
break;
int item_end_pos = strlowerresponse.indexOf("</Commodity>", item_start_pos + 1);
if (item_end_pos <= -1)
break;
String strnode = strlowerresponse.substring(item_start_pos, item_end_pos);
String instrid = getItemValue(strnode, "id");
String instrname = getItemValue(strnode, "name");
String strsellrate = getItemValue(strnode, "selling_rate");
String strbuyrate = getItemValue(strnode, "buying_rate");
String strhighrate = getItemValue(strnode, "selling_high");
String strlowrate = getItemValue(strnode, "selling_low");
int ordernumber = Integer.parseInt(getItemValue(strnode, "ordernumber"));
if (strbuyrate.equalsIgnoreCase("") || strbuyrate.equalsIgnoreCase("0"))
strbuyrate = "-";
if (strsellrate.equalsIgnoreCase("") || strsellrate.equalsIgnoreCase("0"))
strsellrate = "-";
RateRow commrate = new RateRow("Item" + instrid, instrname, strbuyrate, strsellrate, strlowrate,
strhighrate, ordernumber);
liverates.add(commrate);
liveratesnew.put("Item" + instrid, commrate);
startIndex = item_end_pos;
}
if (trade_type.equalsIgnoreCase("3") || rate_display.equalsIgnoreCase("1")) {
rate_display = "0";
} else {
rate_display = "1";
}
RateRow marketstatus = new RateRow("Marketstatus", rate_display, commodityupdatetime, update_time,
market_closed_msg);
curmarketstatus.add(marketstatus);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (NumberFormatException e) {
System.out.println("not a number");
}
}
public synchronized void removeListener() {
// remove the listener
this.listener = null;
}
public String getItemValue(String strdata, String query) {
String strlowerdata = strdata.toLowerCase();
query = query.toLowerCase();
int item_start_pos = strlowerdata.indexOf("<" + query + ">") + query.length() + 2;
if (item_start_pos < query.length() + 2)
return "";
int item_end_pos = strlowerdata.indexOf("</" + query + ">");
if (item_end_pos < query.length() + 2)
return "";
String item_value = strdata.substring(item_start_pos, item_end_pos);
item_value = item_value.trim();
return item_value;
}
private void currentmarketstatusscheduleGenerator(final ArrayList<RateRow> nmarketstatus, long waitTime) {
dispatcher.schedule(new TimerTask() {
@Override
public void run() {
long nextWaitTime;
synchronized (nmarketstatus) {
if (listener != null) {
for (RateRow marketstatus : nmarketstatus) {
final HashMap<String, String> markstatus = new HashMap<String, String>();
markstatus.put("desc", marketstatus.getInstrumentName());
markstatus.put("ratedisplay", marketstatus.getBidRate());
markstatus.put("comupdatetime", marketstatus.getAskRate());
markstatus.put("updatetime", marketstatus.getLow_price());
markstatus.put("msg", marketstatus.getHigh_price());
listener.onEvent(marketstatus.getInstrumentName(), markstatus, false);
}
}
nmarketstatus.clear();
nextWaitTime = 500;
}
currentmarketstatusscheduleGenerator(curmarketstat us, nextWaitTime);
}
}, waitTime);
}
private void liveratescheduleGenerator(final ArrayList<RateRow> displayrates, long waitTime) {
dispatcher.schedule(new TimerTask() {
@Override
public void run() {
long nextWaitTime;
synchronized (displayrates) {
if (listener != null) {
for (RateRow curbidaskrates : displayrates) {
final HashMap<String, String> bidaskevent = new HashMap<String, String>();
bidaskevent.put("key", curbidaskrates.getItemName());
bidaskevent.put("desc", curbidaskrates.getInstrumentName());
bidaskevent.put("bid", curbidaskrates.getBidRate());
bidaskevent.put("ask", curbidaskrates.getAskRate());
bidaskevent.put("low", curbidaskrates.getLow_price());
bidaskevent.put("high", curbidaskrates.getHigh_price());
bidaskevent.put("order", Integer.toString(curbidaskrates.getDisplayorder()) );
listener.onActualStatus(curbidaskrates.getItemName (), bidaskevent, false, 2);
// listener.onEvent(curbidaskrates.getItemName(),
// bidaskevent, false);
}
}
displayrates.clear();
nextWaitTime = 500;
synchronized (liveratesnew) {
if ((liveratesnew.size() != liveratesold.size())) {
if (liveratesnew.size() > liveratesold.size()) {
synchronized (liveratesold) {
liveratesnew.forEach((k, v) -> {
if (!liveratesold.containsKey(liveratesnew.get(k).get ItemName())) {
liveratesold.put(liveratesnew.get(k).getItemName() , v);
final HashMap<String, String> liverateevent = new HashMap<String, String>();
liverateevent.put("key", liveratesnew.get(k).getItemName());
liverateevent.put("desc", liveratesnew.get(k).getInstrumentName());
liverateevent.put("bid", liveratesnew.get(k).getBidRate());
liverateevent.put("ask", liveratesnew.get(k).getAskRate());
liverateevent.put("low", liveratesnew.get(k).getLow_price());
liverateevent.put("high", liveratesnew.get(k).getHigh_price());
liverateevent.put("order",
Integer.toString(liveratesnew.get(k).getDisplayord er()));
listener.onActualStatus(liveratesnew.get(k).getIte mName(), liverateevent,
false, 1);
}
});
}
} else if (liveratesnew.size() < liveratesold.size()) {
final HashMap<String, String> removeoldkey = new HashMap<String, String>();
synchronized (liveratesold) {
for (Entry<String, RateRow> e : liveratesold.entrySet()) {
String key = e.getKey();
if (!liveratesnew.containsKey(liveratesold.get(key))) {
listener.onDeleteStatus(liveratesold.get(key).getI temName());
removeoldkey.put(key, liveratesold.get(key).getItemName());
// liveratesold.remove(liveratesold.get(key).getItemN ame());
}
}
}
for (Entry<String, String> e : removeoldkey.entrySet()) {
String key = e.getKey();
// listener.onDeleteStatus(key);
liveratesold.remove(key);
}
removeoldkey.clear();
}
}
}
}
sendGet();
liveratescheduleGenerator(liverates, nextWaitTime);
}
}, waitTime);
}
public void setFeedListener(ExternalFeedListener listener) {
this.listener = listener;
}
public void sendRates(String itemName) {
liveratesnew.forEach((k, v) -> {
final RateRow liverate = liveratesnew.get(k);
if (liverate.getItemName().contains(itemName)) {
dispatcher.schedule(new TimerTask() {
@Override
public void run() {
synchronized (liverate) {
final HashMap<String, String> crates = new HashMap<String, String>();
crates.put("key", liverate.getItemName());
crates.put("desc", liverate.getInstrumentName());
crates.put("bid", liverate.getBidRate());
crates.put("ask", liverate.getAskRate());
crates.put("low", liverate.getLow_price());
crates.put("high", liverate.getHigh_price());
crates.put("order", Integer.toString(liverate.getDisplayorder()));
listener.onActualStatus(liverate.getItemName(), crates, true, 1);
}
}
}, 0);
}
});
}
}
-
September 1st, 2017, 02:26 PM
#4
Hi rvkvino,
please try to remove possible race conditions as pointed out earlier; after that, we will evaluate whether to proceed with code inspection.
Thanks and Regards,
Gianluca
-
September 4th, 2017, 03:36 PM
#5
As You have mentioned I have checked the code many times and I have return the size checking code inside the synchronized blocks only. This issue not happening all the times. Some of the times only this issue happening and I need to restart the LS server then only get it work properly.
-
September 4th, 2017, 04:12 PM
#6
Ok rkvino,
we need the LS server log to investigate the issue. Before that, please update lighstreamer_log_conf.xml as follows:
<logger name="LightstreamerLogger.subscriptions" level="DEBUG"/>
<logger name="LightstreamerLogger.pump" level="DEBUG"/>
and try to reproduce the problem.
Then, send the log file to support@lightstreamer.com.
Most important, please precisely describe what you are expecting in terms of rows to be displayed, providing us with a simple example of real data.
Thanks
Gianluca
-
September 12th, 2017, 06:11 PM
#7
We are facing this issue often. My remote adapter produces only 2 rows of stocks. Example Stock-A bid-123 ask-124 and Stock-B bid 122 ask 125 like this. But I'm receiving some times Stock-X with this 2 stocks and totally showing 3 stocks. This Stock-X was created previously and deleted. Even now not in a database also. While facing this issue I need to restart the LS server. If I restart LS server then it disappears Stock-X but after some times it again shows in the stock list. But this Stock-X now deleted from the server. Now I feel like this stock stored in LS server cookies or cache. If it stores in LS server in-memory means how do I delete all the in memory data and reproduce again.
-
September 13th, 2017, 10:28 AM
#8
Since you talk about "rows" I suppose that you are referring to an item in COMMAND mode which contains the stocks as keys.
Can you please confirm that?
If you have an item in COMMAND mode and you send a DELETE event for a key, and then you see the key appearing again upon new subscriptions, this is unexpected.
However, only by watching at a concrete log, like instructed by Gianluca, can we find what is wrong.
One possibility is that you send a DELETE event for the key but immediately later, because of lack of synchronization, you send an UPDATE event for the same key. In this case, the Server would fix the inconsistency by converting the UPDATE into a new ADD. You can detect the case in the log, by finding a WARN like this:
Unexpected UPDATE event for key XXX. Event propagated as an ADD command.
Anyway, if this happens and you need to clean up the whole item, you can also try invoking clearSnapshot from your Data Adapter.
-
September 13th, 2017, 10:44 AM
#9
I'm using the Command mode only in my adapter, and also I have used delete command when removing stock from server. Now we are having only 2 stocks but it shows 3 stocks unfortunately. When I restart the LS server then it shows 2 stocks. How to clean up the whole item from the server.
-
September 13th, 2017, 11:29 AM
#10
It's not clear to me what you mean by "using the Command mode only in my adapter"; I suppose that the clients also subscribe to your item by specifying COMMAND mode, am I right?
Hence, the second part of my previous answer applies.
Similar Threads
-
By Alessandro in forum Adapter SDKs
Replies: 4
Last Post: October 24th, 2011, 09:33 AM
-
By ganeshk in forum Adapter SDKs
Replies: 1
Last Post: September 22nd, 2010, 06:52 PM
-
By sjohn in forum Adapter SDKs
Replies: 3
Last Post: January 7th, 2010, 08:57 AM
-
By niravmehta2009 in forum General
Replies: 4
Last Post: January 3rd, 2010, 08:03 AM
-
By mnenchev in forum General
Replies: 3
Last Post: August 10th, 2009, 03:22 PM
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
All times are GMT +1. The time now is 02:12 PM.
Bookmarks