In katta-0.6.3, the LuceneClient class allows us to set up a timeout
value for the search. We tried that function and found some
interesting things. We did see the LuceneClient interrupted the slow
queries although the actual used time might be a little longer than
the timeout value. This is understandable. However, from what we saw,
we suspect that the involved katta nodes still continue to work on the
slow queries even after the LuceneClient times out the search. Could
you please help us make sure whether it is the case? If it is, do you
think there is an easy way to ask katta nodes to stop the work related
to the slow query right, too, after the LuceneClient timed out the
query?
Part of the response from Johannes:
So basically you want that when the timeout happens not only the querying threads on the client, but also the threads on the lucene-nodes stop there work, right ?
As far as i know that isn't implemented yet.
So this would be an improvement ticked! Not sure how hard to implement.
Basically we should send the timeout to the nodes as well. There we have to use a collector to query lucene. We could use http://lucene.apache.org/java/3_0_2/api/core/org/apache/lucene/search/TimeLimitingCollector.html.
Description
From Hongchao, dev-list:
In katta-0.6.3, the LuceneClient class allows us to set up a timeout
value for the search. We tried that function and found some
interesting things. We did see the LuceneClient interrupted the slow
queries although the actual used time might be a little longer than
the timeout value. This is understandable. However, from what we saw,
we suspect that the involved katta nodes still continue to work on the
slow queries even after the LuceneClient times out the search. Could
you please help us make sure whether it is the case? If it is, do you
think there is an easy way to ask katta nodes to stop the work related
to the slow query right, too, after the LuceneClient timed out the
query?
Part of the response from Johannes:
So basically you want that when the timeout happens not only the querying threads on the client, but also the threads on the lucene-nodes stop there work, right ?
As far as i know that isn't implemented yet.
So this would be an improvement ticked! Not sure how hard to implement.
Basically we should send the timeout to the nodes as well. There we have to use a collector to query lucene. We could use http://lucene.apache.org/java/3_0_2/api/core/org/apache/lucene/search/TimeLimitingCollector.html.
I modified the LuceneServer that it now receives the client-timeout from the client.
It converts this client-timeout to a server-timeout by multiplying it with a configurable value lucene.collector.timeout-percentage (value between 0-1, default is 0.75).
So if the client-tiemout is 10000 ms and the lucene.collector.timeout-percentage is configured to 0.75 then the server-timeout is 7500 ms.
If the timeout on the server exeeds the search will be stopped and the results, which were collected so far, will be returned.
This should have 2 effects:
long-running queries don't consume server capacity anymore after the client stopped waiting on the results
depending on the configuration these long-running queries now returning partial results before the client-timeout is expected
You can configure lucene.collector.timeout-percentage in the katta.node.properties.
Johannes Zillmann added a comment - 01/Feb/11 04:50 PM I modified the LuceneServer that it now receives the client-timeout from the client.
It converts this client-timeout to a server-timeout by multiplying it with a configurable value lucene.collector.timeout-percentage (value between 0-1, default is 0.75).
So if the client-tiemout is 10000 ms and the lucene.collector.timeout-percentage is configured to 0.75 then the server-timeout is 7500 ms.
If the timeout on the server exeeds the search will be stopped and the results, which were collected so far, will be returned.
This should have 2 effects:
long-running queries don't consume server capacity anymore after the client stopped waiting on the results
depending on the configuration these long-running queries now returning partial results before the client-timeout is expected
You can configure lucene.collector.timeout-percentage in the katta.node.properties.
Improvement
Resolved
Major
use timeout from Client in the LuceneServer as well
It converts this client-timeout to a server-timeout by multiplying it with a configurable value lucene.collector.timeout-percentage (value between 0-1, default is 0.75).
So if the client-tiemout is 10000 ms and the lucene.collector.timeout-percentage is configured to 0.75 then the server-timeout is 7500 ms.
If the timeout on the server exeeds the search will be stopped and the results, which were collected so far, will be returned.
This should have 2 effects:
You can configure lucene.collector.timeout-percentage in the katta.node.properties.