Hi Dojo,
We have this error in the SLSearch log file. Can you clarify what it means ?
2024/06/12 19:58:24.742|SLDBConnection|DestroyScrollContext|INF|0|206|Failed to destroy scroll context for id DnF1ZXJ5VGhlbkZldGNoAwAAAAAATodDFlJhQnNKbVI5UWY2WkpuMGtsQUlURGcAAAAAAGzilxZ3Z1lLdU9WdlFvMk8wYU5CTzdJeER3AAAAAABs4pgWd2dZS3VPVnZRbzJPMGFOQk83SXhEdw== with exception: Elasticsearch.Net.ElasticsearchClientException: The remote server returned an error: (404) Not Found.. Call: Status code 404 from: DELETE /_search/scroll ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.HttpWebRequest.GetResponse()
at Elasticsearch.Net.HttpWebRequestConnection.Request[TResponse](RequestData requestData)
--- End of inner exception stack trace ---
at Elasticsearch.Net.Transport`1.HandleElasticsearchClientException(RequestData data, Exception clientException, IElasticsearchResponse response)
at Elasticsearch.Net.Transport`1.FinalizeResponse[TResponse](RequestData requestData, IRequestPipeline pipeline, List`1 seenExceptions, TResponse response)
at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
at Elasticsearch.Net.ElasticLowLevelClient.ClearScroll[TResponse](PostData body, ClearScrollRequestParameters requestParameters)
at Nest.ElasticClient.Nest.IHighLevelToLowLevelDispatcher.Dispatch[TRequest,TQueryString,TResponse](TRequest request, Func`3 responseGenerator, Func`3 dispatch)
at Skyline.DataMiner.SLSearch.Elastic6.StorageTypes.ElasticStorageType`1.DestroyScrollContext(String scrollId)
Hi Arunkrishna,
First of all some context: we use the Elastic scroll API to fetch Elastic data in pages. These scrolls have to be cleaned up when we finish using them as they keep a copy of the data in memory on the Elastic server. If we don't do that those scrolls are removed after 15 minutes (by default you can only have 500 active scrolls).
The exception you're seeing is caused by triggering another delete on an already deleted scroll. Nothing to worry about.
As a side note: I think it makes more sense to put time into replacing the scroll API by the search_after parameter (as suggested by the Elastic docs), for which we already have a task on our backlog, than to investigate the origin of this problem.
Let me know if you have any other questions!
Kind regards,
Xander