Hello,
I'm trying to set up a several DMAs with external Elastic cluster. I set a necessary settings in DBConfiguration.xml
<DatabaseConfiguration>
<SearchConfiguration>
<ElasticConnections>
<ElasticCluster priorityOrder="0">
<Hosts>1.1.1.1,1.1.1.2,1.1.1.3</Hosts>
<Username />
<Password></Password>
<Prefix>dma_43201</Prefix>
<FileOffloadIdentifier>DM_Cluster</FileOffloadIdentifier>
</ElasticCluster>
</ElasticConnections>
</SearchConfiguration>
</DatabaseConfiguration>
After starting DMA I got a lot of errors like this :
2021/07/28 05:30:33.734|SLDBConnection|SLDataGateway.Repositories|INF|0|1|2021-07-28T05:30:33.724|DEBUG|DataGateway.Storage.Elastic.LogEntry (elastic_log_entry)|CreateIndexTemplateIfNotExists - Attempt 1/10 failed with exception: Elasticsearch.Net.ElasticsearchClientException: The remote server returned an error: (400) Bad Request.. Call: Status code 400 from: PUT /_template/dma_43201-logentries. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [indexhelper : {dynamic=true, dynamic_templates=[{stringfield={mapping={type=keyword, fields={search={fielddata=false, analyzer=SLSearch_analyzer, type=text}, lower={normalizer=SLSearch_normalizer, type=keyword}, lowerwhitespace={fielddata=false, analyzer=whitespace_lower, type=text}}}, match_mapping_type=string}}, {doublefield={mapping={type=double}, match_mapping_type=double}}, {longfield={mapping={type=long}, match_mapping_type=long}}]}]" CausedBy: "Type: mapper_parsing_exception Reason: "Root mapping definition has unsupported parameters: [indexhelper : {dynamic=true, dynamic_templates=[{stringfield={mapping={type=keyword, fields={search={fielddata=false, analyzer=SLSearch_analyzer, type=text}, lower={normalizer=SLSearch_normalizer, type=keyword}, lowerwhitespace={fielddata=false, analyzer=whitespace_lower, type=text}}}, match_mapping_type=string}}, {doublefield={mapping={type=double}, match_mapping_type=double}}, {longfield={mapping={type=long}, match_mapping_type=long}}]}]"" ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.GetResponse()
at Elasticsearch.Net.HttpWebRequestConnection.Request[TResponse](RequestData requestData)
--- End of inner exception stack trace -
I found that "mapper_parsing_exception" might appear during to client/server incompatibility. Current version (7.13.4) of Elasticsearch doesn't support mapping types as described here https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html
So the questions is : which version of Elasticsearch compatible with DM (I tried 10.1.8)? Have you tested it with 7.13?
Hi Anatoly,
Please see a similar question:
Can my DataMiner System use an existing Elastic cluster?
The version should match 6.x.x (preferably 6.8.x, so DataMiner can leverage the free security features).
Anatoly,
This is certainly under exploration, however this comes with many technical challenges. Note that there is huge list of breaking items by Elasticsearch https://www.elastic.co/guide/en/elasticsearch/reference/7.x/breaking-changes-7.0.html , so it will be reflected under the roadmap once we are at that stage.
Thanks, Miguel. I will try to move forward with 6.8 release.
Have you intention to update DM for Elastic 7.x in future versions?