Hi,
In Dataminer 10.x we have a couple of DB tables with their name beginning with analytics_*, e.g.,
analytics_changepointsv1, analytics_wavestream, etc.
Where can we find more information about these tables? In which analytics features are they used and what is their purpose? How easily they can grow?
In the result below from the Cassandra BPA, one of these tables was signaled as having a large partition for a specific element. What shall we do?
Thanks in advance.
As the name suggests, these tables are indeed used by the analytics features. The large partition size for analytics_wavestream is likely caused by an element with a large table, as it stores a model for each (trended) entry in the table, and analytics_wavestream is partitioned per element. We are aware that partition sizes of around a 100 MiB can happen in some circumstances, but according to our Cassandra expert, this should not really cause problems in most cases. Moreover, we do not expect the partition size to grow over time, as there is just one model per trended parameter. There is also a limit on the number of parameters that are monitored by Behavioral Anomaly Detection.
However, if you want to reduce the partition size, a solution could be to try and split the data over multiple elements. I am not sure, but this might also help with some of the large partition sizes for other tables.
Here is an overview of what data is stored for which feature:
- analytics_alarmfocus: stores one model for Alarm Focus for each monitored parameter that had an alarm in the last two weeks, or each monitored table cell. The amount of data in the table should be more or less stable after two weeks.
- analytics_arrowwindows_v1: this is an old table that was used by the Trend Icons before version 10.2.4.
- analytics_changepointalarmentries: used to keep track of the suggestions/alarms created by Behavioral Anomaly Detection. The amount of data should not keep growing over time, as the TTL of each record is 8 days.
- analytics_changepoints_v1: keeps the detected changepoints for one year. In case of elements with large tables and a lot of changepoints, partition sizes can become quite large here. In more recent versions we moved to analytics_changepoints_v2 with smaller partitions (one partition per parameter or table cell).
- analytics_parameterinfo_v1: keeps some data per trended parameter that is used for updating models in several features. The amount of data should be stable.
- analytics_trendalarms_v1: this is an old table that is not used anymore in newer versions
- analytics_trendalarms: used to keep track of the suggestions/alarms created by Proactive Cap Detection. The amount of data should not keep growing over time, as the TTL of each record is 60 days.
- analytics_waveletlevels_v2: this is an old table that is not used anymore in newer versions.
- analytics_wavestream: keeps a model per trended parameter or table cell for Proactive Cap Detection and Behavioral Anomaly Detection. The amount of data is related to the number of monitored parameters and should not grow over time.
Great idea! It’s on our to-do list.
Excellent explanation Tobe! Many thanks.
Wondering if some of this info could be added to our online documentation, e.g., in here: https://docs.dataminer.services/develop/devguide/Connector/AdvancedDataMinerDataPersistenceNoSqlCassandra.html?q=nosql