I am trying to configure an alarm template which would raise an alarm when a normally varying signal (in this case representing an audio signal level) suddenly stops its normal pattern and remains fixed at a constant value. This would seem to be a classic example for Flatline anomaly detection. However I can't determine exactly what type and duration of flatline behaviour will actually raise a Flatline anomaly alarm.
To test this, I wrote a simple virtual protocol which creates a varying value by using a pseudo random number in a QAction to update a parameter, but added an additional configuration parameter that allowed me to fix the value at the most recent value and not update it, i.e. the normally varying signal now becomes a flatline. However despite trying all options of alarm threshold types (Normal, Relative, Absolute or Rate) in the alarm template for the level parameter, I cannot get an Flatline anomaly alarm to be raised. I even tried in the protocol between explicitly setting the parameter value in the QAction with the same value versus just not updating it in the flatline case, but still no luck.
The only way I can reliably create a alarm when the level stops varying seems to be use an alarm template with an alarm threshold type of rate with an alarm high (or low) level of delta 0 and a Hysteresis On setting of sufficient delay to ensure that it doesn't raise an alarm for the "normal" varying signal. Unfortunately the alarm description in this case is not very descriptive for the flatline condition we are trying to detect. I guess I could use a correlation rule trigger for the rate alarm to add a new alarm with a more meaningful textual description, but this seems to be a very convoluted method for what should apparently be provided by the anomaly detection system.
For information the DataMiner system is running 10.3.0.0.-13902-CU11.
Is there something obvious I am missing?
Hi Paul,
The current flatline detection functionality is based on a model that learns the normal range of time intervals between value updates of a parameter. When a value update takes too long and is significantly longer than expected by the model, a flatline event is triggered.
In your version 10.3.0.0.-13902-CU11, the model monitored the time interval between parameter value updates. In more recent versions, this was changed to monitor the time interval between actual value changes.
The model is updated in real-time with incoming parameter changes. However, it needs to receive a sufficient amount of parameter changes before it can detect and trigger flatline events. In your experiment, the algorithm/model might not have had enough time to learn: the required learning period depends on the polling time, but typically, about one day of varying data is needed before a flatline change point can be detected. Additionally, a flatline change point will only be considered anomalous (leading to an alarm or suggestion event) if there is at least one week of data on the parameter.
We understand that this may not yet meet your needs, and we are currently considering improvements and next steps. Your feedback is very much welcomed. The duration of flatline behavior cannot yet be specified, but we are considering adding this extra configuration to the anomaly alarming settings. If the user could specify a duration for flatline behavior to be considered anomalous, the model would no longer need a learning period to detect it, allowing events to be triggered earlier.
As a side note: another option for you might be to tag the behavioral pattern of varying behavior changing into flatline behavior and let the pattern matching feature detect new occurrences. Working with pattern matching | DataMiner docs.
Hi Paul,
There are a few considerations when setting up the anomaly detection, which might be causing your configuration not to render the expected results. Below are some of them:
- Ensure that the target parameters are trended (real time and average)
- Anomalies are created as suggestions events by default and they need be turned into alarms via the alarm template. See (Configuring anomaly detection alarms for specific parameters | DataMiner Docs)
- The anomaly detection sensitivity changes depending on the DataMiner version. Above link also explains this.
Please also see below resources for more information on anomaly detection:
- Working with behavioral anomaly detection | DataMiner Docs
- Behavioral anomaly detection | DataMiner Docs
Thanks,
Thanks for comments Rene, but I had already read these documents and configured the alarm template as indicated etc.
The only reference as to the actual conditions that will trigger a flatline anomaly is that it seems that the condition will take need to persist for an hour, which is too long for the case I am trying to detect. (That said it didn’t even trigger after an hour with my test protocol, so who knows what the internal logic is!).
Ideally for the flatline anomaly detection to be useful, you need to be able to specify the duration of the flatline scenario you are trying to detect.
At this point it looks like I will have to come up with an alternative approach.