Hi Dojo,
I noticed something strange with the Correlation Engine for which I don’t know if this is default behavior.
My setup has a Correlation Rule that executes a script when an alarm occurs on a spine switch element.
When I create 3 alarms on my element, by changing the Interfaces Admin Status, I notice that the script is only executed once.
Only when I enable option “Trigger on single events. Don’t maintain active tree status.” my script is executed for every change. But in that case it’s also executed every time the alarm is cleared (I know that I can work around this by filtering on Alarm Severity).
My question is how far it’s normal behavior that my script is only executed once while 3 unique alarms are created?
All of this is by design.
For a correlation rule, all active alarm trees which match the specified alarm filter are being kept track of in a bucket. In your example, all alarm trees elements running the Arista Switch protocol will be kept track of in one single bucket.
For a bucket, the correlation engine rules execute their actions when the rule condition starts being true for the set of alarm trees inside. (if there are no conditions specified, the condition is true as soon as there is one alarm tree in the bucket). Clear actions (if any) are executed once the rule condition stops being true. If extra alarms are being added to (or removed from) the bucket while the condition is fulfilled, only actions marked as “Execute on base alarm updates” will execute.
In your case, the script will run when the set of active Arista Switch alarm trees contains at least one alarm on the “*SPINE*” element.
- On your first alarm, the rule will fire.
- On the second alarm, the rule will not fire as the bucket already fulfilled the condition
- On the third alarm, the rule will not fire as the bucket already fulfilled the condition
Note that in your case you’re only filtering on the protocol. This might cause other unwanted alarms to also be tracked in the bucket. Moving the “*SPINE*” element filter to the alarm filter section could help there.
Note that you can split up the incoming alarms into multiple buckets through the “group by” option, e.g. by element or by table index.
The “trigger on single events” option can also be used if the state keeping is not required (combined with a more precise alarm filter to prevent the same alarm tree from triggering the actions multiple times)
Hi Jens,
Since you are working with a table, have you tried grouping by table index?
Hi Miguel, it works when grouping by table index. But why is that? I would expect that unique alarms have exactly the same behavior without the need of adding alarm grouping.
Hi Jens,
When creating a correlation rule without any alarm grouping, all the alarms will be grouped by default in the same ‘bucket’. For that reason the correlation rule will only be triggered once. When adding alarm grouping (in your case grouped by table index) you will create a bucket for each row in the table, so actions will be executed per row
I’m so glad I found this – trying to figure out how we can leverage on this – but in that case, let’s say I have 2 different parameters in alarm on the same element, without alarm grouping, would they default to the same bucket? Or does it depend on how the filter is built?
Technically, I’d like to keep a different bucket (if it makes sense) for each different root alarm ID in the element, so that the rule can be reapplied for each new alarm on the element, but I can maintain the active tree status – not interested in having all the the different alarms, for different KPIs, grouped in the same alarm bucket, especially if my filter is built on “Parameter Description (by element)”
Hi Wouter, thank you for the very detailed explanation. Much appreciated!