We’ve seen a situation before whereby Dataminer Error alarms (Dataminer Run-time) get stuck in Cube after we resolve the error (by restarting/failing over the DMA) – but restarting Cube clears these alarms.
Today we have had the opposite happen, the Error alarms are still present on the html5 dashboard, but are not present in Cube’s console window. This is displayed regardless of which DMA the web browser is connected via.
Has anyone seen this situation before?
We’ve seen alarms staying active before, but not the way you describe it that they stay in the web apps while they are gone in Cube.
The active alarms are being cached in the WebAPIs hosted in IIS on the DMA server. This does not restart when you restart DataMiner, although it should reinitialize when the DMA comes active (maybe something goes wrong here). An interesting test would be to restart IIS on the DMA server, do you then still get those alarms?
Glad to hear! If this would happen again, or if you would be able to reproduce it, then please let us know so we can look further into it.
As a followup comment, Support have pointed out that a bug was fixed in 10.2.0 CU8, related to closed alarms not being migrated to the Elastic dms-alarms index.
Which might have been the route cause of the issue.
We have resolved it before we could try your suggestion. The alarms were raised from DMA-D, which was also restarted when we fixed the original error. Our dashboards normally connect to DMA-A or DMA-C. When we connected our browser to the dashboard on DMA-D, the alarms cleared. At this point, refreshing any dashboard connected to DMA-A or DMA-C also resolved them.
It sounds like we achieved something similar to what you were saying around alarms being cached and the act of connecting to DMA-D cleared the cached information. Thanks for your suggestion, it was going to be the next thing we were going to try 🙂