DSM Reporting (new module): Ability to Report on Removed Elements or Table Records

Solved1.62K viewsDashboards data retention Reports
3
1 Comment

What would be the recommended approach to have the ability to report on elements that have already been deleted from DM or table records that have been removed from a table?

After some discussions we have come up with 2 possibilities, but we are looking for a greater exposure to the use case.

Report on Deleted Table Records

  1. Protocol level: We could implement a mechanism to not delete the table records for x amount of time (similar to DVE handling)
    1. This option, while easy to implement, imposes countless possibilities of things that could go wrong and we end up losing the data stored in the table. It also adds complexity to the handling of data within a protocol, which should be focused on polling performance instead
  2. Software feature to be able to report on deleted entries from within the specified data retention range (RT, AVG 5 mins, etc)
    1. This could be an option in the new reports and dashboards module, via which we could be able to report on table records (trended) that are no longer there but that are still within the data retention period

Report on Deleted Element (decommissioned device or platform)

  1. The easy approach is to advise that the element is not removed and in combination with the Report on Deleted Table Records features we could provide the functionality
    1. The one drawback we see here is that customers would have to keep the elements within the system for reporting purposes only
  2. Software feature to run reports on deleted elements and element records that fall within the configured data retention configuration

Your ideas and insights are appreciated.

Thank you,

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 11th July 2023
Gelber Mahecha [SLC] [DevOps Enabler] commented

Could it be possible to expand the use case(s) under which such reporting is required?

2 Answers

3
12.94K 1 Comment

Hi Rene

Regarding the deleted table records, if this is just for a few critical elements then it could make sense to introduce that feature as part of the protocol.

But if you need to have it for a table in general, then it would be better to be a software feature that way the changes needed to adapt protocols would be fewer.

Regarding the report on deleted elements, the IDP solution has a feature that allows it to detect when an element, that was previously managed by the solution, has been deleted.

Depending on the type of information you need on your report this could work for your use case.

Also, as part of the IDP solution, there is also the option of decommissioning an element with the following option.

This will allow you to execute a custom script that can perform your decommissioning flow and if needed generate a report.

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 11th July 2023
Rene De Posada [SLC] [DevOps Advocate] commented

Thank you for your feedback. This IDP feature is definitely something that we can explore a part of the complete setup.

To expand a bit more on the use case:

The customer is interested in being able to report on removed entities (Entries from tables or entire elements) and the target parameters are those that had trending data available at the time of the removal, obviously. We also need to be able to report on alarms affecting the entities (parameters, element), of course. The specific request is for 2 years, but we could use the native data retention mechanisms with some enhancements if necessary.

We have had a few discussions with the developers, and given that this is an EPM (former CPE) environment. Handling this request at the protocol level seems to us like a liability in the long term.

Your response is appreciated.

Thank you,
2
1.16K 1 Comment

Rene,

First, let me explain the difference in database:

  • MySQL:
    • Performance data (trend)
      • when an element is deleted, the performance data is deleted as well.  (data_x and dataavg_x) tables.
      • when a row is deleted, the performance data fades out
    • Fault data (alarms)
      • faded out
  • Cassandra (single node and cluster)
    • Both performance and fault data is always faded out

This means that when it concerns data that fades out, that all the information is still available in the database.

What information is reachable:

  • Performance data is  accessible in DataMiner Cube for metrics where the trending has been disabled after a while.  You can drill down on the parameter, and in the trending tab hit 'Load' to see the data which is still available in the database.
  • Fault data is also still visible in DataMiner Cube even if your element or row was removed from the system. (Alarm history query)
  • There is an alarm-list component in soft launch in dashboards.  This component also has the possibility to do a history query, and shows alarms from deleted rows and elements.

Now, for your use-case, how do you want to access this information?  As the biggest problem will probably be which queries to execute.

Hope this helps you further.

Rene De Posada [SLC] [DevOps Advocate] Posted new comment 31st October 2020
Rene De Posada [SLC] [DevOps Advocate] commented

Jan – Thanks for your response. The use case is simple: Ability to use the dashboards to report on KPIs (trended parameters), even after an element has been removed or an entry from a table has been deleted.

The most simple approach seems to be the integration of a setting on a specific set of dashboard components that allow for the inclusion of deleted elements or table entries.

The time for which deleted elements/ table entries would remain in the Database could be incorporated alongside the existing settings towards data retention.

The same applies for Alarms but as you mentioned, the alarm component in dashboards seems to already cover the capability.

Thank you,