Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Updates & Insights
  • Questions
  • Learning
    • E-learning Courses
    • Empower Replay: Limited Edition
    • Tutorials
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
      • About the DevOps Program
      • DataMiner DevOps Support
  • Downloads
  • More
    • DataMiner Releases & Updates
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

How to truncate trend data

Solved515 views16th October 2024Cassandra trending
5
paul farndell [DevOps Advocate]411 30th September 2024 0 Comments

How can you delete any trend data older than X days from a specific element in a local Cassandra setup?

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 16th October 2024

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
1
Alexander Verriest [SLC] [DevOps Advocate]531 Posted 11th October 2024 0 Comments

Hi Paul,

In System Settings – time to live you can specify how long you wish specific data types to be kept in the database for.

This can be configured globally, or per protocol (version).

Setting this will ensure that the data written to Cassandra has a time to live (ttl) set, and will be marked for deletion after this period has passed. Note that this does not affect any data already present prior to setting this value.

Your question is about deleting it for a specific element though, so unless there are no other elements sharing the same protocol version, it is not possible to configure this in DataMiner.

You can however manually send a query to the Cassandra database to delete specific data you want.

In System Center – Tools, there is a Query Executor you can use to send queries to your local database. A query to request trend points looks like this. The Cassandra table we want to query is the “data” table:
select * from data where d=[DMAID] and ei=[ELEMENTID] and p=[PARAMETER] and i='[TABLEINDEX]’ and w=[0, 5 or 60 depending on whether you wish to query real time points, 5min averaged points or 1hour averaged points] and t > ‘[Timestamp]’;

d, ei, p, i, w and t are all Partitioning Keys in Cassandra, and I don’t believe you can send a delete statement using range queries (< or >) on a Partitioning Key column in Cassandra.
You would have to create a script that fetches the rows that match a range query first, to then send separate delete queries for each row separately. This will probably be a heavy operation if there are a lots of rows to go through.

Naturally you should be very careful in doing this and only delete the data you know is safe to delete.

Hope this helps!

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 16th October 2024
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs