Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Tutorials
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • Integration Studio
      • System Architecture
      • DataMiner Releases & Updates
      • DataMiner Apps
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

How do Cassandra Snapshots affect system hard drive space and Dataminer backups failing?

Solved1.26K views16th March 2023Cassandra Cassandra snapshots Skyline DataMiner Backup
3
Ryan Reuss [SLC] [DevOps Member]490 9th December 2020 0 Comments

I have a Dataminer system that the Cassandra snapshots are not being deleted on the time trace table. This causes the file explorer to represent a time trace folder of over 80GB. Drilling down, I notice the snapshot folder inside this table is showing 68GB(5 snapshot folders of around 14GB each). I was under the assumption that a snapshot is only a reference to this memory and not holding actual space as when I check the size in node tool these snapshots are not containing any size.

My question is - Is this related to my backups failing on this server because it is indicating we are out of space, when running the windirstat I can only see the time trace snapshots taking up most of the HD?

The timetrace table is only around 13GB without snapshots.

I am in the process of manually clearing the snapshots and figuring out why they aren't being cleared on a separate note.

Ryan Reuss [SLC] [DevOps Member] Selected answer as best 16th March 2023

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
3
Michiel Saelen [SLC] [DevOps Enabler]5.63K Posted 11th December 2020 0 Comments

Hi Ryan,

Probably you will find entries in the log files of Cassandra mentioning that it was not able to delete the snapshots.
E.g. WARN [RMI TCP Connection(298)-127.0.0.1] 2019-04-23 20:01:33,328 SnapshotDeletingTask.java:43 - Failed to delete snapshot [C:\ProgramData\Cassandra\SLDMADB\data-47e9f990bfc811e7823dab7ff8019ae2\snapshots\dataminer-307c274c-5470-4ee7-837b-39bb95ce534c]. Will retry after further sstable deletions. Folder will be deleted on JVM shutdown or next node restart on crash.

There is a known problem problem for Cassandra running on Cassandra that prevents snapshots to be deleted. Due to the way Windows file system and snapshots work, it is possible that some of the snapshots can't be deleted. Cassandra could still have a file handle open. When this occurs, the error above will be shown and Cassandra will eventually delete it (once the sstable is compacted away, or after a Cassandra restart).

Cassandra creates hardlinks to manage the backups. this means the data is saved once on disk, and has multiple references to it. In essence they should only take the amount of space once (the listsnapshots will tell you the truesize of the snapshot ). It is not sure how windows handles these calculations (and windirstats), if they measure them twice, or not.

In the end, Cassandra will fail eventually on deleting snapshots and will keep retrying, but it may take multiple compactions until it finally manages to delete the snapshots. If space is really an issue and you need to get rid of the snapshot, you can always quickly reboot the Cassandra after doing a clearsnaphot (this will 100% definitely, no exception delete all snapshots).

Let us know if this would not solve your problem

Ryan Reuss [SLC] [DevOps Member] Selected answer as best 16th March 2023
You are viewing 1 out of 1 answers, click here to view all answers.
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas

Recent questions

Correlation Engine: “Test rule” doesn’t result in a hit, despite functional rule 1 Answer | 1 Vote
When using the Setter = true attribute, will the copy action always be executed first? 1 Answer | 2 Votes
Multiple Set on Table parameters for DVE’s 1 Answer | 2 Votes

Question Tags

adl2099 (115) alarm (62) Alarm Console (82) alarms (100) alarm template (83) Automation (223) automation scipt (111) Automation script (167) backup (71) Cassandra (180) Connector (109) Correlation (69) Correlation rule (52) Cube (150) Dashboard (194) Dashboards (188) database (83) DataMiner Cube (57) DIS (81) DMS (71) DOM (140) driver (65) DVE (56) Elastic (83) Elasticsearch (115) elements (80) Failover (104) GQI (159) HTTP (76) IDP (74) LCA (152) low code app (166) low code apps (93) lowcodeapps (75) MySQL (53) protocol (203) QAction (83) security (88) SNMP (86) SRM (337) table (54) trending (87) upgrade (62) Visio (539) Visual Overview (345)
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin