Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Tutorials
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • Integration Studio
      • System Architecture
      • DataMiner Releases & Updates
      • DataMiner Apps
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

Migration to DB Cluster ( Cassandra and Elastic) doubts

Solved1.05K views19th June 2023Cassandra DB database migration Elasticsearch
4
Bruno Sousa [DevOps Member]669 7th June 2023 0 Comments

Hello

We are currently migrating from the "per DMA" local databases to a "centralized cluster" databases approach, both for Cassandra and Elastic.

We need to have two databases instances in the Cassandra and Elastic DB centralized clusters though: one database for the pre-prod Dataminer instance and other for the production Dataminer instance.

We are starting with the migration of the pre-prod dataminer instance to these centralized clusters.

We followed the migration procedure and some doubts arose:

1- We have multiple keyspaces (one per table) in the Cassandra cenralized cluster after the migration. Is supposed to be this way?

2- We don't have alarm tables in the Cassandra centralized cluster. Is supposed to be this way? All alarms will only be stored in the Elastic Search centralized instance?

3- From now on, all the data (both trend and alarm) will only be kept in the centralized cluster and not on the local DMA databases right?

Thank you.

Best regards

Bruno Sousa

Bruno Sousa [DevOps Member] Selected answer as best 19th June 2023

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
3
Michiel Saelen [SLC] [DevOps Enabler]5.63K Posted 19th June 2023 1 Comment

Hi Bruno,

1 - Yes they created more keyspaces for the tables, this is to have more freedom to configure what replication strategy/factor you want to apply for every set of data. We did get complaints about the amount of keyspaces to manage, so we are looking to combine again a couple of tables into a known set of keyspaces. This will make it easier to manage the user permissions on the keyspaces if you do this.

2 - Yes, Alarms have been migrated towards Elastic. The reason for this is that we have to retrieve alarms in many different ways (e.g. based on view level, element level, service level, ...) in order to do this efficiently we stored the alarms multiple times in Cassandra with different keys for retrieval. This was causing some overhead and for systems with a lot of alarms (and big alarms), we saw a heavy load on Cassandra because of this. So the decision was made to move it to Elastic which allows us to use the indexing capabilities from Elastic instead of storing it multiple times.

3 - Once your data is migrated from the local DB to the central DB, the local DB is no longer needed. And as you probably know already the centralized DB has many benefits scaling independently, swarming capabilities in upcoming releases, ...

Bruno Sousa [DevOps Member] Posted new comment 19th June 2023
Bruno Sousa [DevOps Member] commented 19th June 2023

Hi Michiel,

Thank you for the clarification!

Best regards
Bruno Sousa

You are viewing 1 out of 1 answers, click here to view all answers.
Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas

Recent questions

Web Applications exception in Cube due to invalid certificate 0 Answers | 0 Votes
Redundancy Groups and Alarming – Duplicate Alarms 0 Answers | 0 Votes
Correlation Engine: “Test rule” doesn’t result in a hit, despite functional rule 1 Answer | 3 Votes

Question Tags

adl2099 (115) alarm (62) Alarm Console (82) alarms (100) alarm template (83) Automation (223) automation scipt (111) Automation script (167) backup (71) Cassandra (180) Connector (109) Correlation (69) Correlation rule (52) Cube (151) Dashboard (194) Dashboards (188) database (83) DataMiner Cube (57) DIS (81) DMS (71) DOM (140) driver (65) DVE (56) Elastic (83) Elasticsearch (115) elements (80) Failover (104) GQI (159) HTTP (76) IDP (74) LCA (152) low code app (166) low code apps (93) lowcodeapps (75) MySQL (53) protocol (203) QAction (83) security (88) SNMP (86) SRM (337) table (54) trending (87) upgrade (62) Visio (539) Visual Overview (345)
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin