Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Tutorials
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • Integration Studio
      • System Architecture
      • DataMiner Releases & Updates
      • DataMiner Apps
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

DataMiner and Cassandra Latency KPIs

Solved1.89K views16th June 2024Cassandra DataMiner latency specs
4
Rene De Posada [SLC] [DevOps Advocate]1.89K 18th June 2020 0 Comments

All,

We have been doing some digging through the documentation trying to find recommended latency values for:

1- DMA-DMA
2- Cassandra Node - Cassandra Node
3- Cassandra Seeds/Nodes- DMA/DMS

The only recommendation we have found so far is that of the Indexing Engine (50ms). Could someone here indicate if Skyline has any KPIs that could be used as a guide when installing a DataMiner System? This becomes particularly important once we start supporting the centralized Cassandra clusters, since the cluster could be composed of DMAs and nodes hosted in different data centers.

Thank you in advance,

Rene De Posada [SLC] [DevOps Advocate] Selected answer as best 7th July 2020

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
4
Michiel Saelen [SLC] [DevOps Enabler]5.63K Posted 18th June 2020 2 Comments

Hi Rene,

For Cassandra there are two important ways of communication between nodes:

  • Internode communications (gossip)
    This is needed to provide information between nodes on the Cassandra cluster in a scalable way.
    To have good gossip between nodes it is important that you have the same seed list in your config on each node (cassandra.yaml file).
    Making every node a seed node is NOT recommended (they recommend three for every DataCenter) => this is hard to achieve with current Cassandra architecture.
    A node can be marked down (to prevent a coordinator node to send to this one on every query request).
    The threshold to mark a node as down is the phi_convict_threshold (located in the cassandra.yaml file).
    The threshold takes into account network performance, workload, and historical conditions.
    In simple words: "The node is marked down if the node does not respond within a certain time. This time is the average time * phi_convict_threshold."
  • Requests (e.g. Read Requests)
    A coordinator will wait on another node to handle the request based on a timeout time in ms (settings available in the cassandra.yaml file e.g. read_request_timeout_in_ms). This is important for a query to succeed or not.
    If you want to make sure you are working with the latest data stored you need to use the Quorum consistency level on your queries. Meaning that more than 50% of your nodes that hold your data (based on replication factor) will need to answer. In case of a FO setup we build currently a Cassandra cluster of 2 nodes with a replication factor of two, so if we would use Quorum we would need both nodes to be online (or not marked as down) all the time. That is why we use consistency level 1 at the moment.

So in general you can have any latency between your Cassandra nodes, however based on your configuration it will or will not give problems.

Rene De Posada [SLC] [DevOps Advocate] Selected answer as best 7th July 2020
Rene De Posada [SLC] [DevOps Advocate] commented 18th June 2020

Michiel – Thank you for your reply. A quick clarification would be: Are all the settings mentioned configurable by us? Meaning, can we set any latency value we want once we learn about the latency expected between data centers? Also, you are now mentioning 3 seeds per data center, but I assume this will become a reality once we go into the centralized architecture. What would be the recommendation for a system where each DMA will have 3 Cassandra nodes associated? Will replication factor 3 still be convenient?

Michiel Saelen [SLC] [DevOps Enabler] commented 30th June 2020

Hi Rene – The default settings that are configured should work in normal circumstances.
If there is a need to change these settings, I believe you can also expect some degradation in capabilities, so if possible, try to improve your network instead. When the listen address of your node is listed as a seed it means that the node will start up without contacting a seed in the cluster. If you want to replace a seed node (with the same ip) you first need to remove the ip of the new node from the seed list on all the nodes before you add it to the Cassandra Cluster. In other words a seed is very useful into larger clusters to minimize the communication needed to bring a node online or to join/remove nodes, however we need to be careful when performing actions on seed nodes itself. For replication factor they advise mostly to have at least 3 as from that point you can be consistent with your data and cope with a node going down. To be consistent in your data you need to write/read to more than 50% (R+W >100%) of your nodes to make sure you have the latest data. From the moment your data is replicated/available on three nodes you can cope with one node being done and still have more than 50% online to be consistent. You can have a replication factor of 3 from the moment you have 3 nodes, in this case all nodes will contain 100% of all the tokens.

Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas

Recent questions

Alarm Dashboard PDF/CSV Export 0 Answers | 0 Votes
Is the Microsoft SharePoint Connector Still Usable 0 Answers | 0 Votes
Is the Microsoft SharePoint Connector Still Usable 0 Answers | 0 Votes

Question Tags

adl2099 (115) alarm (62) Alarm Console (82) alarms (100) alarm template (83) Automation (223) automation scipt (111) Automation script (167) backup (71) Cassandra (180) Connector (108) Correlation (68) Cube (150) Dashboard (194) Dashboards (188) database (83) DataMiner Cube (57) DIS (81) DMS (71) DOM (139) driver (65) DVE (55) Elastic (83) Elasticsearch (115) elements (80) Failover (104) GQI (159) HTTP (76) IDP (74) LCA (151) low code app (166) low code apps (93) lowcodeapps (75) MySQL (53) protocol (203) QAction (83) security (88) services (51) SNMP (86) SRM (337) table (54) trending (87) upgrade (62) Visio (539) Visual Overview (345)
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin