Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Tutorials
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • Integration Studio
      • System Architecture
      • DataMiner Releases & Updates
      • DataMiner Apps
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

Replace a dead node in a Cassandra cluster

Solved1.16K views19th July 2023Cassandra
1
Ive Herreman [SLC] [DevOps Enabler]13.50K 23rd February 2021 0 Comments

Hi,

I have a DataMiner environment that uses a Cassandra cluster with replication factor 2.
What is the best way to replace a dead node (e.g. in case of a server crash)?

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 19th July 2023

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
0
Brent Alleweireldt [SLC]1.53K Posted 23rd February 2021 2 Comments

Hey Ive,

Replacing a dead node is a 3-part process.Note that the following procedure is for the standard 2 node failover setup. Custom clusters might need a different process depending on their configuration.

  • Part 1: prepare the new node
    • Install cassandra
      1. Place the cassandra files under C:\Program files\cassandra (Currently the best way is to copy them from the other server, DO NO copy the "c:\program files\data" folder)
      2. Go to c:\program files\cassandra\conf\cassandra.yaml
      3. Set the following parameters
      • cluster_name: (name of the dataminer cluster !must be the same as the other node!)
      • data_file_directories: location to save data
      • seeds:Ip of this node and other node
      • disk_optimization_strategy: ssd or spinning
      • listen_address: Ip of this node
      • broadcast_rpc_address: IP of this node
      1. Save and close the file
      2. Open an elevated powershell prompt
      3. Navigate to C:\program files\cassandra\bin
      4. Execute $env:CASSANDRA_HOME = ‘C:\progra~1\Cassandra\’
      5. Execute $env:JAVA_HOME = ‘C:\progra~1\Cassandra\Java\’
      6. Execute cassandra.ps1 –install
      7. Open port 9042 and 7000 on the firewall
    • In the registry under "Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Apache Software Foundation\Procrun 2.0\cassandra\Parameters\Java" add "-Dcassandra.replace_address_first_boot=<dead_node_ip>" to the Options-key
    • In cassandra.yaml add or set "auto_bootstrap:true"
  • Part 2: Updating the old node
    • In the cassandra.yaml under "-seeds:" replace the ip of the dead node with the ip of the new one.
    • Restart the node
  • Part 3: Start the new node
    • The new node should automatically start bootstrapping the data: this can be followed with nodetool netstats
    • If the node was down longer than the max_hint_window_in_ms. then you must also run a nodetool repair

Hope this helps, if there any questions feel free to leave a comment.

Marieke Goethals [SLC] [DevOps Catalyst] Selected answer as best 19th July 2023
Ive Herreman [SLC] [DevOps Enabler] commented 24th February 2021

Thanks Brent,

What differences are there when I want to apply this in a Cassandra cluster containing 3+ nodes (not linked to failover).

Brent Alleweireldt [SLC] commented 24th February 2021

Then it will mainly depend if the dead node was a seed node or not.
If the dead node was a seed node, then you will have to execute “part 2” on all the other nodes.

If the dead node was a standard data node, then you can skip part 2.

It is also a good practice to check if other Yaml settings were changed besides the ones mentioned.

Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas

Recent questions

DOM Definition relations returned in Definition query 0 Answers | 0 Votes
Alarm Dashboard PDF/CSV Export 1 Answer | 0 Votes
Is the Microsoft SharePoint Connector Still Usable 0 Answers | 0 Votes

Question Tags

adl2099 (115) alarm (62) Alarm Console (82) alarms (100) alarm template (83) Automation (223) automation scipt (111) Automation script (167) backup (71) Cassandra (180) Connector (108) Correlation (68) Cube (150) Dashboard (194) Dashboards (188) database (83) DataMiner Cube (57) DIS (81) DMS (71) DOM (140) driver (65) DVE (55) Elastic (83) Elasticsearch (115) elements (80) Failover (104) GQI (159) HTTP (76) IDP (74) LCA (152) low code app (166) low code apps (93) lowcodeapps (75) MySQL (53) protocol (203) QAction (83) security (88) services (51) SNMP (86) SRM (337) table (54) trending (87) upgrade (62) Visio (539) Visual Overview (345)
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin