Configuring High Availability Clusters
Cisco Nexus Dashboard Data Broker supports high availability clustering in active/active mode with up to five controllers. To use high availability clustering with Cisco Nexus Dashboard Data Broker, you must edit the config.ini file for each instance of Cisco Nexus Dashboard Data Broker.
NDDB release 3.10.4 supports 3-node clusters only.
In case of a split-brain scenario, 3-node clusters are handled as follows:
Cluster health is indicated as Yellow. A minumum of two nodes should be up and connected in the cluster for the cluster to be in the operational state. If not, the cluster nodes will move to a non-operational state. Override option is not available. Fix the VM and/or network link, as required.
Note |
IPv6 is supported in centralized Nexus Dashboard Data Broker mode only, it is not supported in Embedded mode. |
Cluster Indicator |
Cluster Status |
Recommendation |
---|---|---|
Green |
Operational |
No recommendation as the status is operational. |
Yellow |
Some of the cluster nodes are not available |
Do not make any changes or add to the existing Nexus Dashboard Data Broker configuration. |
Red |
The node is isolated from the cluster. |
Do not make any changes or add to the existing Nexus Dashboard Data Broker configuration. Note: For two node cluster, you need to override in any one of the cluster node only, to ensure regular operation. |
Before you begin
-
All IP addresses must be reachable and capable of communicating with each other.
-
All switches in the cluster must connect to all of the controllers.
-
All controllers must have the same HA clustering configuration information in the config.ini files.
-
All controllers must have the same information in the ndb/configuration/startup directory.
-
If using cluster passwords, all controllers must have the same password configured in the ndbjgroups.xml file.
-
To mark a node as a preferred primary, add the required node IP address as the first node in the list of supernodes in the config.ini file. To change the preferred primary node, stop the ndb controller and make the required modifications to the config.ini file.
Note |
The data broker controller checks for the number of configured supernodes (in the config.ini file), and if the number is less than three, an error is displayed indicating that 2-node cluster is not supported. |
Procedure
Step 1 |
Open a command window on one of the instances in the cluster. |
Step 2 |
Navigate to the ndb/configuration directory that was created when you installed the software. |
Step 3 |
Use any text editor to open the config.ini file. |
Step 4 |
Locate the following text:
|
Step 5 |
Example:IPv4 example.
Example:IPv6 example.
|
Step 6 |
Save the file and exit the editor. |
Step 7 |
Repeat Step 3 through Step 7 for each instance of Cisco Nexus Dashboard Data Broker in the cluster. |
Step 8 |
Restart Cisco Nexus Dashboard Data Broker. For Nexus Dashboard Data Broker cluster deployment, the expected latency between the node is three seconds, with three retries. You can configure the latency time and the number of maximum retries. See the procedure, below. |
What to do next
(Optional) Use this procedure to configure the delay time for a node and the number of retries.
-
Open a command window on one of the instances in the cluster.
-
Navigate to the ndb configuration directory.
-
Use any text editor to open the ndbjgroups.xml file.
-
Locate the following text:
FD timeout="3000" max_tries="3"/
-
Modify the Latency Time value and maximum_tries value.
-
Save the file and exit the editor.
-
Repeat the above steps for all the instances of the cluster.
Password Protecting High Availability Clusters
Procedure
Step 1 |
Open a command window on one of the instances in the cluster. |
Step 2 |
Navigate to the ndb/configuration directory. |
Step 3 |
Use any text editor to open the ndbjgroups.xml file. |
Step 4 |
Locate the following text:
|
Step 5 |
Remove the comments from the AUTH line. Example:
|
Step 6 |
(Optional) Change the password in the auth_value attribute. |
Step 7 |
Save the file and exit the editor. |
Adding a Standby Node
Beginning with release 3.10.4, you can add a standby node to support a cluster. You need to append -standby
to the IP address of the standby node to indicate the standby node while configuring the supernodes in the config.ini
file. An example is shown here:
supernodes=<ip1>;<ip2>;<ip3>;<ip4>-standby
Release 3.10.4 supports 3-node clusters only. The cluster is said to be fully healthy if all the three nodes are working fine, partially healthy if two (out of the three nodes) are working fine. If only one node is working fine, that is, two nodes (out of the three
nodes of a cluster) are down, then, the cluster is unhealthy. You need to manually start
the standby node, which forms a cluster with the running node.
The standby node terminates automatically:
-
If the running node gets terminated abruptly.
-
After recovery, if the two nodes that were earlier down are now up.
Guidelines and limitations for standby nodes
-
You cannot
start
the standby node, if all the nodes in a cluster are down. One node needs to be running to form a cluster with the standby node. -
You cannot
start
the standby node, if the cluster is healthy (that is, if two nodes in a three node cluster are working fine). -
Ensure that the cluster is unhealthy before starting the standby node. Ensure that the nodes of the cluster are down, and not a disruption in connection between the nodes.
Consider a scenario wherein, node 1 and node 2 are located together and node 3 and the standby node are located together elsewhere. If nodes 1 and 2 are temporarily disconnected from node 3 and the standby node, it gives a false impression that nodes 1 and 2 are down, and based on this information, if the standby node is started, then the standby node and node 3 form a cluster (even when nodes 1,2 are up). This will lead to a configuration mismatch/ loss when the connection between node 1- node 2 and node 3-standby node is restored.