DRBD split-brain solution in primary/primary setup

Recently one of our customers DRBD cluster has detected split-brain:

node2 kernel: block drbd1: Split-Brain detected, dropping connection!
node2 kernel: block drbd1: helper command: /sbin/drbdadm split-brain minor-1
node2 kernel: block drbd1: helper command: /sbin/drbdadm split-brain minor-1 exit code 0 (0x0)

This is  primary/primary cluster. And to fix this issue we had to perform manual procedures since we don’t use automatic recovery feature.
According to the DRDB official documentation the solution is next.

drbdadm secondary
drbdadm connect --discard-my-data

The problem is drbdadm doesn’t have  ‘–discard-my-data’  option and exits with error:

[root@node2 ~]# drbdadm connect --discard-my-data wwwdata
drbdadm: unrecognized option '--discard-my-data'
try 'drbdadm help'
[root@node2 ~]#

The issue can be solved in two ways.

1. To perform command this way to pass this option directly to drbdsetup:

drbdadm -- --discard-my-data connect wwwdata

2.  To use option –verbose to get low-level drbdsetup output and run it with ‘–discard-my-data’ :

drbdsetup 1 net 10.209.1.1:7789 10.209.1.2:7789 C --set-defaults --create-device --allow-two-primaries  --discard-my-data

After DRBD will sync you can change role of split-brain victim to primary.

drbdadm primary wwwdata

Problem solved.

[root@node2 ~]# cat /proc/drbd
version: 8.3.7 (api:88/proto:86-91)
GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by root@node2.example.com, 2010-01-28 08:37:51
1:cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r----
ns:125609294 nr:194649913 dw:320259208 dr:36731102 al:34129 bm:4042 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0
[root@node2 ~]#

Didn’t find the answer to your question? Ask it our administrators to reply we will publish on website.

Leave a Reply

Your email address will not be published.