Ceph由于更换IP地址导致MON异常

摘要: 由于机器要更换IP地址，更改重启后，各ceph监控进程出现异常，无法启动，本文用于记录并说明该问题。

引言

由于机器要更换IP地址，更改重启后，各ceph监控进程出现异常，无法启动，本文用于记录并说明该问题。

 

现象

更换IP地址后，直接启动mon无法启动，总是发现报绑定IP地址的错误。初步猜测为：
进程需要绑定本地的IP:PORT进行监听，地址已改，而绑定的还是修改前的地址，绑定失败，所以报错。

 

[root@gnop029-ct-zhejiang_wenzhou-16-20 ceph]# ./startmon.sh 
monmaptool: monmap file /etc/ceph/monmap
monmaptool: /etc/ceph/monmap exists, --clobber to overwrite
'/var/lib/ceph/mon/ceph-a' already exists and is not empty: monitor may already exist
=== mon.a === 
Starting Ceph mon.a on ceph-10...
2015-11-23 13:23:47.855887 7f4a5a36a7a0 -1 WARNING: 'mon addr' config option 101.71.4.20:6789/0 does not match monmap file
         continuing with monmap configuration
2015-11-23 13:23:47.856064 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:47.856085 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind. Trying again in 5 seconds 
2015-11-23 13:23:52.856281 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:52.856319 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind. Trying again in 5 seconds 
2015-11-23 13:23:57.856622 7f4a5a36a7a0 -1 accepter.accepter.bind unable to bind to 101.67.163.20:6789: (99) Cannot assign requested address
2015-11-23 13:23:57.856657 7f4a5a36a7a0 -1 accepter.accepter.bind was unable to bind after 3 attempts: (99) Cannot assign requested address
2015-11-23 13:23:57.856670 7f4a5a36a7a0 -1 unable to bind monitor to 101.67.163.20:6789/0
failed: 'ssh ceph-10 ulimit -n 32768;  /usr/bin/ceph-mon -i a --pid-file /var/run/ceph/mon.a.pid -c /etc/ceph/ceph.conf --cluster ceph '

 

 

解决办法

修改ceph.conf配置文件，并重新生成monmap文件，即可启动mon进程

重新生成monmap文件

 

sudo monmaptool --create --generate -c /etc/ceph/ceph.conf /etc/ceph/monmap

 

也可以通过命令得到新的monmap

monmaptool --create --add mon.a 101.71.4.20:6789 --add mon.b 101.71.4.21:6789   --add mon.c 101.71.4.22:6789   --add mon.d 101.71.4.23:6789    --add mon.e 101.71.4.24:6789 		--fsid c6e7e7d9-2b91-4550-80b0-6fa46d0644f6   --clobber monmap

monmaptool --print monmap

得到monmap后，重新注入monmap

ceph-mon -i monid --inject-monmap monmap

按照上述办法，即可解决IP地址修改导致监控进程无法启动的问题。

[root@ceph0 ceph]# sudo /etc/init.d/ceph -a start mon.ceph0
=== mon.ceph0 === 
Starting Ceph mon.ceph0 on ceph0...
Starting ceph-create-keys on ceph0...

[root@ceph0 ceph]# ps aux | grep mon
root      3768  1.0  0.0 226328 16784 pts/0    Sl   11:28   0:00 /usr/bin/ceph-mon -i ceph0 --pid-file /var/run/ceph/mon.ceph0.pid -c /etc/ceph/ceph.conf --cluster ceph