LVM故障导致RHCS启动故障-创新互联

1、故障提示

抓取资源管理日志发现提示如下错误

tail -f /var/log/cluster/rgmanager.log

May  6 18:21:24 yktdb1 rgmanager[17425]: State change: Local UP

May  6 18:21:24 yktdb1 rgmanager[17425]: Starting stopped service service:yktoracle

May  6 18:21:24 yktdb1 rgmanager[18533]: [lvm] HA LVM:  Improper setup detected

May  6 18:21:24 yktdb1 rgmanager[18555]: [lvm] * "volume_list" not specified in lvm.conf.

May  6 18:21:24 yktdb1 rgmanager[17425]: start on lvm "yktoracledb" returned 1 (generic error)

May  6 18:21:24 yktdb1 rgmanager[17425]: #68: Failed to start service:yktoracle; return value: 1

May  6 18:21:24 yktdb1 rgmanager[17425]: Stopping service service:yktoracle

May  6 18:21:25 yktdb1 rgmanager[18586]: [script] Executing /etc/init.d/dbora stop

May  6 18:21:25 yktdb1 rgmanager[18682]: [fs] stop: Could not match /dev/yktoracledb/oracledblv with a real device

May  6 18:21:25 yktdb1 rgmanager[18720]: [lvm] HA LVM:  Improper setup detected

May  6 18:21:25 yktdb1 rgmanager[18742]: [lvm] * "volume_list" not specified in lvm.conf.

May  6 18:21:25 yktdb1 rgmanager[18778]: [lvm] Deactivating yktoracledb/oracledblv

May  6 18:21:25 yktdb1 rgmanager[18800]: [lvm] Making resilient : lvchange -an yktoracledb/oracledblv

May  6 18:21:25 yktdb1 rgmanager[18825]: [lvm] Resilient command: lvchange -an yktoracledb/oracledblv --config devices{filter=["a|/dev/mapper/LUN-1800G|","a|/dev/mappe

May  6 18:21:26 yktdb1 rgmanager[17425]: Service service:yktoracle is recovering

May  6 18:21:26 yktdb1 rgmanager[17425]: #71: Relocating failed service service:yktoracle

May  6 18:21:26 yktdb1 rgmanager[17425]: Service service:yktoracle is stopped

May  6 18:21:35 yktdb1 rgmanager[17425]: State change: 192.168.10.2 UP

May  6 18:21:35 yktdb1 rgmanager[17425]: Starting stopped service service:yktoracle

May  6 18:21:36 yktdb1 rgmanager[18886]: [lvm] HA LVM:  Improper setup detected

May  6 18:21:36 yktdb1 rgmanager[18908]: [lvm] * "volume_list" not specified in lvm.conf.

May  6 18:21:36 yktdb1 rgmanager[17425]: start on lvm "yktoracledb" returned 1 (generic error)

May  6 18:21:36 yktdb1 rgmanager[17425]: #68: Failed to start service:yktoracle; return value: 1

May  6 18:21:36 yktdb1 rgmanager[17425]: Stopping service service:yktoracle

May  6 18:21:36 yktdb1 rgmanager[18939]: [script] Executing /etc/init.d/dbora stop

May  6 18:21:36 yktdb1 rgmanager[19035]: [fs] stop: Could not match /dev/yktoracledb/oracledblv with a real device

May  6 18:21:36 yktdb1 rgmanager[19073]: [lvm] HA LVM:  Improper setup detected

May  6 18:21:37 yktdb1 rgmanager[19095]: [lvm] * "volume_list" not specified in lvm.conf.

May  6 18:21:37 yktdb1 rgmanager[19131]: [lvm] Deactivating yktoracledb/oracledblv

May  6 18:21:37 yktdb1 rgmanager[19153]: [lvm] Making resilient : lvchange -an yktoracledb/oracledblv

May  6 18:21:37 yktdb1 rgmanager[19178]: [lvm] Resilient command: lvchange -an yktoracledb/oracledblv --config devices{filter=["a|/dev/mapper/LUN-1800G|","a|/dev/mappe

May  6 18:21:37 yktdb1 rgmanager[17425]: Service service:yktoracle is recovering

May  6 18:21:37 yktdb1 rgmanager[17425]: #71: Relocating failed service service:yktoracle

May  6 18:21:39 yktdb1 rgmanager[17425]: Service service:yktoracle is stopped

查看lvdiskplay  对应的oracledblv 状态提示 Not available

在/dev/yktoraclevg/下面竟然没有这个oracledblv

除非把clvmd停止后才这个在/dev/yktoarclevg/里就可以看了

查了好多资料都不知道怎么回事

查到一个service clvmd status 后发现 集群 vg和lv都是显示none

这一下让我找到了问题所在

直接用命令vgchange -cy yktoracledb

在查看service clvmd status

[root@yktdb1 ~]#  service clvmd status

clvmd (pid  7550) 正在运行...

Clustered Volume Groups: yktoracledb

Active clustered Logical Volumes: oracledblv ysbaklv test

[root@yktdb1 ~]#

已经可以看见集群共享的vg和lv了

在查看集群状态正常了服务也启动了，然后对这个两个节点测试是否可以正常切换。

[root@yktdb1 ~]# clustat

Cluster Status for ytkcluter @ Sun May  7 11:53:49 2017

Member Status: Quorate

 Member Name                         ID  Status

 ------ ----                         ---- ------

 192.168.10.1                           1 Online, Local, rgmanager

 192.168.10.2                           2 Online, rgmanager

 Service Name                    Owner (Last)                    State

 ------- ----                    ----- ------                    -----

 service:yktoracle                  192.168.10.1                    started

另外有需要云服务器可以了解下创新互联cdcxhl.cn，海内外云服务器15元起步，三天无理由+7*72小时售后在线，公司持有idc许可证，提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案，具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势，专为企业上云打造定制，能够满足用户丰富、多元化的应用场景需求。