CRS-1006 , CRS-0215 故障一例

时间：2017-08-07 来源：

　　安装好sles 10 sp3 + Oracle 10g RAC之后，在配置监听器时，总是提示主机bo2dbp上的监听服务已经在运行，忽略错误之后手动在bo2dbp节点上启，总是收到TNS-12545: Connect failed because target host or object does not exist错误信息。后来发现节点bo2dbp的vip总是漂移到另一节点bo2dbs上，原来这才是罪魁祸首 ??????

　　1、集群环境netca之后，节点bo2dbp监听无法启动

　　手动尝试启动监听仍不成功，查看了listener.ora以及监听日志未发现任何异常，查看主机/etc/hosts配置也正常

　　无奈之下，删除监听器

　　oracle@bo2dbp:~> crs_stat -t

　　Name Type Target State Host

　　------------------------------------------------------------

　　ora....BP.lsnr application ONLINE OFFLINE

　　ora.bo2dbp.gsd application ONLINE ONLINE bo2dbp

　　ora.bo2dbp.ons application ONLINE ONLINE bo2dbp

　　ora.bo2dbp.vip application ONLINE ONLINE bo2dbs

　　ora....BS.lsnr application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.gsd application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.ons application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.vip application ONLINE ONLINE bo2dbs

　　2、节点bo2dbp.vip被漂移到bo2dbs

　　oracle@bo2dbp:~/product/10.2.0/crs_1/bin> crs_stat -t

　　Name Type Target State Host

　　------------------------------------------------------------

　　ora.bo2dbp.gsd application ONLINE ONLINE bo2dbp

　　ora.bo2dbp.ons application ONLINE ONLINE bo2dbp

　　ora.bo2dbp.vip application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.gsd application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.ons application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.vip application ONLINE ONLINE bo2dbs

　　3、尝试为bo2dbp进行relocate也不听使唤

　　oracle@bo2dbp:~/product/10.2.0/crs_1/bin> ./crs_relocate ora.bo2dbp.vip -c bo2dbp

　　Attempting to stop `ora.bo2dbp.vip` on member `bo2dbs`

　　Stop of `ora.bo2dbp.vip` on member `bo2dbs` succeeded.

　　Attempting to start `ora.bo2dbp.vip` on member `bo2dbp`

　　Start of `ora.bo2dbp.vip` on member `bo2dbp` failed.

　　Attempting to start `ora.bo2dbp.vip` on member `bo2dbs`

　　Start of `ora.bo2dbp.vip` on member `bo2dbs` succeeded.

　　CRS-0217: Could not relocate resource 'ora.bo2dbp.vip'.

　　4、crs_stop 出马

　　oracle@bo2dbp:~/product/10.2.0/crs_1/bin> crs_stop ora.bo2dbp.vip

　　Attempting to stop `ora.bo2dbp.vip` on member `bo2dbs`

　　Stop of `ora.bo2dbp.vip` on member `bo2dbs` succeeded.

　　oracle@bo2dbp:~/product/10.2.0/crs_1/bin> crs_start ora.bo2dbp.vip

　　Attempting to start `ora.bo2dbp.vip` on member `bo2dbp`

　　Start of `ora.bo2dbp.vip` on member `bo2dbp` failed.

　　Attempting to start `ora.bo2dbp.vip` on member `bo2dbs`

　　Start of `ora.bo2dbp.vip` on member `bo2dbs` succeeded.

　　我晕，又漂移到bo2dbs节点

　　5、狠一点，直接将节点2bo2dbs的crs停掉，这下没得漂了吧 :)

　　bo2dbs:/u01/app/oracle/product/10.2.0/crs_1/bin # ./crsctl stop crs

　　Stopping resources. This could take several minutes.

　　Successfully stopped CRS resources.

　　Stopping CSSD.

　　Shutting down CSS daemon.

　　Shutdown request successfully issued.

　　6、露出马脚

　　oracle@bo2dbp:~/product/10.2.0/crs_1/bin> crs_start ora.bo2dbp.vip

　　Attempting to start `ora.bo2dbp.vip` on member `bo2dbp`

　　Start of `ora.bo2dbp.vip` on member `bo2dbp` failed.

　　Remote start for `ora.bo2dbp.vip` failed on member `bo2dbs`

　　CRS-1006: No more members to consider

　　CRS-0215: Could not start resource 'ora.bo2dbp.vip'.

　　7、check log

　　oracle@bo2dbp:~/product/10.2.0/crs_1/log/bo2dbp/racg> tail -50 ora.bo2dbp.vip.log

　　2012-08-30 16:18:55.942: [ RACG][4250161648] [20571][4250161648][ora.bo2dbp.vip]: end for resource = ora.bo2dbp.vip, action = start, status = 1, time = 6.390s

　　2012-08-30 16:19:30.968: [ RACG][4041843184] [20878][4041843184][ora.bo2dbp.vip]: checkIf: Default gateway is not defined (host=bo2dbp)

　　Interface eth0 checked failed (host=bo2dbp)

　　Invalid parameters, or failed to bring up VIP (host=bo2dbp)

　　原来醉鬼祸首尽然是没有缺省网关，我倒倒倒......

　　记得之前的安装就碰到过在节点2上没有缺省网关导致vipca不能继续的情形。这次又碰上了。汗。。。

　　oracle@bo2dbp:~> cat /etc/sysconfig/network/routes

　　cat: /etc/sysconfig/network/routes: No such file or directory

　　oracle@bo2dbp:~> ssh bo2dbs cat /etc/sysconfig/network/routes

　　default 192.168.7.254 - -

　　使用root帐户为其追加网关后，一切ok.

　　oracle@bo2dbs:~> crs_stat -t

　　Name Type Target State Host

　　------------------------------------------------------------

　　ora.bo2dbp.gsd application ONLINE ONLINE bo2dbp

　　ora.bo2dbp.ons application ONLINE ONLINE bo2dbp

　　ora.bo2dbp.vip application ONLINE ONLINE bo2dbp

　　ora.bo2dbs.gsd application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.ons application ONLINE ONLINE bo2dbs

　　ora.bo2dbs.vip application ONLINE ONLINE bo2dbs

　　8、问题环境

　　sles 10/sp3 + Oracle 10g(10.2.0.4)

　　9、小结:

　　a、细心细心再细心

　　b、根据相应的日志快速定位故障

　　(以上内容摘于网络，如有侵权，请告之，将第一时间删除)