未解决
此帖子已超过 5 年
15 消息
0
10048
存储突然链接失败，请教如何查找原因？
各位好！
今天发现这样一个问题，我的同事反馈使用PL/SQL Developer登录oracle数据库失败，然后我登录服务器发现oracle实例未启动，于是尝试启动
SQL> startup
ORACLE instance started.
Total System Global Area 4764729344 bytes
Fixed Size 2027160 bytes
Variable Size 419430760 bytes
Database Buffers 4328521728 bytes
Redo Buffers 14749696 bytes
ORA-00205: error in identifying control file, check alert log for more info
SQL>
根据提示，我查看alert日志
Sun Oct 6 21:08:37 2013
Thread 1 advanced to log sequence 357916
Current log# 3 seq# 357916 mem# 0: /clusdsk/oracle/oradata/ORCL/redo03.log
Sun Oct 6 21:26:27 2013
Errors in file /opt/oracle/admin/ORCL/bdump/ORCL_lgwr_17049.trc:
ORA-00345: redo log write error block 41385 count 2
ORA-00312: online log 3 thread 1: '/clusdsk/oracle/oradata/ORCL/redo03.log'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 41385
Additional information: -1
Sun Oct 6 21:26:28 2013
Errors in file /opt/oracle/admin/ORCL/bdump/ORCL_ckpt_17051.trc:
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '/clusdsk/oracle/oradata/ORCL/control03.ctl'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 3
Additional information: -1
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '/clusdsk/oracle/oradata/ORCL/control02.ctl'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 3
Additional information: -1
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '/clusdsk/oracle/oradata/ORCL/control01.ctl'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 3
Additional information: -1
Sun Oct 6 21:26:28 2013
Errors in file /opt/oracle/admin/ORCL/bdump/ORCL_ckpt_17051.trc:
ORA-00221: error on write to control file
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '/clusdsk/oracle/oradata/ORCL/control03.ctl'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 3
Additional information: -1
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '/clusdsk/oracle/oradata/ORCL/control02.ctl'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 3
Additional information: -1
ORA-00206: error in writing (block 3, # blocks 1) of control file
ORA-00202: control file: '/clusdsk/oracle/oradata/ORCL/control01.ctl'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 3
Additional information: -1
Sun Oct 6 21:26:28 2013
KCF: write/open error block=0x29497 online=1
file=19 /clusdsk/oracle/oradata/ORCL/TBS12.dbf
error=27072 txt: 'Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 169111
Additional information: -1'
Sun Oct 6 21:26:28 2013
KCF: write/open error block=0x294a7 online=1
file=19 /clusdsk/oracle/oradata/ORCL/TBS12.dbf
error=27072 txt: 'Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 169127
Additional information: -1'
Sun Oct 6 21:26:28 2013
CKPT: terminating instance due to error 221
Sun Oct 6 21:26:28 2013
Automatic datafile offline due to write error on
file 19: /clusdsk/oracle/oradata/ORCL/TBS12.dbf
Sun Oct 6 21:26:28 2013
Automatic datafile offline due to write error on
file 19: /clusdsk/oracle/oradata/ORCL/TBS12.dbf
Sun Oct 6 21:26:28 2013
Errors in file /opt/oracle/admin/ORCL/bdump/ORCL_dbw1_17047.trc:
ORA-00221: error on write to control file
Sun Oct 6 21:26:28 2013
Errors in file /opt/oracle/admin/ORCL/bdump/ORCL_dbw0_17045.trc:
ORA-00221: error on write to control file
Sun Oct 6 21:26:29 2013
Errors in file /opt/oracle/admin/ORCL/bdump/ORCL_lgwr_17049.trc:
ORA-00340: IO error processing online log 3 of thread 1
ORA-00345: redo log write error block 41385 count 2
ORA-00312: online log 3 thread 1: '/clusdsk/oracle/oradata/ORCL/redo03.log'
ORA-27072: File I/O error
Linux-x86_64 Error: 30: Read-only file system
Additional information: 4
Additional information: 41385
Additional information: -1
Sun Oct 6 21:26:34 2013
Instance terminated by CKPT, pid = 17051
然后，我感觉是和存储链路有关
于是，访问磁盘试试看
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 29G 5.5G 22G 21% /
/dev/cciss/c0d0p5 89G 5.1G 80G 7% /home
/dev/cciss/c0d0p1 99M 20M 75M 21% /boot
tmpfs 3.8G 0 3.8G 0% /dev/shm
none 3.8G 104K 3.8G 1% /var/lib/xenstored
/dev/sda1 825G 164G 619G 21% /clusdsk
$ cd /clusdsk
$ ls
ls: reading directory .: Input/output error
$
接着，我又查看了系统日志
# cat messages
Oct 6 04:03:24 dbs-n2 syslogd 1.4.1: restart.
Oct 6 21:26:11 dbs-n2 kernel: qla2xxx 0000:0b:00.0: LOOP DOWN detected (2 6 0).
Oct 6 21:26:27 dbs-n2 kernel: rport-0:0-0: blocked FC remote port time out: saving binding
Oct 6 21:26:27 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 21:26:27 dbs-n2 kernel: end_request: I/O error, dev sda, sector 214919
Oct 6 21:26:27 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 21:26:27 dbs-n2 kernel: end_request: I/O error, dev sda, sector 396623951
Oct 6 21:26:27 dbs-n2 kernel: Buffer I/O error on device sda1, logical block 49577986
Oct 6 21:26:27 dbs-n2 kernel: lost page write due to I/O error on sda1
Oct 6 21:26:27 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 21:26:27 dbs-n2 kernel: end_request: I/O error, dev sda, sector 398196815
Oct 6 21:26:27 dbs-n2 kernel: Buffer I/O error on device sda1, logical block 49774594
Oct 6 21:26:27 dbs-n2 kernel: lost page write due to I/O error on sda1
Oct 6 21:26:27 dbs-n2 kernel: Aborting journal on device sda1.
Oct 6 21:26:27 dbs-n2 kernel: EXT3-fs error (device sda1) in ext3_reserve_inode_write: Journal has aborted
Oct 6 21:26:27 dbs-n2 last message repeated 2 times
Oct 6 21:26:28 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 21:26:28 dbs-n2 kernel: end_request: I/O error, dev sda, sector 12423
Oct 6 21:26:28 dbs-n2 kernel: Buffer I/O error on device sda1, logical block 1545
Oct 6 21:26:28 dbs-n2 kernel: lost page write due to I/O error on sda1
Oct 6 21:26:28 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 21:26:28 dbs-n2 kernel: end_request: I/O error, dev sda, sector 63
Oct 6 21:26:28 dbs-n2 kernel: Buffer I/O error on device sda1, logical block 0
Oct 6 21:26:28 dbs-n2 kernel: lost page write due to I/O error on sda1
Oct 6 21:26:28 dbs-n2 kernel: EXT3-fs error (device sda1) in ext3_dirty_inode: Journal has aborted
Oct 6 21:26:28 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 21:26:28 dbs-n2 kernel: end_request: I/O error, dev sda, sector 63
Oct 6 21:26:28 dbs-n2 kernel: Buffer I/O error on device sda1, logical block 0
Oct 6 21:26:28 dbs-n2 kernel: lost page write due to I/O error on sda1
Oct 6 21:26:28 dbs-n2 kernel: ext3_abort called.
Oct 6 21:26:28 dbs-n2 kernel: EXT3-fs error (device sda1): ext3_journal_start_sb: Detected aborted journal
Oct 6 21:26:28 dbs-n2 kernel: Remounting filesystem read-only
Oct 6 21:43:09 dbs-n2 kernel: qla2xxx 0000:0b:00.0: LIP reset occured (f8f7).
Oct 6 21:43:09 dbs-n2 kernel: qla2xxx 0000:0b:00.0: LIP occured (f8f7).
Oct 6 21:43:09 dbs-n2 kernel: qla2xxx 0000:0b:00.0: LOOP UP detected (4 Gbps).
Oct 6 21:48:04 dbs-n2 kernel: qla2xxx 0000:0b:00.0: LOOP DOWN detected (2 6 0).
Oct 6 21:48:20 dbs-n2 kernel: rport-0:0-0: blocked FC remote port time out: saving binding
Oct 6 22:33:44 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 22:33:44 dbs-n2 kernel: end_request: I/O error, dev sda, sector 1050153047
Oct 6 22:33:44 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 22:33:44 dbs-n2 kernel: end_request: I/O error, dev sda, sector 1050153047
Oct 6 22:38:31 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 22:38:31 dbs-n2 kernel: end_request: I/O error, dev sda, sector 12375
Oct 6 22:39:33 dbs-n2 kernel: sd 0:0:0:0: SCSI error: return code = 0x00010000
Oct 6 22:39:33 dbs-n2 kernel: end_request: I/O error, dev sda, sector 12375
Oct 6 22:41:44 dbs-n2 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
Oct 6 22:41:47 dbs-n2 modclusterd: shutdown succeeded
Oct 6 22:41:48 dbs-n2 ricci: shutdown succeeded
Oct 6 22:41:49 dbs-n2 smartd[6631]: smartd received signal 15: Terminated
Oct 6 22:41:49 dbs-n2 smartd[6631]: smartd is exiting (exit status 0)
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Got SIGTERM, quitting.
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80::6ab5:99ff:fe78:7f22.
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.191.0.21.
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Leaving mDNS multicast group on interface virbr0.IPv6 with address fe80::200:ff:fe00:0.
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Leaving mDNS multicast group on interface virbr0.IPv4 with address 192.168.122.1.
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Leaving mDNS multicast group on interface eth1.IPv6 with address fe80::6ab5:99ff:fe78:7f24.
Oct 6 22:41:49 dbs-n2 avahi-daemon[5524]: Leaving mDNS multicast group on interface eth1.IPv4 with address 10.9.0.21.
Oct 6 22:41:49 dbs-n2 oddjobd: oddjobd shutdown succeeded
Oct 6 22:41:50 dbs-n2 libvirtd: 22:41:50.025: warning : Shutting down on signal 15
Oct 6 22:41:50 dbs-n2 heartbeat: : WARN: Shutdown delayed until current resource activity finishes.
Oct 6 22:41:56 dbs-n2 heartbeat: : info: Heartbeat shutdown in progress. (4753)
Oct 6 22:41:56 dbs-n2 heartbeat: : info: Received shutdown notice from 'dbs-n1'.
Oct 6 22:41:56 dbs-n2 heartbeat: : info: Resource takeover cancelled - shutdown in progress.
Oct 6 22:41:56 dbs-n2 heartbeat: : info: Giving up all HA resources.
Oct 6 22:41:56 dbs-n2 ResourceManager[9308]: info: Releasing resource group: dbs-n1 IPaddr::192.191.0.22/24/eth0 Filesystem::/dev/sda1::/clusdsk::ext3 oracledb
Oct 6 22:41:56 dbs-n2 ResourceManager[9308]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:42:00 dbs-n2 ResourceManager[9308]: info: Running /etc/ha.d/resource.d/Filesystem /dev/sda1 /clusdsk ext3 stop
Oct 6 22:42:01 dbs-n2 Filesystem[9469]: INFO: Running stop for /dev/sda1 on /clusdsk
Oct 6 22:42:01 dbs-n2 Filesystem[9469]: INFO: Trying to unmount /clusdsk
Oct 6 22:42:01 dbs-n2 kernel: ext3_abort called.
Oct 6 22:42:01 dbs-n2 kernel: EXT3-fs error (device sda1): ext3_put_super: Couldnt clean up the journal
Oct 6 22:42:01 dbs-n2 Filesystem[9469]: INFO: unmounted /clusdsk successfully
Oct 6 22:42:01 dbs-n2 Filesystem[9458]: INFO: Success
Oct 6 22:42:01 dbs-n2 ResourceManager[9308]: info: Running /etc/ha.d/resource.d/IPaddr 192.191.0.22/24/eth0 stop
Oct 6 22:42:01 dbs-n2 IPaddr[9588]: INFO: ifconfig eth0:0 down
Oct 6 22:42:01 dbs-n2 IPaddr[9559]: INFO: Success
Oct 6 22:42:01 dbs-n2 heartbeat: : info: All HA resources relinquished.
Oct 6 22:42:02 dbs-n2 heartbeat: : info: Link dbs-n1:eth1 dead.
Oct 6 22:42:02 dbs-n2 ipfail: : info: Link Status update: Link dbs-n1/eth1 now has status dead
Oct 6 22:42:02 dbs-n2 heartbeat: : info: killing /usr/lib64/heartbeat/ipfail process group 4791 with signal 15
Oct 6 22:42:03 dbs-n2 ipfail: : info: Asking other side for ping node count.
Oct 6 22:42:03 dbs-n2 ipfail: : info: Checking remote count of ping nodes.
Oct 6 22:42:04 dbs-n2 heartbeat: : info: killing HBFIFO process 4768 with signal 15
Oct 6 22:42:04 dbs-n2 heartbeat: : info: killing HBWRITE process 4769 with signal 15
Oct 6 22:42:04 dbs-n2 heartbeat: : info: killing HBREAD process 4770 with signal 15
Oct 6 22:42:04 dbs-n2 heartbeat: : info: killing HBWRITE process 4771 with signal 15
Oct 6 22:42:04 dbs-n2 heartbeat: : info: killing HBREAD process 4772 with signal 15
Oct 6 22:42:04 dbs-n2 heartbeat: : info: Core process 4768 exited. 5 remaining
Oct 6 22:42:04 dbs-n2 heartbeat: : info: Core process 4769 exited. 4 remaining
Oct 6 22:42:04 dbs-n2 heartbeat: : info: Core process 4771 exited. 3 remaining
Oct 6 22:42:04 dbs-n2 heartbeat: : info: Core process 4772 exited. 2 remaining
Oct 6 22:42:04 dbs-n2 heartbeat: : info: Core process 4770 exited. 1 remaining
Oct 6 22:42:04 dbs-n2 heartbeat: : info: dbs-n2 Heartbeat shutdown complete.
Oct 6 22:42:04 dbs-n2 logd: : info: Waiting for pid=4660 to exit
Oct 6 22:42:04 dbs-n2 logd: : info: logd_term_write_action: received SIGTERM
Oct 6 22:42:04 dbs-n2 logd: : info: Exiting write process
Oct 6 22:42:05 dbs-n2 logd: : info: Pid 4660 exited
Oct 6 22:42:05 dbs-n2 saslauthd[6606]: server_exit : master exited: 6606
Oct 6 22:42:06 dbs-n2 xinetd[4645]: Exiting...
Oct 6 22:42:10 dbs-n2 hcid[4374]: Got disconnected from the system message bus
Oct 6 22:42:11 dbs-n2 rpc.statd[4295]: Caught signal 15, un-registering and exiting.
Oct 6 22:42:11 dbs-n2 auditd[4163]: The audit daemon is exiting.
Oct 6 22:42:11 dbs-n2 kernel: audit(1381070531.476:50470): audit_pid=0 old=4163 by auid=4294967295
Oct 6 22:42:11 dbs-n2 pcscd: pcscdaemon.c:572:signal_trap() Preparing for suicide
Oct 6 22:42:12 dbs-n2 pcscd: hotplug_libusb.c:376:HPRescanUsbBus() Hotplug stopped
Oct 6 22:42:12 dbs-n2 pcscd: readerfactory.c:1379:RFCleanupReaders() entering cleaning function
Oct 6 22:42:12 dbs-n2 pcscd: pcscdaemon.c:532:at_exit() cleaning /var/run
Oct 6 22:42:12 dbs-n2 kernel: Kernel logging (proc) stopped.
Oct 6 22:42:12 dbs-n2 kernel: Kernel log daemon terminating.
Oct 6 22:42:13 dbs-n2 exiting on signal 15
Oct 6 22:45:52 dbs-n2 syslogd 1.4.1: restart.
Oct 6 22:45:52 dbs-n2 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Oct 6 22:45:52 dbs-n2 kernel: Bootdata ok (command line is ro root=LABEL=/ rhgb quiet)
Oct 6 22:45:52 dbs-n2 kernel: Linux version 2.6.18-164.el5xen (mockbuild@x86-003.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Tue Aug 18 15:59:52 EDT 2009
Oct 6 22:45:52 dbs-n2 kernel: BIOS-provided physical RAM map:
Oct 6 22:45:52 dbs-n2 kernel: Xen: 0000000000000000 - 00000001dc8fa000 (usable)
Oct 6 22:45:52 dbs-n2 kernel: DMI 2.7 present.
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x10] lapic_id[0x20] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x08] lapic_id[0x10] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x18] lapic_id[0x30] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x14] lapic_id[0x24] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x14] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x34] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x12] lapic_id[0x22] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x12] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x32] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x16] lapic_id[0x26] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x16] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x36] disabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
Oct 6 22:45:52 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x09] lapic_id[0x11] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x19] lapic_id[0x31] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x15] lapic_id[0x25] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x15] enabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x35] enabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
Oct 6 22:45:53 dbs-n2 rpc.statd[4213]: Version 1.0.9 Starting
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x13] lapic_id[0x23] enabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x13] enabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x33] enabled)
Oct 6 22:45:53 dbs-n2 rpc.statd[4213]: statd running as root. chown /var/lib/nfs/statd/sm to choose different user
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x17] lapic_id[0x27] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x17] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x37] disabled)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
Oct 6 22:45:53 dbs-n2 kernel: ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
Oct 6 22:45:53 dbs-n2 kernel: IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
Oct 6 22:45:53 dbs-n2 kernel: ACPI: IOAPIC (id[0x00] address[0xfec80000] gsi_base[24])
Oct 6 22:45:53 dbs-n2 kernel: IOAPIC[1]: apic_id 0, version 32, address 0xfec80000, GSI 24-47
Oct 6 22:45:53 dbs-n2 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
Oct 6 22:45:53 dbs-n2 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Oct 6 22:45:53 dbs-n2 kernel: Setting APIC routing to xen
Oct 6 22:45:53 dbs-n2 kernel: Using ACPI (MADT) for SMP configuration information
Oct 6 22:45:53 dbs-n2 kernel: Allocating PCI resources starting at e6000000 (gap: e4000000:1ac00000)
Oct 6 22:45:53 dbs-n2 kernel: Built 1 zonelists. Total pages: 1951994
Oct 6 22:45:53 dbs-n2 kernel: Kernel command line: ro root=LABEL=/ rhgb quiet
Oct 6 22:45:53 dbs-n2 kernel: Initializing CPU#0
Oct 6 22:45:53 dbs-n2 kernel: PID hash table entries: 4096 (order: 12, 32768 bytes)
Oct 6 22:45:53 dbs-n2 kernel: Xen reported: 2399.364 MHz processor.
Oct 6 22:45:53 dbs-n2 kernel: Console: colour VGA+ 80x25
Oct 6 22:45:54 dbs-n2 kernel: Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Oct 6 22:45:54 dbs-n2 kernel: Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Oct 6 22:45:54 dbs-n2 kernel: Software IO TLB enabled:
Oct 6 22:45:54 dbs-n2 kernel: Aperture: 64 megabytes
Oct 6 22:45:54 dbs-n2 kernel: Kernel range: 0xffff88000a478000 - 0xffff88000e478000
Oct 6 22:45:54 dbs-n2 kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Oct 6 22:45:54 dbs-n2 kernel: Memory: 7569128k/7807976k available (2490k kernel code, 229996k reserved, 1388k data, 184k init)
Oct 6 22:45:54 dbs-n2 hcid[4292]: Bluetooth HCI daemon
Oct 6 22:45:54 dbs-n2 kernel: Calibrating delay using timer specific routine.. 6001.33 BogoMIPS (lpj=12002673)
Oct 6 22:45:54 dbs-n2 kernel: Security Framework v1.0.0 initialized
Oct 6 22:45:54 dbs-n2 kernel: SELinux: Initializing.
Oct 6 22:45:54 dbs-n2 hcid[4292]: Register path:/org/bluez fallback:1
Oct 6 22:45:54 dbs-n2 sdpd[4296]: Bluetooth SDP daemon
Oct 6 22:45:54 dbs-n2 kernel: selinux_register_security: Registering secondary module capability
Oct 6 22:45:54 dbs-n2 kernel: Capability LSM initialized as secondary
Oct 6 22:45:54 dbs-n2 kernel: Mount-cache hash table entries: 256
Oct 6 22:45:54 dbs-n2 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Oct 6 22:45:54 dbs-n2 kernel: CPU: L2 cache: 256K
Oct 6 22:45:54 dbs-n2 kernel: CPU: L3 cache: 12288K
Oct 6 22:45:54 dbs-n2 kernel: CPU: Physical Processor ID: 0
Oct 6 22:45:54 dbs-n2 kernel: CPU: Processor Core ID: 0
Oct 6 22:45:54 dbs-n2 kernel: (SMP-)alternatives turned off
Oct 6 22:45:54 dbs-n2 kernel: ACPI: Core revision 20060707
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#1
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#2
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#3
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#4
Oct 6 22:45:54 dbs-n2 pcscd: pcscdaemon.c:507:main() pcsc-lite 1.4.4 daemon ready.
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#5
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#6
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#7
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#8
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#9
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#10
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#11
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#12
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#13
Oct 6 22:45:54 dbs-n2 kernel: Initializing CPU#14
Oct 6 22:45:55 dbs-n2 pcscd: hotplug_libusb.c:402:HPEstablishUSBNotifications() Driver ifd-egate.bundle does not support IFD_GENERATE_HOTPLUG. Using active polling instead.
Oct 6 22:45:55 dbs-n2 kernel: Brought up 16 CPUs
Oct 6 22:45:55 dbs-n2 pcscd: hotplug_libusb.c:411:HPEstablishUSBNotifications() Polling forced every 1 second(s)
Oct 6 22:45:55 dbs-n2 kernel: Initializing CPU#15
Oct 6 22:45:55 dbs-n2 kernel: migration_cost=632
Oct 6 22:45:55 dbs-n2 kernel: checking if image is initramfs... it is
Oct 6 22:45:55 dbs-n2 kernel: Grant table initialized
Oct 6 22:45:55 dbs-n2 kernel: NET: Registered protocol family 16
Oct 6 22:45:55 dbs-n2 kernel: ACPI: bus type pci registered
Oct 6 22:45:55 dbs-n2 kernel: PCI: Not using MMCONFIG.
Oct 6 22:45:55 dbs-n2 kernel: PCI: Using configuration type 1
Oct 6 22:45:55 dbs-n2 kernel: ACPI: Interpreter enabled
Oct 6 22:45:55 dbs-n2 kernel: ACPI: Using IOAPIC for interrupt routing
Oct 6 22:45:55 dbs-n2 kernel: ACPI: No dock devices found.
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Root Bridge (0000:00)
Oct 6 22:45:55 dbs-n2 kernel: PCI: Transparent bridge - 0000:00:1e.0
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs 5 7 *10 11)
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs 5 7 10 *11)
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs 5 *7 10 11)
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs *5 7 10 11)
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs *5 7 10 11)
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs 5 7 10 11) *0, disabled.
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs 5 7 *10 11)
Oct 6 22:45:55 dbs-n2 kernel: ACPI: PCI Interrupt Link (IRQs 5 *7 10 11)
Oct 6 22:45:55 dbs-n2 kernel: Linux Plug and Play Support v0.97 (c) Adam Belay
Oct 6 22:45:55 dbs-n2 kernel: pnp: PnP ACPI init
Oct 6 22:45:55 dbs-n2 kernel: pnp: PnP ACPI: found 11 devices
Oct 6 22:45:55 dbs-n2 kernel: xen_mem: Initialising balloon driver.
Oct 6 22:45:55 dbs-n2 kernel: usbcore: registered new driver usbfs
Oct 6 22:45:55 dbs-n2 kernel: usbcore: registered new driver hub
Oct 6 22:45:55 dbs-n2 kernel: PCI: Using ACPI for IRQ routing
Oct 6 22:45:55 dbs-n2 kernel: PCI: If a device doesnt work, try "pci=routeirq". If it helps, post a report
Oct 6 22:45:55 dbs-n2 kernel: NetLabel: Initializing
Oct 6 22:45:55 dbs-n2 kernel: NetLabel: domain hash size = 128
Oct 6 22:45:55 dbs-n2 kernel: NetLabel: protocols = UNLABELED CIPSOv4
Oct 6 22:45:55 dbs-n2 kernel: NetLabel: unlabeled traffic allowed by default
Oct 6 22:45:55 dbs-n2 kernel: pnp: 00:01: ioport range 0x408-0x40f has been reserved
Oct 6 22:45:55 dbs-n2 kernel: pnp: 00:01: ioport range 0x4d0-0x4d1 has been reserved
Oct 6 22:45:55 dbs-n2 kernel: pnp: 00:01: ioport range 0x700-0x71f has been reserved
Oct 6 22:45:55 dbs-n2 kernel: PCI: Bridge: 0000:00:01.0
Oct 6 22:45:55 dbs-n2 kernel: IO window: 4000-4fff
Oct 6 22:45:55 dbs-n2 kernel: MEM window: fb700000-fbbfffff
Oct 6 22:45:55 dbs-n2 kernel: PREFETCH window 0x00000000e6000000-0x00000000e60fffff
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:02.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:03.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:04.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:05.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:06.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:07.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:08.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: 5000-5fff
Oct 6 22:45:56 dbs-n2 kernel: MEM window: fbf00000-fbffffff
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window 0x00000000e6100000-0x00000000e61fffff
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:09.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: PCI: Bridge: 0000:00:0a.0
Oct 6 22:45:56 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:56 dbs-n2 kernel: MEM window: disabled.
Oct 6 22:45:57 dbs-n2 kernel: PREFETCH window: disabled.
Oct 6 22:45:57 dbs-n2 kernel: PCI: Bridge: 0000:00:1c.0
Oct 6 22:45:57 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:57 dbs-n2 kernel: MEM window: f2000000-f5ffffff
Oct 6 22:45:57 dbs-n2 kernel: PREFETCH window 0x00000000e6200000-0x00000000e62fffff
Oct 6 22:45:57 dbs-n2 kernel: PCI: Bridge: 0000:00:1c.2
Oct 6 22:45:57 dbs-n2 kernel: IO window: disabled.
Oct 6 22:45:57 dbs-n2 kernel: MEM window: f6000000-f9ffffff
Oct 6 22:45:57 dbs-n2 kernel: PREFETCH window 0x00000000e6300000-0x00000000e63fffff
Oct 6 22:45:57 dbs-n2 kernel: PCI: Bridge: 0000:00:1c.4
Oct 6 22:45:57 dbs-n2 kernel: IO window: 3000-3fff
Oct 6 22:45:57 dbs-n2 kernel: MEM window: f1d00000-f1ffffff
Oct 6 22:45:57 dbs-n2 kernel: PREFETCH window 0x00000000e6400000-0x00000000e64fffff
Oct 6 22:45:57 dbs-n2 kernel: PCI: Bridge: 0000:00:1e.0
Oct 6 22:45:57 dbs-n2 kernel: IO window: 2000-2fff
Oct 6 22:45:57 dbs-n2 kernel: MEM window: f1c00000-f1cfffff
Oct 6 22:45:57 dbs-n2 kernel: PREFETCH window 0x00000000e8000000-0x00000000efffffff
Oct 6 22:45:57 dbs-n2 hidd[4478]: Bluetooth HID daemon
Oct 6 22:45:57 dbs-n2 kernel: GSI 16 sharing vector 0x98 and IRQ 16
Oct 6 22:45:57 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1c.0 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:45:57 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1c.2 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:45:57 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1c.4 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:45:57 dbs-n2 kernel: NET: Registered protocol family 2
Oct 6 22:45:57 dbs-n2 kernel: IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
Oct 6 22:45:57 dbs-n2 kernel: TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
Oct 6 22:45:57 dbs-n2 kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Oct 6 22:45:57 dbs-n2 kernel: TCP: Hash tables configured (established 262144 bind 65536)
Oct 6 22:45:57 dbs-n2 kernel: TCP reno registered
Oct 6 22:45:57 dbs-n2 kernel: audit: initializing netlink socket (disabled)
Oct 6 22:45:57 dbs-n2 kernel: type=2000 audit(1381070687.472:1): initialized
Oct 6 22:45:57 dbs-n2 kernel: VFS: Disk quotas dquot_6.5.1
Oct 6 22:45:57 dbs-n2 kernel: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Oct 6 22:45:57 dbs-n2 kernel: Initializing Cryptographic API
Oct 6 22:45:57 dbs-n2 kernel: alg: No test for crc32c (crc32c-generic)
Oct 6 22:45:57 dbs-n2 kernel: ksign: Installing public key data
Oct 6 22:45:57 dbs-n2 kernel: Loading keyring
Oct 6 22:45:57 dbs-n2 kernel: - Added public key 4BD2703D74512BB
Oct 6 22:45:57 dbs-n2 kernel: - User ID: Red Hat, Inc. (Kernel Module GPG key)
Oct 6 22:45:57 dbs-n2 kernel: io scheduler noop registered
Oct 6 22:45:57 dbs-n2 kernel: io scheduler anticipatory registered
Oct 6 22:45:57 dbs-n2 kernel: io scheduler deadline registered
Oct 6 22:45:57 dbs-n2 kernel: io scheduler cfq registered (default)
Oct 6 22:45:58 dbs-n2 kernel: assign_interrupt_mode Found MSI capability
Oct 6 22:45:58 dbs-n2 last message repeated 9 times
Oct 6 22:45:58 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1c.0 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:45:58 dbs-n2 kernel: assign_interrupt_mode Found MSI capability
Oct 6 22:45:58 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1c.2 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:45:58 dbs-n2 kernel: assign_interrupt_mode Found MSI capability
Oct 6 22:45:58 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1c.4 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:45:58 dbs-n2 kernel: assign_interrupt_mode Found MSI capability
Oct 6 22:45:58 dbs-n2 kernel: pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU0 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU8 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU4 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU12 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU6 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU14 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU2 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU10 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU1 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU9 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU5 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU13 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU7 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU15 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU3 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: CPU11 (power states: C1[C1] C2[C3] C3[C3])
Oct 6 22:45:58 dbs-n2 kernel: ACPI: Thermal Zone (8 C)
Oct 6 22:45:58 dbs-n2 kernel: Real Time Clock Driver v1.12ac
Oct 6 22:45:58 dbs-n2 kernel: hpet_acpi_add: no address or irqs in _CRS
Oct 6 22:45:58 dbs-n2 kernel: Non-volatile memory driver v1.2
Oct 6 22:45:58 dbs-n2 kernel: Linux agpgart interface v0.101 (c) Dave Jones
Oct 6 22:45:58 dbs-n2 kernel: brd: module loaded
Oct 6 22:45:58 dbs-n2 kernel: Xen virtual console successfully installed as ttyS0
Oct 6 22:45:58 dbs-n2 kernel: Event-channel device installed.
Oct 6 22:45:58 dbs-n2 kernel: Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
Oct 6 22:45:58 dbs-n2 kernel: ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Oct 6 22:45:58 dbs-n2 automount[4518]: lookup_read_master: lookup(nisplus): couldnt locate nis+ table auto.master
Oct 6 22:45:58 dbs-n2 kernel: ide-floppy driver 0.99.newide
Oct 6 22:45:58 dbs-n2 kernel: usbcore: registered new driver hiddev
Oct 6 22:45:58 dbs-n2 kernel: usbcore: registered new driver usbhid
Oct 6 22:45:58 dbs-n2 kernel: drivers/usb/input/hid-core.c: v2.6:USB HID core driver
Oct 6 22:45:58 dbs-n2 kernel: PNP: PS/2 Controller at 0x60,0x64 irq 1,12
Oct 6 22:45:59 dbs-n2 kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Oct 6 22:45:59 dbs-n2 kernel: serio: i8042 AUX port at 0x60,0x64 irq 12
Oct 6 22:45:59 dbs-n2 kernel: mice: PS/2 mouse device common for all mice
Oct 6 22:45:59 dbs-n2 kernel: md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
Oct 6 22:45:59 dbs-n2 kernel: md: bitmap version 4.39
Oct 6 22:45:59 dbs-n2 kernel: TCP bic registered
Oct 6 22:45:59 dbs-n2 kernel: Initializing IPsec netlink socket
Oct 6 22:45:59 dbs-n2 kernel: NET: Registered protocol family 1
Oct 6 22:45:59 dbs-n2 kernel: NET: Registered protocol family 17
Oct 6 22:45:59 dbs-n2 kernel: Initalizing network drop monitor service
Oct 6 22:45:59 dbs-n2 kernel: Write protecting the kernel read-only data: 478k
Oct 6 22:45:59 dbs-n2 kernel: input: AT Translated Set 2 keyboard as /class/input/input0
Oct 6 22:45:59 dbs-n2 kernel: logips2pp: Detected unknown logitech mouse model 0
Oct 6 22:45:59 dbs-n2 kernel: GSI 17 sharing vector 0x49 and IRQ 17
Oct 6 22:45:59 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1d.7 -> GSI 20 (level, low) -> IRQ 17
Oct 6 22:45:59 dbs-n2 kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller
Oct 6 22:45:59 dbs-n2 kernel: ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
Oct 6 22:45:59 dbs-n2 kernel: ehci_hcd 0000:00:1d.7: debug port 1
Oct 6 22:45:59 dbs-n2 kernel: ehci_hcd 0000:00:1d.7: irq 17, io mem 0xf1bf0000
Oct 6 22:45:59 dbs-n2 kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
Oct 6 22:45:59 dbs-n2 kernel: usb usb1: configuration #1 chosen from 1 choice
Oct 6 22:45:59 dbs-n2 kernel: hub 1-0:1.0: USB hub found
Oct 6 22:46:00 dbs-n2 kernel: hub 1-0:1.0: 8 ports detected
Oct 6 22:46:00 dbs-n2 kernel: USB Universal Host Controller Interface driver v3.0
Oct 6 22:46:00 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1d.0 -> GSI 20 (level, low) -> IRQ 17
Oct 6 22:46:00 dbs-n2 logd: : info: logd started with default configuration.
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
Oct 6 22:46:00 dbs-n2 logd: : info: G_main_add_SignalHandler: Added signal handler for signal 15
Oct 6 22:46:00 dbs-n2 logd: : info: G_main_add_SignalHandler: Added signal handler for signal 15
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.0: irq 17, io base 0x00001000
Oct 6 22:46:00 dbs-n2 kernel: usb usb2: configuration #1 chosen from 1 choice
Oct 6 22:46:00 dbs-n2 kernel: hub 2-0:1.0: USB hub found
Oct 6 22:46:00 dbs-n2 kernel: hub 2-0:1.0: 2 ports detected
Oct 6 22:46:00 dbs-n2 kernel: input: ImPS/2 Logitech Wheel Mouse as /class/input/input1
Oct 6 22:46:00 dbs-n2 kernel: GSI 18 sharing vector 0x51 and IRQ 18
Oct 6 22:46:00 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1d.1 -> GSI 23 (level, low) -> IRQ 18
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.1: irq 18, io base 0x00001020
Oct 6 22:46:00 dbs-n2 kernel: usb usb3: configuration #1 chosen from 1 choice
Oct 6 22:46:00 dbs-n2 kernel: hub 3-0:1.0: USB hub found
Oct 6 22:46:00 dbs-n2 kernel: hub 3-0:1.0: 2 ports detected
Oct 6 22:46:00 dbs-n2 kernel: GSI 19 sharing vector 0x59 and IRQ 19
Oct 6 22:46:00 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1d.2 -> GSI 22 (level, low) -> IRQ 19
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4
Oct 6 22:46:00 dbs-n2 heartbeat: : info: Version 2 support: false
Oct 6 22:46:00 dbs-n2 kernel: uhci_hcd 0000:00:1d.2: irq 19, io base 0x00001040
Oct 6 22:46:00 dbs-n2 kernel: usb usb4: configuration #1 chosen from 1 choice
Oct 6 22:46:00 dbs-n2 heartbeat: : WARN: Logging daemon is disabled --enabling logging daemon is recommended
Oct 6 22:46:00 dbs-n2 kernel: hub 4-0:1.0: USB hub found
Oct 6 22:46:01 dbs-n2 heartbeat: : info: **************************
Oct 6 22:46:01 dbs-n2 kernel: hub 4-0:1.0: 2 ports detected
Oct 6 22:46:01 dbs-n2 heartbeat: : info: Configuration validated. Starting heartbeat 2.1.4
Oct 6 22:46:01 dbs-n2 kernel: ACPI: PCI Interrupt 0000:00:1d.3 -> GSI 23 (level, low) -> IRQ 18
Oct 6 22:46:01 dbs-n2 heartbeat: : info: heartbeat: version 2.1.4
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:00:1d.3: UHCI Host Controller
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 5
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:00:1d.3: irq 18, io base 0x00001060
Oct 6 22:46:01 dbs-n2 kernel: usb usb5: configuration #1 chosen from 1 choice
Oct 6 22:46:01 dbs-n2 kernel: hub 5-0:1.0: USB hub found
Oct 6 22:46:01 dbs-n2 kernel: hub 5-0:1.0: 2 ports detected
Oct 6 22:46:01 dbs-n2 kernel: GSI 20 sharing vector 0x61 and IRQ 20
Oct 6 22:46:01 dbs-n2 kernel: ACPI: PCI Interrupt 0000:02:00.4 -> GSI 17 (level, low) -> IRQ 20
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:02:00.4: UHCI Host Controller
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:02:00.4: new USB bus registered, assigned bus number 6
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:02:00.4: port count misdetected? forcing to 2 ports
Oct 6 22:46:01 dbs-n2 kernel: uhci_hcd 0000:02:00.4: irq 20, io base 0x00003c00
Oct 6 22:46:01 dbs-n2 kernel: usb usb6: configuration #1 chosen from 1 choice
Oct 6 22:46:01 dbs-n2 kernel: hub 6-0:1.0: USB hub found
Oct 6 22:46:01 dbs-n2 kernel: hub 6-0:1.0: 2 ports detected
Oct 6 22:46:01 dbs-n2 kernel: SCSI subsystem initialized
Oct 6 22:46:01 dbs-n2 kernel: HP CISS Driver (v 3.6.20-RH3)
Oct 6 22:46:01 dbs-n2 kernel: GSI 21 sharing vector 0x69 and IRQ 21
Oct 6 22:46:01 dbs-n2 kernel: ACPI: PCI Interrupt 0000:05:00.0 -> GSI 28 (level, low) -> IRQ 21
Oct 6 22:46:01 dbs-n2 kernel: cciss0: <0x323a> at PCI 0000:05:00.0 IRQ 240 using DAC
Oct 6 22:46:01 dbs-n2 kernel: blocks= 286677120 block_size= 512
Oct 6 22:46:01 dbs-n2 kernel: heads= 255, sectors= 32, cylinders= 35132
Oct 6 22:46:01 dbs-n2 kernel:
Oct 6 22:46:01 dbs-n2 kernel: blocks= 286677120 block_size= 512
Oct 6 22:46:02 dbs-n2 kernel: heads= 255, sectors= 32, cylinders= 35132
Oct 6 22:46:02 dbs-n2 kernel:
Oct 6 22:46:02 dbs-n2 xinetd[4561]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in.
Oct 6 22:46:02 dbs-n2 kernel: cciss/c0d0: p1 p2 p3 p4 < p5 >
Oct 6 22:46:02 dbs-n2 xinetd[4561]: Started working: 0 available services
Oct 6 22:46:02 dbs-n2 kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Oct 6 22:46:02 dbs-n2 kernel: QLogic Fibre Channel HBA Driver
Oct 6 22:46:02 dbs-n2 kernel: GSI 22 sharing vector 0x91 and IRQ 22
Oct 6 22:46:02 dbs-n2 kernel: ACPI: PCI Interrupt 0000:0b:00.0 -> GSI 31 (level, low) -> IRQ 22
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0: Found an ISP2432, irq 22, iobase 0xffffc2000002a000
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0: Configuring PCI space...
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0: Configure NVRAM parameters...
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0: Verifying loaded RISC code...
Oct 6 22:46:02 dbs-n2 kernel: usb 6-1: new full speed USB device using uhci_hcd and address 2
Oct 6 22:46:02 dbs-n2 kernel: usb 6-1: configuration #1 chosen from 1 choice
Oct 6 22:46:02 dbs-n2 kernel: input: HP Virtual Keyboard as /class/input/input2
Oct 6 22:46:02 dbs-n2 kernel: input: USB HID v1.01 Keyboard on usb-0000:02:00.4-1
Oct 6 22:46:02 dbs-n2 kernel: input: HP Virtual Keyboard as /class/input/input3
Oct 6 22:46:02 dbs-n2 kernel: input: USB HID v1.01 Mouse on usb-0000:02:00.4-1
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0: Allocated (64 KB) for EFT...
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0: Allocated (1413 KB) for firmware dump...
Oct 6 22:46:02 dbs-n2 kernel: scsi0 : qla2xxx
Oct 6 22:46:02 dbs-n2 kernel: qla2xxx 0000:0b:00.0:
Oct 6 22:46:02 dbs-n2 kernel: QLogic Fibre Channel HBA Driver: 8.03.00.10.05.04-k
Oct 6 22:46:02 dbs-n2 kernel: QLogic HPAE311A - PCI-Express 4Gb Fibre Channel HBA
Oct 6 22:46:02 dbs-n2 kernel: ISP2432: PCIe (2.5Gb/s x4) @ 0000:0b:00.0 hdma+, host#=0, fw=4.04.09 (486)
Oct 6 22:46:02 dbs-n2 kernel: Initializing USB Mass Storage driver...
Oct 6 22:46:02 dbs-n2 kernel: usbcore: registered new driver usb-storage
Oct 6 22:46:02 dbs-n2 kernel: USB Mass Storage support registered.
Oct 6 22:46:02 dbs-n2 kernel: device-mapper: uevent: version 1.0.3
Oct 6 22:46:02 dbs-n2 kernel: device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel@redhat.com
Oct 6 22:46:02 dbs-n2 kernel: device-mapper: dm-raid45: initialized v0.2594l
Oct 6 22:46:02 dbs-n2 kernel: kjournald starting. Commit interval 5 seconds
Oct 6 22:46:02 dbs-n2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 6 22:46:02 dbs-n2 kernel: SELinux: Disabled at runtime.
Oct 6 22:46:02 dbs-n2 kernel: type=1404 audit(1381070718.142:2): selinux=0 auid=4294967295 ses=4294967295
Oct 6 22:46:02 dbs-n2 kernel: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.9.3 (March 17, 2009)
Oct 6 22:46:02 dbs-n2 kernel: ACPI: PCI Interrupt 0000:03:00.0 -> GSI 16 (level, low) -> IRQ 16
Oct 6 22:46:02 dbs-n2 kernel: eth0: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem f4000000, IRQ 16, node addr 68b599787f22
Oct 6 22:46:02 dbs-n2 kernel: ACPI: PCI Interrupt 0000:03:00.1 -> GSI 17 (level, low) -> IRQ 20
Oct 6 22:46:02 dbs-n2 kernel: eth1: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem f2000000, IRQ 20, node addr 68b599787f24
Oct 6 22:46:02 dbs-n2 kernel: GSI 23 sharing vector 0x99 and IRQ 23
Oct 6 22:46:02 dbs-n2 kernel: ACPI: PCI Interrupt 0000:04:00.0 -> GSI 18 (level, low) -> IRQ 23
Oct 6 22:46:02 dbs-n2 kernel: eth2: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem f8000000, IRQ 23, node addr 68b599787f26
Oct 6 22:46:02 dbs-n2 kernel: GSI 24 sharing vector 0xA1 and IRQ 24
Oct 6 22:46:03 dbs-n2 kernel: ACPI: PCI Interrupt 0000:04:00.1 -> GSI 19 (level, low) -> IRQ 24
Oct 6 22:46:03 dbs-n2 kernel: eth3: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem f6000000, IRQ 24, node addr 68b599787f28
Oct 6 22:46:03 dbs-n2 kernel: input: PC Speaker as /class/input/input4
Oct 6 22:46:03 dbs-n2 kernel: ACPI: PCI Interrupt 0000:02:00.2 -> GSI 17 (level, low) -> IRQ 20
Oct 6 22:46:03 dbs-n2 kernel: Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
Oct 6 22:46:03 dbs-n2 kernel: Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
Oct 6 22:46:03 dbs-n2 kernel: Floppy drive(s): fd0 is 1.44M
Oct 6 22:46:03 dbs-n2 kernel: floppy0: no floppy controllers found
Oct 6 22:46:03 dbs-n2 kernel: Floppy drive(s): fd0 is 1.44M
Oct 6 22:46:03 dbs-n2 heartbeat: : info: Heartbeat generation: 1292541790
Oct 6 22:46:03 dbs-n2 kernel: floppy0: no floppy controllers found
Oct 6 22:46:03 dbs-n2 kernel: lp: driver loaded but no devices found
Oct 6 22:46:03 dbs-n2 kernel: NET: Registered protocol family 10
Oct 6 22:46:03 dbs-n2 heartbeat: : info: glib: UDP Broadcast heartbeat started on port 694 (694) interface eth1
Oct 6 22:46:03 dbs-n2 kernel: lo: Disabled Privacy Extensions
Oct 6 22:46:03 dbs-n2 kernel: IPv6 over IPv4 tunneling driver
Oct 6 22:46:03 dbs-n2 kernel: ACPI: Power Button (FF)
Oct 6 22:46:03 dbs-n2 kernel: md: Autodetecting RAID arrays.
Oct 6 22:46:03 dbs-n2 kernel: md: autorun ...
Oct 6 22:46:03 dbs-n2 heartbeat: : info: glib: UDP Broadcast heartbeat closed on port 694 interface eth1 - Status: 1
Oct 6 22:46:03 dbs-n2 kernel: md: ... autorun DONE.
Oct 6 22:46:03 dbs-n2 heartbeat: : info: glib: ping heartbeat started.
Oct 6 22:46:03 dbs-n2 kernel: device-mapper: multipath: version 1.0.5 loaded
Oct 6 22:46:03 dbs-n2 heartbeat: : info: G_main_add_TriggerHandler: Added signal manual handler
Oct 6 22:46:03 dbs-n2 kernel: EXT3 FS on cciss/c0d0p2, internal journal
Oct 6 22:46:03 dbs-n2 heartbeat: : info: G_main_add_TriggerHandler: Added signal manual handler
Oct 6 22:46:03 dbs-n2 kernel: kjournald starting. Commit interval 5 seconds
Oct 6 22:46:03 dbs-n2 heartbeat: : info: G_main_add_SignalHandler: Added signal handler for signal 17
Oct 6 22:46:03 dbs-n2 kernel: EXT3 FS on cciss/c0d0p5, internal journal
Oct 6 22:46:03 dbs-n2 heartbeat: : info: Local status now set to: 'up'
Oct 6 22:46:03 dbs-n2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 6 22:46:03 dbs-n2 kernel: kjournald starting. Commit interval 5 seconds
Oct 6 22:46:03 dbs-n2 kernel: EXT3 FS on cciss/c0d0p1, internal journal
Oct 6 22:46:03 dbs-n2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Oct 6 22:46:03 dbs-n2 kernel: Adding 16386292k swap on /dev/cciss/c0d0p3. Priority:-1 extents:1 across:16386292k
Oct 6 22:46:03 dbs-n2 kernel: IA-32 Microcode Update Driver: v1.14-xen
Oct 6 22:46:03 dbs-n2 kernel: Loading iSCSI transport class v2.0-871.
Oct 6 22:46:03 dbs-n2 kernel: 802.1Q VLAN Support v1.8 Ben Greear
Oct 6 22:46:03 dbs-n2 kernel: All bugs added by David S. Miller
Oct 6 22:46:03 dbs-n2 kernel: cxgb3i: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.
Oct 6 22:46:03 dbs-n2 kernel: iscsi: registered transport (cxgb3i)
Oct 6 22:46:04 dbs-n2 kernel: Broadcom NetXtreme II CNIC Driver cnic v2.0.0 (March 21, 2009)
Oct 6 22:46:04 dbs-n2 kernel: cnic: Added CNIC device: eth0
Oct 6 22:46:04 dbs-n2 kernel: cnic: Added CNIC device: eth1
Oct 6 22:46:04 dbs-n2 kernel: cnic: Added CNIC device: eth2
Oct 6 22:46:04 dbs-n2 kernel: cnic: Added CNIC device: eth3
Oct 6 22:46:04 dbs-n2 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.0.1e (June 22, 2009)
Oct 6 22:46:04 dbs-n2 kernel: iscsi: registered transport (bnx2i)
Oct 6 22:46:04 dbs-n2 kernel: scsi1 : Broadcom Offload iSCSI Initiator
Oct 6 22:46:04 dbs-n2 kernel: scsi2 : Broadcom Offload iSCSI Initiator
Oct 6 22:46:04 dbs-n2 kernel: scsi3 : Broadcom Offload iSCSI Initiator
Oct 6 22:46:04 dbs-n2 kernel: scsi4 : Broadcom Offload iSCSI Initiator
Oct 6 22:46:04 dbs-n2 kernel: iscsi: registered transport (tcp)
Oct 6 22:46:04 dbs-n2 kernel: iscsi: registered transport (iser)
Oct 6 22:46:04 dbs-n2 kernel: bnx2: eth0: using MSIX
Oct 6 22:46:04 dbs-n2 kernel: ADDRCONF(NETDEV_UP): eth0: link is not ready
Oct 6 22:46:04 dbs-n2 kernel: bnx2i: LOM is not enable to offload iSCSI connections, dev=eth0
Oct 6 22:46:04 dbs-n2 kernel: bnx2i: LOM is not enable to offload iSCSI connections, dev=eth0
Oct 6 22:46:04 dbs-n2 kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON
Oct 6 22:46:04 dbs-n2 kernel: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Oct 6 22:46:04 dbs-n2 kernel: bnx2: eth1: using MSIX
Oct 6 22:46:04 dbs-n2 kernel: ADDRCONF(NETDEV_UP): eth1: link is not ready
Oct 6 22:46:04 dbs-n2 kernel: bnx2i: LOM is not enable to offload iSCSI connections, dev=eth1
Oct 6 22:46:04 dbs-n2 kernel: bnx2i: LOM is not enable to offload iSCSI connections, dev=eth1
Oct 6 22:46:04 dbs-n2 kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON
Oct 6 22:46:04 dbs-n2 kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
Oct 6 22:46:04 dbs-n2 kernel: device-mapper: dm-log-clustered: (built Jul 27 2009 16:28:42) installed
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: Core ver 2.10
Oct 6 22:46:04 dbs-n2 kernel: NET: Registered protocol family 31
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: HCI device and connection manager initialized
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: HCI socket layer initialized
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: L2CAP ver 2.8
Oct 6 22:46:04 dbs-n2 heartbeat: : info: Link dbs-n1:eth1 up.
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: L2CAP socket layer initialized
Oct 6 22:46:04 dbs-n2 heartbeat: : info: Status update for node dbs-n1: status up
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: RFCOMM socket layer initialized
Oct 6 22:46:04 dbs-n2 heartbeat: : WARN: G_CH_dispatch_int: Dispatch function for read child took too long to execute: 60 ms (> 50 ms) (GSource: 0x101b5858)
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: RFCOMM TTY layer initialized
Oct 6 22:46:04 dbs-n2 heartbeat: : info: Link 192.191.0.167:192.191.0.167 up.
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: RFCOMM ver 1.8
Oct 6 22:46:04 dbs-n2 heartbeat: : info: Status update for node 192.191.0.167: status ping
Oct 6 22:46:04 dbs-n2 harc[4690]: info: Running /etc/ha.d/rc.d/status status
Oct 6 22:46:04 dbs-n2 kernel: Bluetooth: HIDP (Human Interface Emulation) ver 1.1
Oct 6 22:46:04 dbs-n2 heartbeat: : WARN: G_CH_dispatch_int: Dispatch function for read child took too long to execute: 120 ms (> 50 ms) (GSource: 0x101b8c18)
Oct 6 22:46:05 dbs-n2 heartbeat: : info: Link dbs-n2:eth1 up.
Oct 6 22:46:05 dbs-n2 heartbeat: : WARN: G_SIG_dispatch: Dispatch function for SIGCHLD was delayed 110 ms (> 100 ms) before being called (GSource: 0x101b8458)
Oct 6 22:46:05 dbs-n2 heartbeat: : info: G_SIG_dispatch: started at 1718072208 should have started at 1718072197
Oct 6 22:46:05 dbs-n2 heartbeat: : info: Comm_now_up(): updating status to active
Oct 6 22:46:05 dbs-n2 heartbeat: : info: Local status now set to: 'active'
Oct 6 22:46:05 dbs-n2 heartbeat: : info: Starting child client "/usr/lib64/heartbeat/ipfail" (201,204)
Oct 6 22:46:05 dbs-n2 heartbeat: : info: Status update for node dbs-n1: status active
Oct 6 22:46:05 dbs-n2 heartbeat: : info: Starting "/usr/lib64/heartbeat/ipfail" as uid 201 gid 204 (pid 4707)
Oct 6 22:46:05 dbs-n2 harc[4708]: info: Running /etc/ha.d/rc.d/status status
Oct 6 22:46:13 dbs-n2 ipfail: : info: Ping node count is balanced.
Oct 6 22:46:16 dbs-n2 heartbeat: : info: local resource transition completed.
Oct 6 22:46:16 dbs-n2 heartbeat: : info: Initial resource acquisition complete (T_RESOURCES(us))
Oct 6 22:46:16 dbs-n2 heartbeat: : info: No local resources to acquire.
Oct 6 22:46:16 dbs-n2 heartbeat: : info: remote resource transition completed.
Oct 6 22:46:59 dbs-n2 heartbeat: : info: dbs-n1 wants to go standby
Oct 6 22:46:59 dbs-n2 heartbeat: : info: standby: acquire resources from dbs-n1
Oct 6 22:46:59 dbs-n2 heartbeat: : info: acquire local HA resources (standby).
Oct 6 22:46:59 dbs-n2 heartbeat: : info: local HA resource acquisition completed (standby).
Oct 6 22:46:59 dbs-n2 heartbeat: : info: Standby resource acquisition done .
Oct 6 22:47:00 dbs-n2 heartbeat: : info: remote resource transition completed.
Oct 6 22:49:08 dbs-n2 ipfail: : info: Telling other node that we have more visible ping nodes.
Oct 6 22:49:13 dbs-n2 heartbeat: : info: dbs-n1 wants to go standby
Oct 6 22:49:13 dbs-n2 ipfail: : info: Ping node count is balanced.
Oct 6 22:49:25 dbs-n2 heartbeat: : info: standby: acquire resources from dbs-n1
Oct 6 22:49:25 dbs-n2 heartbeat: : info: acquire all HA resources (standby).
Oct 6 22:49:25 dbs-n2 ResourceManager[4812]: info: Acquiring resource group: dbs-n1 IPaddr::192.191.0.22/24/eth0 Filesystem::/dev/sda1::/clusdsk::ext3 oracledb
Oct 6 22:49:25 dbs-n2 IPaddr[4839]: INFO: Resource is stopped
Oct 6 22:49:25 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/IPaddr 192.191.0.22/24/eth0 start
Oct 6 22:49:26 dbs-n2 IPaddr[4937]: INFO: Using calculated netmask for 192.191.0.22: 255.255.255.0
Oct 6 22:49:26 dbs-n2 IPaddr[4937]: INFO: eval ifconfig eth0:0 192.191.0.22 netmask 255.255.255.0 broadcast 192.191.0.255
Oct 6 22:49:26 dbs-n2 kernel: send_arp uses obsolete (PF_INET,SOCK_PACKET)
Oct 6 22:49:26 dbs-n2 IPaddr[4908]: INFO: Success
Oct 6 22:49:26 dbs-n2 Filesystem[5055]: WARNING: Couldn't find device . Expected /dev/??? to exist
Oct 6 22:49:26 dbs-n2 Filesystem[5044]: INFO: Resource is stopped
Oct 6 22:49:26 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/Filesystem /dev/sda1 /clusdsk ext3 start
Oct 6 22:49:26 dbs-n2 Filesystem[5130]: INFO: Running start for /dev/sda1 on /clusdsk
Oct 6 22:49:26 dbs-n2 Filesystem[5130]: ERROR: Couldn't find device . Expected /dev/??? to exist
Oct 6 22:49:26 dbs-n2 Filesystem[5119]: ERROR: Illegal argument
Oct 6 22:49:26 dbs-n2 ResourceManager[4812]: ERROR: Return code 2 from /etc/ha.d/resource.d/Filesystem
Oct 6 22:49:26 dbs-n2 ResourceManager[4812]: CRIT: Giving up resources due to failure of Filesystem::/dev/sda1::/clusdsk::ext3
Oct 6 22:49:26 dbs-n2 ResourceManager[4812]: info: Releasing resource group: dbs-n1 IPaddr::192.191.0.22/24/eth0 Filesystem::/dev/sda1::/clusdsk::ext3 oracledb
Oct 6 22:49:26 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:26 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:27 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:27 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:27 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:28 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:28 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:28 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:29 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:29 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:29 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:30 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:30 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:31 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:32 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:32 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:32 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:33 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:33 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:33 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:34 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:34 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:34 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:35 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:35 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:35 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:36 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:36 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:36 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: info: Retrying failed stop operation
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: ERROR: Resource script for oracledb probably not LSB-compliant.
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: WARN: it (oracledb) MUST succeed on a stop when already stopped
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: WARN: Machine reboot narrowly avoided!
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/Filesystem /dev/sda1 /clusdsk ext3 stop
Oct 6 22:49:37 dbs-n2 Filesystem[5601]: WARNING: Couldn't find device . Expected /dev/??? to exist
Oct 6 22:49:37 dbs-n2 Filesystem[5601]: INFO: Running stop for /dev/sda1 on /clusdsk
Oct 6 22:49:37 dbs-n2 Filesystem[5590]: INFO: Success
Oct 6 22:49:37 dbs-n2 ResourceManager[4812]: info: Running /etc/ha.d/resource.d/IPaddr 192.191.0.22/24/eth0 stop
Oct 6 22:49:37 dbs-n2 IPaddr[5707]: INFO: ifconfig eth0:0 down
Oct 6 22:49:37 dbs-n2 IPaddr[5678]: INFO: Success
Oct 6 22:49:37 dbs-n2 heartbeat: : info: all HA resource acquisition completed (standby).
Oct 6 22:49:37 dbs-n2 heartbeat: : info: Standby resource acquisition done .
Oct 6 22:49:38 dbs-n2 heartbeat: : info: remote resource transition completed.
Oct 6 22:49:46 dbs-n2 gpm[5749]: *** info :
Oct 6 22:49:46 dbs-n2 gpm[5749]: Started gpm successfully. Entered daemon mode.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Found user 'avahi' (UID 70) and group 'avahi' (GID 70).
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Successfully dropped root privileges.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: avahi-daemon 0.6.16 starting up.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Successfully called chroot().
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Successfully dropped remaining capabilities.
Oct 6 22:49:47 dbs-n2 kernel: Bridge firewalling registered
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Loading service file /services/sftp-ssh.service.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: New relevant interface eth1.IPv6 for mDNS.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface eth1.IPv6 with address fe80::6ab5:99ff:fe78:7f24.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: New relevant interface eth1.IPv4 for mDNS.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface eth1.IPv4 with address 10.9.0.21.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: New relevant interface eth0.IPv6 for mDNS.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::6ab5:99ff:fe78:7f22.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: New relevant interface eth0.IPv4 for mDNS.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.191.0.21.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Network interface enumeration completed.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Registering new address record for fe80::6ab5:99ff:fe78:7f24 on eth1.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Registering new address record for 10.9.0.21 on eth1.
Oct 6 22:49:47 dbs-n2 kernel: ip_tables: (C) 2000-2006 Netfilter Core Team
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Registering new address record for fe80::6ab5:99ff:fe78:7f22 on eth0.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Registering new address record for 192.191.0.21 on eth0.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: Registering HINFO record with values 'X86_64'/'LINUX'.
Oct 6 22:49:47 dbs-n2 avahi-daemon[5871]: New relevant interface virbr0.IPv4 for mDNS.
Oct 6 22:49:47 dbs-n2 kernel: Netfilter messages via NETLINK v0.30.
Oct 6 22:49:48 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface virbr0.IPv4 with address 192.168.122.1.
Oct 6 22:49:48 dbs-n2 kernel: ip_conntrack version 2.4 (8192 buckets, 65536 max) - 304 bytes per conntrack
Oct 6 22:49:48 dbs-n2 avahi-daemon[5871]: Registering new address record for 192.168.122.1 on virbr0.
Oct 6 22:49:48 dbs-n2 avahi-daemon[5871]: New relevant interface virbr0.IPv6 for mDNS.
Oct 6 22:49:48 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface virbr0.IPv6 with address fe80::200:ff:fe00:0.
Oct 6 22:49:48 dbs-n2 avahi-daemon[5871]: Registering new address record for fe80::200:ff:fe00:0 on virbr0.
Oct 6 22:49:49 dbs-n2 dnsmasq[6474]: started, version 2.45 cachesize 150
Oct 6 22:49:49 dbs-n2 dnsmasq[6474]: compile time options: IPv6 GNU-getopt no-ISC-leasefile no-DBus no-I18N TFTP
Oct 6 22:49:49 dbs-n2 dnsmasq[6474]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h
Oct 6 22:49:49 dbs-n2 dnsmasq[6474]: reading /etc/resolv.conf
Oct 6 22:49:49 dbs-n2 dnsmasq[6474]: using nameserver 192.191.0.20#53
Oct 6 22:49:49 dbs-n2 dnsmasq[6474]: read /etc/hosts - 6 addresses
Oct 6 22:49:49 dbs-n2 avahi-daemon[5871]: Server startup complete. Host name is dbs-n2.local. Local service cookie is 287356982.
Oct 6 22:49:50 dbs-n2 avahi-daemon[5871]: Service "SFTP File Transfer on dbs-n2" (/services/sftp-ssh.service) successfully established.
Oct 6 22:49:51 dbs-n2 xenstored: Checking store ...
Oct 6 22:49:52 dbs-n2 xenstored: Checking store complete.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: Withdrawing address record for 192.191.0.21 on eth0.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: Leaving mDNS multicast group on interface eth0.IPv4 with address 192.191.0.21.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: iface.c: interface_mdns_mcast_join() called but no local address available.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: Interface eth0.IPv4 no longer relevant for mDNS.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: Withdrawing address record for fe80::6ab5:99ff:fe78:7f22 on eth0.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: Leaving mDNS multicast group on interface eth0.IPv6 with address fe80::6ab5:99ff:fe78:7f22.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: iface.c: interface_mdns_mcast_join() called but no local address available.
Oct 6 22:49:52 dbs-n2 avahi-daemon[5871]: Interface eth0.IPv6 no longer relevant for mDNS.
Oct 6 22:49:52 dbs-n2 kernel: device vif0.0 entered promiscuous mode
Oct 6 22:49:52 dbs-n2 kernel: xenbr0: topology change detected, propagating
Oct 6 22:49:52 dbs-n2 kernel: xenbr0: port 1(vif0.0) entering forwarding state
Oct 6 22:49:52 dbs-n2 kernel: bnx2: peth0: using MSIX
Oct 6 22:49:52 dbs-n2 kernel: ADDRCONF(NETDEV_UP): peth0: link is not ready
Oct 6 22:49:52 dbs-n2 kernel: bnx2i: LOM is not enable to offload iSCSI connections, dev=peth0
Oct 6 22:49:53 dbs-n2 heartbeat: : ERROR: glib: Error sending packet: Network is unreachable
Oct 6 22:49:53 dbs-n2 heartbeat: : info: glib: euid=99 egid=99
Oct 6 22:49:53 dbs-n2 heartbeat: : ERROR: write_child: write failure on ping 192.191.0.167.: Network is unreachable
Oct 6 22:49:54 dbs-n2 kernel: bnx2i: LOM is not enable to offload iSCSI connections, dev=peth0
Oct 6 22:49:55 dbs-n2 heartbeat: : ERROR: glib: Error sending packet: Network is unreachable
Oct 6 22:49:55 dbs-n2 heartbeat: : info: glib: euid=99 egid=99
Oct 6 22:49:55 dbs-n2 heartbeat: : ERROR: write_child: write failure on ping 192.191.0.167.: Network is unreachable
Oct 6 22:49:55 dbs-n2 kernel: bnx2: peth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON
Oct 6 22:49:55 dbs-n2 kernel: ADDRCONF(NETDEV_CHANGE): peth0: link becomes ready
Oct 6 22:49:56 dbs-n2 kernel: device peth0 entered promiscuous mode
Oct 6 22:49:56 dbs-n2 kernel: xenbr0: topology change detected, propagating
Oct 6 22:49:56 dbs-n2 kernel: xenbr0: port 2(peth0) entering forwarding state
Oct 6 22:49:57 dbs-n2 avahi-daemon[5871]: New relevant interface eth0.IPv4 for mDNS.
Oct 6 22:49:57 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.191.0.21.
Oct 6 22:49:57 dbs-n2 avahi-daemon[5871]: Registering new address record for 192.191.0.21 on eth0.
Oct 6 22:49:57 dbs-n2 heartbeat: : WARN: node 192.191.0.167: is dead
Oct 6 22:49:57 dbs-n2 heartbeat: : info: Link 192.191.0.167:192.191.0.167 dead.
Oct 6 22:49:57 dbs-n2 ipfail: : info: Status update: Node 192.191.0.167 now has status dead
Oct 6 22:49:57 dbs-n2 harc[6780]: info: Running /etc/ha.d/rc.d/status status
Oct 6 22:49:57 dbs-n2 kernel: Hangcheck: starting hangcheck timer 0.9.0 (tick is 30 seconds, margin is 180 seconds).
Oct 6 22:49:57 dbs-n2 kernel: Hangcheck: Using monotonic_clock().
Oct 6 22:49:58 dbs-n2 modclusterd: startup succeeded
Oct 6 22:49:58 dbs-n2 ipfail: : info: NS: We are dead. :<
Oct 6 22:49:58 dbs-n2 ipfail: : info: Link Status update: Link 192.191.0.167/192.191.0.167 now has status dead
Oct 6 22:49:58 dbs-n2 oddjobd: oddjobd startup succeeded
Oct 6 22:49:58 dbs-n2 saslauthd[6953]: detach_tty : master pid is: 6953
Oct 6 22:49:58 dbs-n2 saslauthd[6953]: ipc_init : listening on socket: /var/run/saslauthd/mux
Oct 6 22:49:58 dbs-n2 avahi-daemon[5871]: New relevant interface eth0.IPv6 for mDNS.
Oct 6 22:49:58 dbs-n2 avahi-daemon[5871]: Joining mDNS multicast group on interface eth0.IPv6 with address fe80::6ab5:99ff:fe78:7f22.
Oct 6 22:49:58 dbs-n2 avahi-daemon[5871]: Registering new address record for fe80::6ab5:99ff:fe78:7f22 on eth0.
Oct 6 22:49:58 dbs-n2 ricci: startup succeeded
Oct 6 22:49:58 dbs-n2 smartd[6976]: smartd version 5.38 Copyright (C) 2002-8 Bruce Allen
Oct 6 22:49:58 dbs-n2 smartd[6976]: Home page is http://smartmontools.sourceforge.net/
Oct 6 22:49:58 dbs-n2 smartd[6976]: Opened configuration file /etc/smartd.conf
Oct 6 22:49:58 dbs-n2 smartd[6976]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Oct 6 22:49:58 dbs-n2 smartd[6976]: Problem creating device name scan list
Oct 6 22:49:58 dbs-n2 smartd[6976]: Problem creating device name scan list
Oct 6 22:49:58 dbs-n2 smartd[6976]: In the system's table of devices NO devices found to scan
Oct 6 22:49:58 dbs-n2 smartd[6976]: Monitoring 0 ATA and 0 SCSI devices
Oct 6 22:49:58 dbs-n2 smartd[6978]: smartd has fork()ed into background mode. New PID=6978.
Oct 6 22:49:59 dbs-n2 ipfail: : info: We are dead. :<
Oct 6 22:49:59 dbs-n2 ipfail: : info: Asking other side for ping node count.
Oct 6 22:50:00 dbs-n2 pcscd: winscard.c:304:SCardConnect() Reader E-Gate 0 0 Not Found
Oct 6 22:50:00 dbs-n2 last message repeated 3 times
Oct 6 22:50:00 dbs-n2 heartbeat: : info: Link 192.191.0.167:192.191.0.167 up.
Oct 6 22:50:00 dbs-n2 heartbeat: : info: Status update for node 192.191.0.167: status ping
Oct 6 22:50:00 dbs-n2 ipfail: : info: Link Status update: Link 192.191.0.167/192.191.0.167 now has status up
Oct 6 22:50:00 dbs-n2 ipfail: : info: Status update: Node 192.191.0.167 now has status ping
Oct 6 22:50:00 dbs-n2 ipfail: : info: A ping node just came up.
Oct 6 22:50:01 dbs-n2 ipfail: : info: Asking other side for ping node count.
Oct 6 22:50:01 dbs-n2 ipfail: : info: Giving up because we were told that we have less ping nodes.
Oct 6 22:50:01 dbs-n2 ipfail: : info: Delayed giveup in 4 seconds.
Oct 6 22:50:04 dbs-n2 ipfail: : info: Aborted delayed giveup (3)
Oct 6 22:50:07 dbs-n2 hb_standby[7146]: Going standby .
Oct 6 22:50:08 dbs-n2 heartbeat: : info: dbs-n2 wants to go standby
Oct 6 22:50:08 dbs-n2 heartbeat: : info: standby: dbs-n1 can take our foreign resources
Oct 6 22:50:08 dbs-n2 heartbeat: : info: give up foreign HA resources (standby).
Oct 6 22:50:08 dbs-n2 ResourceManager[7173]: info: Releasing resource group: dbs-n1 IPaddr::192.191.0.22/24/eth0 Filesystem::/dev/sda1::/clusdsk::ext3 oracledb
Oct 6 22:50:08 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:08 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:09 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:09 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:09 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:10 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:10 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:10 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:11 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:12 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:12 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:13 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:13 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:13 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:14 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:14 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:14 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:15 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:15 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:15 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:16 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:16 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:16 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:17 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:17 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:17 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:18 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:18 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:18 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: info: Retrying failed stop operation
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: ERROR: Resource script for oracledb probably not LSB-compliant.
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: WARN: it (oracledb) MUST succeed on a stop when already stopped
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: WARN: Machine reboot narrowly avoided!
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/Filesystem /dev/sda1 /clusdsk ext3 stop
Oct 6 22:50:19 dbs-n2 Filesystem[7590]: WARNING: Couldn't find device . Expected /dev/??? to exist
Oct 6 22:50:19 dbs-n2 Filesystem[7590]: INFO: Running stop for /dev/sda1 on /clusdsk
Oct 6 22:50:19 dbs-n2 Filesystem[7579]: INFO: Success
Oct 6 22:50:19 dbs-n2 ResourceManager[7173]: info: Running /etc/ha.d/resource.d/IPaddr 192.191.0.22/24/eth0 stop
Oct 6 22:50:19 dbs-n2 IPaddr[7667]: INFO: Success
Oct 6 22:50:19 dbs-n2 heartbeat: : info: foreign HA resource release completed (standby).
Oct 6 22:50:19 dbs-n2 heartbeat: : info: Local standby process completed .
Oct 6 22:50:32 dbs-n2 heartbeat: : WARN: 1 lost packet(s) for
Oct 6 22:50:32 dbs-n2 heartbeat: : info: remote resource transition completed.
Oct 6 22:50:32 dbs-n2 heartbeat: : info: No pkts missing from dbs-n1!
Oct 6 22:50:32 dbs-n2 heartbeat: : info: Other node completed standby takeover of foreign resources.
Oct 6 22:50:45 dbs-n2 ipfail: : info: Ping node count is balanced.
Oct 6 22:50:45 dbs-n2 ipfail: : info: Giving up foreign resources (auto_failback).
Oct 6 22:50:45 dbs-n2 ipfail: : info: Delayed giveup in 4 seconds.
Oct 6 22:50:49 dbs-n2 ipfail: : info: giveup() called (timeout worked)
Oct 6 22:50:49 dbs-n2 heartbeat: : info: dbs-n2 wants to go standby
Oct 6 22:50:50 dbs-n2 heartbeat: : info: standby: dbs-n1 can take our foreign resources
Oct 6 22:50:50 dbs-n2 heartbeat: : info: give up foreign HA resources (standby).
Oct 6 22:50:50 dbs-n2 ResourceManager[7733]: info: Releasing resource group: dbs-n1 IPaddr::192.191.0.22/24/eth0 Filesystem::/dev/sda1::/clusdsk::ext3 oracledb
Oct 6 22:50:50 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:50 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:51 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:51 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:51 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:52 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:52 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:52 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:53 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:53 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:53 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:54 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:54 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:54 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:55 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:55 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:55 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:56 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:56 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:56 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:57 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:57 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:57 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:58 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:58 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:58 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:50:59 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:50:59 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:50:59 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: info: Retrying failed stop operation
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/oracledb stop
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: ERROR: Return code 1 from /etc/ha.d/resource.d/oracledb
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: ERROR: Resource script for oracledb probably not LSB-compliant.
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: WARN: it (oracledb) MUST succeed on a stop when already stopped
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: WARN: Machine reboot narrowly avoided!
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/Filesystem /dev/sda1 /clusdsk ext3 stop
Oct 6 22:51:01 dbs-n2 Filesystem[8149]: WARNING: Couldn't find device . Expected /dev/??? to exist
Oct 6 22:51:01 dbs-n2 Filesystem[8149]: INFO: Running stop for /dev/sda1 on /clusdsk
Oct 6 22:51:01 dbs-n2 Filesystem[8138]: INFO: Success
Oct 6 22:51:01 dbs-n2 ResourceManager[7733]: info: Running /etc/ha.d/resource.d/IPaddr 192.191.0.22/24/eth0 stop
Oct 6 22:51:01 dbs-n2 IPaddr[8226]: INFO: Success
Oct 6 22:51:01 dbs-n2 heartbeat: : info: foreign HA resource release completed (standby).
Oct 6 22:51:01 dbs-n2 heartbeat: : info: Local standby process completed .
Oct 6 22:51:03 dbs-n2 heartbeat: : WARN: Standby in progress- new request from dbs-n1 ignored
Oct 6 22:51:14 dbs-n2 heartbeat: : WARN: 1 lost packet(s) for
Oct 6 22:51:14 dbs-n2 heartbeat: : info: remote resource transition completed.
Oct 6 22:51:14 dbs-n2 heartbeat: : info: No pkts missing from dbs-n1!
Oct 6 22:51:14 dbs-n2 heartbeat: : info: Other node completed standby takeover of foreign resources.
Oct 6 22:51:44 dbs-n2 heartbeat: : info: dbs-n1 wants to go standby
Oct 6 22:51:45 dbs-n2 heartbeat: : info: standby: acquire resources from dbs-n1
Oct 6 22:51:45 dbs-n2 heartbeat: : info: acquire local HA resources (standby).
Oct 6 22:51:45 dbs-n2 heartbeat: : info: local HA resource acquisition completed (standby).
Oct 6 22:51:45 dbs-n2 heartbeat: : info: Standby resource acquisition done .
Oct 6 22:51:45 dbs-n2 heartbeat: : info: remote resource transition completed.
#
大家能帮我分析问题出在哪里，及如何确认和排查吗？
wang.hy8166
Jianyun_e442cb
56 消息
1
2013年10月24日 22:00
一般情况下，如果通过重启服务器能解决的问题，极少会是存储问题。
另外，一个基本经验是：有存储故障也不要重启存储来解决问题！重启存储一般只会把问题搞得更复杂，会破坏现场环境，把修复变得更困难，严重时数据丢失！
数据丢失是存储最严重的事故。
存储故障一定要找到原因，针对原因修复故障！不要轻易重启！
liulei_it
3.2K 消息
0
2013年10月24日 23:00
一般系统都会有“句柄”，即使存储中的数据没有了但“句柄”还在运气好还能把丢失的文件找回来，重新启动OS那就啥都没有了。
hy8166
15 消息
0
2013年10月25日 00:00
嗯，看来我得恶补下了。
您有相关的案例文档供学习吗？
wang.hy8166
发件人： lei liu
发送时间： 2013-10-25 14:59
收件人： hy8166
主题： Re: - 存储突然链接失败，请教如何查找原因？
ECN
存储突然链接失败，请教如何查找原因？
reply from lei liu in 综合讨论区 - View the full discussion
一般系统都会有“句柄”，即使存储中的数据没有了但“句柄”还在运气好还能把丢失的文件找回来，重新启动OS那就啥都没有了。
Reply to this message by replying to this email, or go to the message on ECN
Start a new discussion in 综合讨论区 by email or at ECN
Following 存储突然链接失败，请教如何查找原因？ in these streams: Inbox
liulei_it
3.2K 消息
0
2013年10月27日 01:00
http://www.eygle.com/archives/2011/12/oracle_linux_rm_rf.html
http://www.eygle.com/archives/2011/12/oracle_lsof_recovery_database.html
以上两个是来自于业界著名的盖大师的经典案例。
cxemc
362 消息
0
2013年10月27日 20:00
硬件上去排查吧这个问题。