1

we have cluster with redhat machines version 7.2 on pluto05 machine we saw that machine rebooted before one day

uptime
 13:51:06 up 1 day,  1:46,  2 users,  load average: 0.30, 0.14, 0.08

and from messages files we see the following:

Apr 22 12:05:04 pluto05 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="1013" x-info="http://www.rsyslog.com"] start
Apr 22 12:04:51 pluto05 journal: Runtime journal is using 8.0M (max allowed 1.5G, trying to leave 2.3G free of 15.6G available → current limit 1.5G).
Apr 22 12:04:51 pluto05 kernel: Initializing cgroup subsys cpuset
Apr 22 12:04:51 pluto05 kernel: Initializing cgroup subsys cpu
Apr 22 12:04:51 pluto05 kernel: Initializing cgroup subsys cpuacct
Apr 22 12:04:51 pluto05 kernel: Linux version 3.10.0-327.el7.x86_64 (mockbuild@x86-034.build.eng.bos.redhat.com) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Oct 29 17:29:29 EDT 2015
Apr 22 12:04:51 pluto05 kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/vg00-lv_root ro crashkernel=auto rd.lvm.lv=vg00/lv_root rd.lvm.lv=vg00/lv_swap rhgb quiet LANG=en_US.UTF-8
Apr 22 12:04:51 pluto05 kernel: Disabled fast string operations
Apr 22 12:04:51 pluto05 kernel: e820: BIOS-provided physical RAM map:
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009efff] usable
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x000000000009f000-0x000000000009ffff] reserved
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000000dc000-0x00000000000fffff] reserved
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x0000000000100000-0x00000000bfedffff] usable
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000bfee0000-0x00000000bfefefff] ACPI data
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000bfeff000-0x00000000bfefffff] ACPI NVS
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000bff00000-0x00000000bfffffff] usable
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000f0000000-0x00000000f7ffffff] reserved
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x00000000fffe0000-0x00000000ffffffff] reserved
Apr 22 12:04:51 pluto05 kernel: BIOS-e820: [mem 0x0000000100000000-0x000000083fffffff] usable
Apr 22 12:04:51 pluto05 kernel: NX (Execute Disable) protection: active
Apr 22 12:04:51 pluto05 kernel: SMBIOS 2.4 present.
Apr 22 12:04:51 pluto05 kernel: Hypervisor detected: VMware
Apr 22 12:04:51 pluto05 kernel: AGP: No AGP bridge found
Apr 22 12:04:51 pluto05 kernel: e820: last_pfn = 0x840000 max_arch_pfn = 0x400000000
Apr 22 12:04:51 pluto05 kernel: x86 PAT enabled: cpu 0, old 0x7010600070106, new 0x7010600070106
Apr 22 12:04:51 pluto05 kernel: total RAM covered: 32768M
Apr 22 12:04:51 pluto05 kernel: Found optimal setting for mtrr clean up
Apr 22 12:04:51 pluto05 kernel: gran_size: 64K #011chunk_size: 64K #011num_reg: 6  #011lose cover RAM: 0G
Apr 22 12:04:51 pluto05 kernel: e820: last_pfn = 0xc0000 max_arch_pfn = 0x400000000
Apr 22 12:04:51 pluto05 kernel: found SMP MP-table at [mem 0x000f6b30-0x000f6b3f] mapped at [ffff8800000f6b30]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0x00000000-0x000fffff]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0x83fe00000-0x83fffffff]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0x83c000000-0x83fdfffff]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0x800000000-0x83bffffff]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0x00100000-0xbfedffff]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0xbff00000-0xbfffffff]
Apr 22 12:04:51 pluto05 kernel: init_memory_mapping: [mem 0x100000000-0x7ffffffff]
Apr 22 12:04:51 pluto05 kernel: RAMDISK: [mem 0x347a9000-0x363ccfff]

but I cant put the finger on why machine suddenly rebooted ?

note - this machine not rebooted by user

please advice how to capture the issue from the messages file and if this info from messages not help

then how to capture the issue on the next time ?

last | more
reboot   system boot  3.10.0-327.el7.x Sun Apr 22 12:04 - 13:54 (1+01:49)

/var/log/messages before reboot

Apr 22 11:30:01 pluto05 systemd: Starting Session 2279 of user root.
Apr 22 11:30:01 pluto05 systemd: Removed slice user-0.slice.
Apr 22 11:30:01 pluto05 systemd: Stopping user-0.slice.
Apr 22 11:30:09 pluto05 journal: Suppressed 918 messages from /system.slice/avahi-daemon.service
Apr 22 11:32:08 pluto05 journal: Suppressed 4554 messages from /system.slice/avahi-daemon.service
Apr 22 11:33:28 pluto05 journal: Suppressed 110 messages from /system.slice/avahi-daemon.service
Apr 22 11:36:28 pluto05 journal: Suppressed 110 messages from /system.slice/avahi-daemon.service
Apr 22 11:40:01 pluto05 systemd: Created slice user-0.slice.
Apr 22 11:40:01 pluto05 systemd: Starting user-0.slice.
Apr 22 11:40:01 pluto05 systemd: Started Session 2280 of user root.
Apr 22 11:40:01 pluto05 systemd: Starting Session 2280 of user root.
Apr 22 11:40:01 pluto05 systemd: Removed slice user-0.slice.
Apr 22 11:40:01 pluto05 systemd: Stopping user-0.slice.
Apr 22 11:50:01 pluto05 systemd: Created slice user-0.slice.
Apr 22 11:50:01 pluto05 systemd: Starting user-0.slice.
Apr 22 11:50:01 pluto05 systemd: Started Session 2281 of user root.
Apr 22 11:50:01 pluto05 systemd: Starting Session 2281 of user root.
Apr 22 11:50:01 pluto05 systemd: Removed slice user-0.slice.
Apr 22 11:50:01 pluto05 systemd: Stopping user-0.slice.
Apr 22 11:58:35 pluto05 journal: Suppressed 5463 messages from /system.slice/avahi-daemon.service
Apr 22 11:59:05 pluto05 journal: Suppressed 9 messages from /system.slice/avahi-daemon.service
Apr 22 11:59:35 pluto05 journal: Suppressed 1019 messages from /system.slice/avahi-daemon.service
Apr 22 12:00:01 pluto05 systemd: Created slice user-0.slice.
Apr 22 12:00:01 pluto05 systemd: Starting user-0.slice.
Apr 22 12:00:01 pluto05 systemd: Started Session 2282 of user root.
Apr 22 12:00:01 pluto05 systemd: Starting Session 2282 of user root.
Apr 22 12:00:01 pluto05 systemd: Removed slice user-0.slice.
Apr 22 12:00:01 pluto05 systemd: Stopping user-0.slice.
Apr 22 12:00:16 pluto05 journal: Suppressed 817 messages from /system.slice/avahi-daemon.service
Apr 22 12:00:56 pluto05 journal: Suppressed 211 messages from /system.slice/avahi-daemon.service
Apr 22 12:01:01 pluto05 systemd: Created slice user-0.slice.
Apr 22 12:01:01 pluto05 systemd: Starting user-0.slice.
Apr 22 12:01:01 pluto05 systemd: Started Session 2283 of user root.
Apr 22 12:01:01 pluto05 systemd: Starting Session 2283 of user root.
Apr 22 12:01:01 pluto05 systemd: Removed slice user-0.slice.
Apr 22 12:01:01 pluto05 systemd: Stopping user-0.slice.
Apr 22 12:02:43 pluto05 journal: Suppressed 615 messages from /system.slice/avahi-daemon.service

I am also see the following strange in the messages file

Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.21.247.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.39.79.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.58.63.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.31.230.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.55.245.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.60.182.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.245.252.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.45.202.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.47.245.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.45.241.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.58.13.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.60.174.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.61.38.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.58.91.
Apr 22 03:13:47 pluto05 avahi-daemon[1023]: Invalid response packet from host 142.34.42.241.
  • looks like those entries from /var/log/messages are from part of the boot up sequence. could you paste the entries there from when it was shutting down? Also if you could elaborate on what it's used for and it's specs, it may be easier to narrow down common reasons that way (OOM for example) – RobotJohnny Apr 23 '18 at 14:16
  • see my update , hope this help to give more info – yael Apr 23 '18 at 14:30
  • If you have kdump enabled on the system, a dmesg dump taken just before the crash is stored in /var/crash. – Raman Sailopal Apr 23 '18 at 14:48
  • we not have kdump on this linux – yael Apr 23 '18 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.