6

I have Arch Linux running on Dell PowerEdge R210. It is not high loaded, only few scripts are running. There is no any software like web services -- only base system. The server has been running fine for a long time before and no changes have been made but it began shutting down at random times about once every few days. The server is protected by trusted UPS, but it keeps shutting down even beeing plugged directly without UPS so this is not a power problem. I have monitored temperature by executing sensors utility every minute until the server shuts down -- nothing wrong, all sensors keep showing something near 30 deegrees. So this is not a heat problem. The chassis has ipmi enabled so I can execute "power on" ipmi command every time after shutdown and the server boots and run normally.

There is nothing in "restart_cause":

$ ipmitool -H 10.5.5.32 -U root -I lanplus chassis restart_cause
System restart cause: unknown

There is nothing in chassis logs:

$ ipmitool -H 10.5.5.32 -U root -I lanplus sel list
   1 | 07/23/2019 | 06:33:43 | Event Logging Disabled #0x72 | Log area reset/cleared | Asserted
   2 | 07/24/2019 | 09:51:50 | Physical Security #0x73 | General Chassis intrusion () | Asserted
   3 |  Pre-Init  |0000000032| Physical Security #0x73 | General Chassis intrusion () | Asserted
   4 |  Pre-Init  |0000000037| Physical Security #0x73 | General Chassis intrusion () | Deasserted
   5 | 07/24/2019 | 11:29:10 | Physical Security #0x73 | General Chassis intrusion () | Asserted
   6 | 07/24/2019 | 11:29:15 | Physical Security #0x73 | General Chassis intrusion () | Deasserted

Here is "last" command output:

$ last -x
user     pts/0        10.5.5.25        Wed Jul 31 08:31 - 09:38  (01:07)
root     ttyS1                         Wed Jul 31 08:23 - 09:38  (01:14)
reboot   system boot  5.2.1-arch1-1-AR Wed Jul 31 08:23 - 09:38  (01:15)
shutdown system down  5.2.1-arch1-1-AR Tue Jul 30 20:17 - 08:23  (12:06)

There is nothing in system journal.

I have set kernel log level to debug level but only one message appears in console output just before shutting down:

reboot: Power down

Can anybody explain me what does "reboot: Power down" exactly mean? Who or what caused shutting down? Does anyone have any ideas how can I debug this very strange issue?

5

reboot: Power down probably indicates that the command reboot -fp is being executed at the very end of a normal shutdown process.

The last -x output indicates that the shutdown was probably an orderly one. If nobody was logged in at the time, it might have been triggered by an ACPI power button event, i.e. someone pressing the power button for less than 4 seconds.

The chassis log indicates the server chassis was opened a week or so ago. If it was not done by you, that might question the assumption of "no changes have been made".

  • Thank you for Your response, @telcoM. Chassis was opened by me, I was looking for any phisycal damage, reinstalled RAM modules and run the server with connected monitor to perform RAM tests. After the server began shutting down randomly. My "last -x" output is truncated. There are a lot of exactly same "shutdown" entries before. I am always logged in as root in console session and as user over remote ssh connection. And nobody else can log in. And I dont think that someone just trolling me by pressing the power button... – ity Jul 31 at 11:51
  • English is not my native language... can You please explain what do You mean by saying "the shutdown was probably an orderly one"? Did You mean that "reboot -fp" was executed by some script or software? As I understand in that case script should leave some log messages in systemd journal, but there is no such messages in my journal. – ity Jul 31 at 11:53
  • 1
    I meant that something may have caused the OS shutdown procedure to begin, and the last command of normal shutdown scripts is usually reboot -f with an option that defines the actual desired state: powerdown, halt or reboot. It does not look like the power was just removed, nor like the system had suddenly crashed. It might be something like echo o > /proc/sysrq-trigger, though. – telcoM Jul 31 at 12:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.