1

I have a problem with a server that is shutting down for some unknown reason. I check the logs each time it happens and cannot find anything of particular interest which would suggest why its going down.

Can anyone suggest anything I should do to help diagnose the problem?

2
  • 1
    If there is nothing in the logs (even no shutdown message) it is probably a hardware issue. Mar 3, 2013 at 11:52
  • Does cat /proc/sys/kernel/panic say zero? Or cat /etc/sysctl.conf | grep kernel\.panic
    – Runium
    Mar 3, 2013 at 12:30

1 Answer 1

3

Sounds like a hardware problem.

Have you added anything to the system? E.g. RAM, new GPU, HDD, PCI cards?

Physical:

  • Check all cables.
  • Check mounting of RAM.
  • Check mounting of all PCI's.
  • Remount CPU (If nothing else works).
  • Check powercables, both external and internal.

[Edit]: And as pointed out by @vonbrand
Ensure the cooling etc. and do a cleanup if needed.

If you're not, check status by lm_sensors and e.g. add a alarm system for critical values. Here you can also check fan status - and perhaps voltages. If no voltages shown you could check if BIOS reports it. Then check papers on your HW that levels are correct. See below for voltmeter check.


Update BIOS.

You say:

[…] cannot find anything of particular interest which would suggest why its going down.

But are there other data in

/var/log/messages
/var/log/debug
/var/log/kern.log
...

that suggest problems with the system on boot etc?

Do you have mcelog running (Machine Check Exception)? If not install it.

Check HDD S.M.A.R.T. data by using smartmontools. Run memtest, stresstest, etc.

Check dmesg while running tests.

Use one of the many boot CD's e.g.:

And run tests on them you find appropriate.

If it is a Server server - in a offside location, check that there is no janitor using the power outlet to recharge his cellphone.


[Edit2]: PSU check.
Only an extra note on PSU check. An off board check could be good to do – in which case, - if it is an ATX -, you have to remember to shortcut Power on, (typically green), with ground, (typically black), to make the PSU enter on-stage. (With a spare wire, paper clip, etc.)

Then you can measure by placing one voltmeter connector to ground (black) and the other to the coloured ones. Then if you read a positive value outside of ±5% range, or negative outside of ±10% you're typically in trouble. (4.1.4 Voltage Tolerances). The 12V would be easiest to check which should be in the range of 11.40 to 12.60 volt. On a typical ATX that would be:

* Yellow: +12  V;  value ⊆ { 11.40 , …,  12.60 }
* Red   : + 5  V;  value ⊆ {  4.75 , …,   5.50 }
* Orange: + 3.3V;  value ⊆ {  3.135, …,   3.465}
* Brown : + 3.3V;  value ⊆ {  3.135, …,   3.465}
* Blue  : -12  V;  value ⊆ {-13.2  , …, -10.8  }

Check paper of your PSU vendor or more general spec and your Mother Board spec.


5
  • 1
    Check fans, CPU cooling. Shovel out dirt, make sure airflow in not obstructed.
    – vonbrand
    Mar 3, 2013 at 16:31
  • 1
    Check the PSU as well.
    – schaiba
    Mar 3, 2013 at 17:57
  • @schaiba: Yes. Hopefully lm_sensors would track this. Or do you think using a regular voltmeter on the outputs? Depending on location one could have e.g. brownouts. Noise should be regulated by the PSU though e.g. decent grounding is imperative on a critical system. Then one would use ferroresonant regulator etc. as well.
    – Runium
    Mar 3, 2013 at 18:49
  • I would go with measurement apparatus, namely a wattmeter, since lower wattage than normal is a sign of an old/failing PSU. Even simpler, test with a PSU that's known to work.
    – schaiba
    Mar 3, 2013 at 19:11
  • @schaiba: Not sure what you mean. You can't measure amperage without load and with load you for one would have to know how much is expected which would fluctuate immensely. Also there is always varying quality etc on delivery. Read e.g. max wattage.
    – Runium
    Mar 3, 2013 at 21:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .