3

I have a server:

CentOS Linux release 7.3.1611 (Core)
3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6 23:06:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

I think its network connection cutout at one point (its back now). I haven't been able to find anything in /var/log/messages- maybe I just don't know what to look for?

Essesntially I'm looking for two things: If there was a problem with the nic, If the server lost its internet connection.

The second one is obviously harder to figure out (maybe impossible?). Obviously I should have some external monitoring solution, but from an educational perspective where would you look (locally on the host) to solve this mystery?

2
  • unless application, such as mail use internet, and if your local network was OK, I doubt there could be log somewhere to see that.
    – Archemar
    Jan 13, 2017 at 15:34
  • bummer, just curious if there was any sneaky forensics nix nerds know about. Jan 13, 2017 at 15:37

2 Answers 2

2

Check the ring kernel buffer (dmesg) - you should see information for network connectivity events.

2
  • Cool theres some neat stuff in there. I see mostly systemd stuff? Jan 13, 2017 at 15:43
  • dmesg displays information about hardware (connect/disconnect devices, drivers etc)
    – 13dimitar
    Jan 13, 2017 at 15:46
2

Your kernel logs (the most recent viewable by dmesg, older by journalctl -k or wherever configured in /etc/rsyslog.conf) will probably contain messages if a NIC loses its link. And of course you can easily find what those messages look like by unplugging the cable. Example:

Jan 13 11:16:33 Zia kernel: r8169 0000:07:01.0 lan: link down

Loss of Internet connectivity is harder, and typically you'd set something up to detect this (either external monitoring, or monitoring of outside machines from your machine). It's also not really a binary state—you can have partial Internet connectivity (can reach some hosts, but not all). You can look for clues, though:

  • messages from NTP in the journal/logs (e.g., about losing peers), or check the NTP peer stats (if enabled).
  • messages in journal/logs about failed network connections (e.g., if you had fetchmail running routinely, it'd complain it couldn't connect to your POP3 server)
  • VPN connections going down (I get plenty of OpenVPN logs when the Internet connection goes down).
  • sudden 0 load on web (etc.) servers. Many servers log all requests received to some server-specific log.
  • similarly, if you log system load (load average, run queue, etc.), quick drop to 0 is a clue.
  • lack of noise in firewall logs (at least, if you log blocked packets)
  • sudden lack of random worm attacks (I get plenty of failed authentication messages from various services I'm running, and not getting any for hours would be a good clue there was no Internet connectivity)
  • traffic levels, if you're monitoring (e.g., using Cacti, MRTG, collectd, etc.). If you're not, your ISP probably is—maybe they'd be willing to share?
  • if you have other machines that talk to this machine, check those machine's logs. Might have failure messages
  • if you offer services to other people, they might have logs.
  • if you have a support desk, they probably know (from all the calls they got).

Basically, you're doing detective work: you need to look for clues. Without monitoring, there isn't going to be an teh Interwebs is down log message.

1
  • 1
    OTOH Mac OS X spams the system.log with Internet connection appears to be offline messages when it is so.
    – thrig
    Jan 13, 2017 at 16:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .