2

I have a setting where several machines share the same filesystem via NFS. Through a queueing system processing jobs can be submitted to several computing machines (with different properties).

Sometimes, a job will crash and leave behind a core file (with a name like core.1234).

Is there a way to find out which host generated that core file? What its hostname was?

(This is on Linux 64 bits, if it makes a difference).

1
  • I assume you run as the same user on each machine. If this cannot be determined afterwards from the core file, you could make different users on each machine and use a shared group with appropriate umask and permissions. Owner of the core then determines the machine uniquely
    – Anthon
    Nov 26 '13 at 10:40
5

On an ELF system, a core file is almost certainly a valid ELF file.

 $ readelf -a core
 ELF Header:
 Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
 Class:                             ELF32
 Data:                              2's complement, little endian
 Version:                           1 (current)
 OS/ABI:                            UNIX - System V
 ABI Version:                       0
 Type:                              CORE (Core file)
 Machine:                           Intel 80386
 [...]

A platform specific number of "notes" are added to a NOTES segment so that a debugger can find its way around, e.g. for Solaris see core(4), and you'll note the NT_UTSNAME structure which contains the data structure from the uname(2) syscall. elfdump -n is the way to read that, but as far as I know Solaris is the only OS that does this (and I suspect only Solaris 11 elfdump works as hoped).

A simple, though slightly fiddly and not-guaranteed way is to try and fish the HOST or HOSTNAME variables (set by some startup scripts and shells, bash at least sets HOSTNAME) out of the core dump environment. You can do this with gdb, though you need the original binary:

$ gdb /usr/bin/sleep core
[... snip ...]
(gdb) print (char ***) &environ
$1 = (char ***) 0x600bf8
(gdb) print $1[0][0]@10
$2 = {0x7fffffffd9c9 "HOST=myhostname", 0x7fffffffd9d9 "TERM=screen", 
0x7fffffffd9e5 "SHELL=/bin/csh", 
[...]

This prints a chunk of strings from the environ symbol. Though it's a horrible hack strings | grep HOSTNAME= just might work too.

So, short answer to "Is there a way to find out which host generated that core file" is: not easily, and not reliably on Linux.

FWIW, the relevant coredump code on Linux is in fs/binfmt_elf.c, and there is a hook to allow extra "notes" by way of ARCH_HAVE_EXTRA_ELF_NOTES, currently only used on PowerPC.)

A better plan altogether is to use sysctl to set the core file name on each client, as suggested by @jlliagre :

sysctl kernel.core_pattern="%h-%t-%e.core"

(sysctl and ferreting around in /proc are equivalent here, I prefer sysctl since changes can be kept documented in /etc/sysctl.conf and it's used on *BSD systems too.)

1
  • strings | grep HOSTNAME= is what I started using. This is interactive use; so it's OK if it fails a fraction of the time.
    – luispedro
    Nov 28 '13 at 14:24
4

You might run strings core.1234 and find if the hostname appears somewhere. Some OSes like Solaris do put the hostname in the core header but as far as I know, Linux doesn't do it so the strings method would be unreliable with it.

A better approach would be to configure the NFS clients to put their hostnames in the core file names by setting the /proc/sys/kernel/core_pattern file or some distribution specific configuration if core_pattern refer to an executable.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.