Is there a reason why /var/log/lastlog is a huge sparse file (1.1TB)?

Question

I have read some question, that ask advice how to rsync sparse files efficiently mentioning the files /var/log/lastlog and /var/log/faillog. Indeed I myself have stumpled over those files being an "issue" as their being backup via rsync turns them to become "unsparse".

What I hence wonder is, what is the need/backgrounding motivation to have those files as sparse, huge files (in my case it was 1.1TB)?

Also in relationship to this a follow up: Since I was assuming them to be logfiles I do not care about excesively I truncated those files, did I corrupt anything with truncating those files ?

Answer 1

What I hence wonder is, what is the need/backgrounding motivation to have those files as sparse, huge files (in my case it was 1.1TB)?

This is how it's supposed to be.

/var/log/lastlog is not a log file like /var/log/syslog, and its name should be read as "last logins list" rather than "last logfile".

It's maintained by the pam_lastlog(8) module, and it's basically an array like this:

struct lastlog {
    time_t  ll_time;    // 4
    char    ll_line[UT_LINESIZE];   // 32
    char    ll_host[UT_HOSTSIZE];   // 256
} entry[UINT_MAX];

Sizes of the fields on a typical x86-64 machine are in comments; an entry should be 4 + 32 + 256 = 292 bytes.

Every time a program using the pam_lastlog(8) pam module is logging a user in, it will seek to uid * sizeof(struct lastlog) and overwrite the entry corresponding to that user.

did I corrupt anything with truncating those files ?

You did corrupt the output of the lastlog(1) command, which nobody is using anyway ;-)

Answer 2

I have seen this behaviour after we have introduced some (large) LDAP UID numbers co-existing with "normal" UNIX UID numbers in a linux (RHEL 7) machine.

Also, in the "CAVEATS" section of the manual page for "lastlog" it states that

  *"Large gaps in UID numbers will cause the lastlog program to run longer with no output to the screen"*

"lastlog" will appear to hang as it processes entries with the intermediate unused UIDs

Maybe this could be also related to the observed problem, as FreeIPA LDAP UID numbers assigned were much larger than the ones from Linux

Answer 3

Those 2 files are "data" type files. If you check using ls -l or ls -lh command then you will see it's occupying very huge amount of disk space, but actually it's not.

ls -s is the command to check the actual size of these files.

[root@LinuxServer ~]# file /var/log/lastlog /var/log/faillog
/var/log/lastlog: data
/var/log/faillog: data
[root@LinuxServer ~]# ls -l /var/log/lastlog /var/log/faillog
-rw------- 1 root osg       3166464 Jul  8 21:20 /var/log/faillog
-rw-r--r-- 1 root root 304854077980 Jul 14 21:38 /var/log/lastlog
[root@LinuxServer ~]# ls -lh /var/log/lastlog /var/log/faillog
-rw------- 1 root osg  3.1M Jul  8 21:20 /var/log/faillog
-rw-r--r-- 1 root root 284G Jul 14 21:38 /var/log/lastlog
[root@LinuxServer ~]#

The actual size is only few KB's, if you check with the ls -s command (output of -s is in disk blocks).

[root@LinuxServer ~]# ls -s /var/log/lastlog /var/log/faillog
3100 /var/log/faillog   748 /var/log/lastlog

Answer 4

By "truncating the file", without informing (or restarting) the programs that are writing to the log file, YOU have caused the sparse file.

The best fix is to find and fix the problem causing all those log messages.

Another way to avoid the problem is Stop the logging service, use lsof to ensure no other processes have the file open, then truncate the file, and restart logging.

As requested, here's a model of how it (doesn't) work:

A program opens the file for "append", is told that the file ends at block X, and writes block X+1. You "truncate the file", but the program "knows" the next block to write is X+2. Thus, the file will might gave data at the beginning, but definitely has data at block X+2. Ta-Da! Sparse file.