4

I have a file on a Linux system that looks like the following:

May 6 19:12:03 sys-login: user1 172.16.2.102 Login /data/netlogon 13473
May 6 19:15:26 sys-login: user2 172.16.2.107 Login /data/netlogon 14195
May 6 19:28:37 sys-logout: user1 172.16.2.102 Logout /data/netlogon 13473
May 6 19:33:28 sys-logout: user2 172.16.2.107 Logout /data/netlogon 14195
May 8 07:58:50 sys-login: user3 172.16.6.128 Login /data/netlogon 13272
May 8 07:58:50 sys-logout: user3 172.16.6.128 Logout /data/netlogon 13272

And I am trying to calculate the time each user has spent between logging in and logging out in minutes. There will only be one login/logout per user, and I want to generate a report for all users at once.

What I have tried:

I have tried to first extract the users first:

users=$(awk -v RS=" " '/login/{getline;print $0}' data)

which returns the users (logged-in), and then I attempt to extract the time at which they logged in, but I am currently stuck. Any help would be appreciated!

Edit: I am able to get users and dates doing the following:

users=$(grep -o 'user[0-9]' data)
dates=$(grep -o '[0-2][0-9]:[0-5][0-9]:[0-5][0-9]' data)

If I find a complete solution, I will share here.

1
  • What should the output be for those users who have logged in but haven't yet logged out when you run the script? What about a logout for a user with no associated login? – Ed Morton Jun 15 at 14:35
5

Although this site "is not a script-writing service" ;), this is a nice little exercise so I will propose the following awk program. You can save it to a file calc_logtime.awk.

#!/usr/bin/awk -f

/sys-log[^:]+:.*Log/ {
    user=$5
    cmd=sprintf("date -d \"%s %d %s\" \"+%%s\"",$1,$2,$3)
    cmd|getline tst
    close(cmd)

    if ($7=="Login") {
        login[user]=tst
    }
    else if ($7=="Logout") {
        logtime[user]+=(tst-login[user])
        login[user]=0
    }
}

END {
    for (u in logtime) {
    minutes=logtime[u]/60
    printf("%s\t%.1f min\n",u,minutes)
    }
}

This relies on using the GNU date command (part of the standard tools suite on GNU/Linux systems) and on the time format in your log file being as specified. Also note that this doesn't contain many safety checks, but you should get the idea on how to modify it to your needs.

  • It will look for lines that contain both the string sys-log near the beginning and Log near the end to increase selectivity just in case there may be other content. As stated, this is a very rudimentary test, but again, you can get the idea of how to make it more specific.
  • The user will be extracted as the 5th space-separated field of the line.
  • The action will be extracted as the 7th space-separated field of the line.
  • The timestamp of the action will be converted to "seconds since the epoch" by generating a date call via sprintf and delegating the task to the shell.
  • If the action is Login, the timestamp is stored in an array login, with the username as "array index".
  • If the action is Logout, the duration will be calculated and added to an array logtime containing the total log-time for all users so far.
  • At end-of-file, a report will be generated, by iterating over all "array indices" of logtime and converting the logtimes from seconds to minutes by simple division.

You can call it via

awk -f calc_logtime.awk logfile.dat
0
5

With GNU awk for time functions and gensub() and arrays of arrays:

$ cat tst.awk
BEGIN {
    dateFmt = strftime("%Y") " %02d %02d %s"
    months  = "JanFebMarAprMayJunJulAugSepOctNovDec"
}
{
    date = sprintf(dateFmt, (index(months,$1)+2)/3, $2, gensub(/:/," ","g",$3))
    userSecs[$5][$7] = mktime(date)
}
$7 == "Logout" {
    printf "%s %0.2f\n", $5, (userSecs[$5]["Logout"] - userSecs[$5]["Login"]) / 60
    delete userSecs[$5]
}

$ awk -f tst.awk file
user1 16.57
user2 18.03
user3 0.00

That will run orders of magnitude faster than calling Unix date from awk as the latter has to spawn a subshell every time to do so.

If you want to also get a report of users who have logged in but haven't logged out when you run the script, e.g. user4 in this modified input file:

$ cat file
May 6 19:12:03 sys-login: user1 172.16.2.102 Login /data/netlogon 13473
May 6 19:15:26 sys-login: user2 172.16.2.107 Login /data/netlogon 14195
May 6 19:28:37 sys-logout: user1 172.16.2.102 Logout /data/netlogon 13473
May 6 19:33:28 sys-logout: user2 172.16.2.107 Logout /data/netlogon 14195
May 8 07:58:50 sys-login: user3 172.16.6.128 Login /data/netlogon 13272
May 8 07:58:50 sys-logout: user3 172.16.6.128 Logout /data/netlogon 13272
Jun 15 08:30:26 sys-login: user4 172.16.2.107 Login /data/netlogon 14195

Then just tweak the script:

$ cat tst.awk
BEGIN {
    dateFmt = strftime("%Y") " %02d %02d %s"
    months  = "JanFebMarAprMayJunJulAugSepOctNovDec"
}
{
    date = sprintf(dateFmt, (index(months,$1)+2)/3, $2, gensub(/:/," ","g",$3))
    userSecs[$5][$7] = mktime(date)
}
$7 == "Logout" {
    printf "%s %0.2f %s\n", $5, (userSecs[$5]["Logout"] - userSecs[$5]["Login"]) / 60, "Complete"
    delete userSecs[$5]
}
END {
    now = systime()
    for (user in userSecs) {
        printf "%s %0.2f %s\n", user, (now - userSecs[user]["Login"]) / 60, "Partial"
    }
}

$ awk -f tst.awk file
user1 16.57 Complete
user2 18.03 Complete
user3 0.00 Complete
user4 51.10 Partial

If you needed to find cases where a user has logged in again while already logged in with no logout in between or handle logouts with no associated login differently or do anything else then that'd all just be trivial tweaks too.

0
5

The following perl script uses the Date::Parse module from the TimeDate collection to parse the dates & times from each record instead of relying on GNU date to do it. This is probably packaged for your distro (on debian, apt install libtimedate-perl), otherwise install it with cpan.

The script works by using the last field of each input line (which appears to be a session ID) as the top-level key to a Hash-of-Hashes (HoH) data structure called %sessions. Each element of %sessions is an anonymous hash containing the keys user, login, and logout.

Once the entire file has been read in and parsed, the cumulative totals for each user are calculated (and stored in another associative array, %users), and then printed. Output is sorted by username.

#!/usr/bin/perl -l

use strict;
use Date::Parse;

my %sessions;
my %users;

# read the input file, parse dates, store login and logout times into session hash
while (<>) {
  next unless (m/\ssys-log(?:in|out):\s/);

  my ($M, $D, $T, $type, $user, $ip, undef, undef, $s) = split;
  $type =~ s/^sys-|://g;

  $sessions{$s}->{user} = $user;
  $sessions{$s}->{$type} = str2time(join(" ", $M, $D, $T));
  # $session{$s}->{IP} = $ip; # not used
};

# add up session totals for each user
foreach my $s (keys %sessions) {
  # ignore sessions without both a login and logout time, it's
  # impossible to calculate session length.
  next unless ( defined($sessions{$s}->{login}) &&
                defined($sessions{$s}->{logout}) );

  $users{$sessions{$s}->{user}} += $sessions{$s}->{logout} - $sessions{$s}->{login};
};

# print them
foreach my $u (sort keys %users) {
   printf "%s has logged in for %s minutes\n", $u, int($users{$u}/60); 
};

Save it as, e.g., login-times.pl and make it executable with chmod +x login-times.pl. Run it like:

$ ./login-times.pl data
user1 has logged in for 16 minutes
user2 has logged in for 18 minutes
user3 has logged in for 0 minutes

FYI, the data in the %sessions HoH looks like this:

%sessions = {
  13272 => { login => 1620424730, logout => 1620424730, user => "user3" },
  13473 => { login => 1620292323, logout => 1620293317, user => "user1" },
  14195 => { login => 1620292526, logout => 1620293608, user => "user2" },
}

It is entirely possible for a session to not have either a login or a logout timestamp. It would be easy to print a message to STDERR if either were missing. Or to handle such anomalies however you choose. The script above just ignores them.

For completeness, the data in %users ends up looking like this:

%users = { user1 => 994, user2 => 1082, user3 => 0 }

BTW, these data structures were printed with the Data::Dump module, which is very useful for debugging etc. Debian package name is libdata-dump-perl, other distros probably have it. Otherwise, install it with cpan.

To print these, I added the following at the end of the script:

use Data::Dump qw(dump);
print "%sessions = ", dump(\%sessions);
print "%users = ", dump(\%users)

Finally, the IP address is captured with the split function in the script but not used. This could easily be added to the session hash and used to print a one-line summary of each login & logout pair. The Date::Format module from the same Time::Date collection could be used to format the dates.

For example:

  1. add use Date::Format; after the use Date::Parse; line

  2. Uncomment $session{$s}->{IP} = $ip; in the while(<>) loop.

  3. Use something like the following to print out the data:

my $tfmt = "%Y-%m-%d %H:%M:%S";

printf "%s\t%-20s\t%-20s\t%7s\t%s\n", "USER", "LOGIN", "LOGOUT", "MINUTES", "IP";

# sort the session keys by their 'user' fields.
foreach my $s (sort { $sessions{$a}->{user} cmp $sessions{$b}->{user} } keys %sessions) {
  my $in  = $sessions{$s}->{login};
  my $out = $sessions{$s}->{logout};
  next unless ($in && $out);

  my $user = $sessions{$s}->{user};
  my $ip   = $sessions{$s}->{IP};

  my $minutes = int(($out-$in)/60);
  $in  = time2str($tfmt,$in); 
  $out = time2str($tfmt,$out);

  printf "%s\t%-20s\t%-20s\t%7i\t%s\n", $user, $in, $out, $minutes, $ip;
};

Output would be like:

USER    LOGIN                   LOGOUT                  MINUTES IP
user1   2021-05-06 19:12:03     2021-05-06 19:28:37          16 172.16.2.102
user2   2021-05-06 19:15:26     2021-05-06 19:33:28          18 172.16.2.107
user3   2021-05-08 07:58:50     2021-05-08 07:58:50           0 172.16.6.128
1

This sounds like a job for dateutils. Fish out the relevant pieces with awk:

awk -v OFS='\t' '
$4 == "sys-login:"  { login[$5]  = $1" "$2" "$3 }
$4 == "sys-logout:" { logout[$5] = $1" "$2" "$3 }
END {
  for (user in login)
    print user, login[user], logout[user]
}' infile

Output:

user1   May 6 19:12:03  May 6 19:28:37
user2   May 6 19:15:26  May 6 19:33:28
user3   May 8 07:58:50  May 8 07:58:50

And pipe it to a while-loop:

while IFS=$'\t' read username starttime endtime; do
  printf "%s\t%s\n" $username \
    $(dateutils.ddiff -i "%b %d %H:%M:%S" -f "%S" "$starttime" "$endtime")
done

Output:

user1   994
user2   1082
user3   0

Note: you can change the ddiff command's -f switch to choose a different time format. Here we're using seconds-elapsed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.