3

How to sort a file containing below? (s=second, h=hour, d=day m=minute)

1s
2s
1h
2h
1m
2m
2s
1d
1m
  • 3
    Are there always only whole numbers, nothing like 1h30m40s or 1.30h? – jimmij Oct 15 '17 at 10:02
6
awk '{ unitvalue=$1; }; 
    /s/ { m=1 }; /m/ { m=60 }; /h/ { m=3600 }; /d/ { m=86400 }; 
    { sub("[smhd]","",unitvalue); unitvalue=unitvalue*m; 
    print unitvalue " " $1; }' input |
        sort -n | awk '{ print $2 }'
1s
2s
2s
1m
1m
2m
1h
2h
1d
4

First version - FPAT is used

gawk '
BEGIN {
    FPAT="[0-9]+|[smhd]";
}
/s/ { factor = 1 }
/m/ { factor = 60 }
/h/ { factor = 3600 }
/d/ { factor = 86400 }
{
    print $1 * factor, $0;
}' input.txt | sort -n | awk '{print $2}'

FPAT - A regular expression describing the contents of the fields in a record. When set, gawk parses the input into fields, where the fields match the regular expression, instead of using the value of the FS variable as the field separator.

Second version

I was surprised to discover, that without FPAT it also works. It is caused the number conversion mechanism of awk - How awk Converts Between Strings and Numbers, namely:

A string is converted to a number by interpreting any numeric prefix of the string as numerals: "2.5" converts to 2.5, "1e3" converts to 1,000, and "25fix" has a numeric value of 25. Strings that can’t be interpreted as valid numbers convert to zero.

gawk '
/s/ { factor = 1 }
/m/ { factor = 60 }
/h/ { factor = 3600 }
/d/ { factor = 86400 }
{
    print $0 * factor, $0;
}' input.txt | sort -n | awk '{print $2}'

Input (changed a little bit)

1s
122s
1h
2h
1m
2m
2s
1d
1m

Output

Note: 122 seconds more than 2 minutes, so it sorted after 2m.

1s
2s
1m
1m
2m
122s
1h
2h
1d
  • 1
    +1 I like the clever use of FPAT. This could easily be expanded to accept and handle time values like 1d3h10m40s. – David Foerster Oct 15 '17 at 16:44
  • @DavidFoerster I looked to your awk answer and discovered interesting fact: strings like 1s, 3d, 4m converting to the integer by awk itself, without problems. So, they can be used for math operations directly - without splitting by regex. I was added second version of the solution and an explanation of this behaviour too. – MiniMax Oct 15 '17 at 19:42
3

This an extension of MiniMax’ answer that can handle a broader range of duration value like 1d3h10m40s.

GNU Awk program (stored in parse-times.awk for the sake of this answer):

#!/usr/bin/gawk -f
BEGIN{
  FPAT = "[0-9]+[dhms]";
  duration["s"] = 1;
  duration["m"] = 60;
  duration["h"] = duration["m"] * 60;
  duration["d"] = duration["h"] * 24;
}

{
  t=0;
  for (i=1; i<=NF; i++)
    t += $i * duration[substr($i, length($i))];
  print(t, $0);
}

Invocation:

gawk -f parse-times.awk input.txt | sort -n -k 1,1 | cut -d ' ' -f 2
2

If you only have times in the format of your question:

sort -k 1.2,1.2 -k 1.1,1.1 <file>

Where <file> is the file your data resides in. This command sorts on the second letter (ascending) and then sorts on the first letter (ascending). This works because it just so happes that the ordering of the letters for the time units (d > h > m > s) is exactly the order we want (day > hours > minutes > seconds).

  • 1
    Just like the other (now deleted) answer, this assumes durations are single-digit... – don_crissti Oct 15 '17 at 10:59
  • @don_crissti I answered the question because in the worst case I can just delete and in the best case this is exactly what he was looking for. I thought this was a better approach than waiting for an edit of the question (which potentially takes a long time, so by then the question might be lost). – PawkyPenguin Oct 15 '17 at 11:32
2

Solution in Python 3:

#!/usr/bin/python3
import re, fileinput

class RegexMatchIterator:
    def __init__(self, regex, string, error_on_incomplete=False):
        self.regex = regex
        self.string = string
        self.error_on_incomplete = error_on_incomplete
        self.pos = 0

    def __iter__(self):
        return self

    def __next__(self):
        match = self.regex.match(self.string, self.pos)
        if match is not None:
            if match.end() > self.pos:
                self.pos = match.end()
                return match
            else:
                fmt = '{0!s} returns an empty match at position {1:d} for "{3!r}"'

        elif self.error_on_incomplete and self.pos < len(self.string):
            if isinstance(self.error_on_incomplete, str):
                fmt = self.error_on_incomplete
            else:
                fmt = '{0!s} didn\'t match the suffix {3!r} at position {1:d} of {2!r}'

        else:
            raise StopIteration(self.pos)

        raise ValueError(fmt.format(
            self.regex, self.pos, self.string, self.string[self.pos:]))


DURATION_SUFFIXES = { 's': 1, 'm': 60, 'h': 3600, 'd': 24*3600 }
DURATION_PATTERN = re.compile(
    '(\\d+)(' + '|'.join(map(re.escape, DURATION_SUFFIXES.keys())) + ')')

def parse_duration(s):
    return sum(
        int(m.group(1)) * DURATION_SUFFIXES[m.group(2)]
        for m in RegexMatchIterator(DURATION_PATTERN, s,
            'Illegal duration string {3!r} at position {1:d}'))


if __name__ == '__main__':
    with fileinput.input() as f:
        result = sorted((l.rstrip('\n') for l in f), key=parse_duration)
    for item in result:
        print(item)

As you can see I spent about ⅔ of the line count towards a useful iterator over regex.match() results because regex.finditer() doesn't tie matches to the beginning of the current region and there are no other suitable ways to iterate over match results. *grrr*

0

Building off of @hauke-laging's fine answer I wanted to expand some additional details to scale up his answer for use with a more broad set of problems. His solution assumes the input would be a single column. If you have a random field that you'd like to perform the same operation on you can do the following.

Sample data

Say I have the following output that was generated by this command:

$ oc get po -o wide --all-namespaces | grep ContainerCreat
ns8 pod8   0/1 ContainerCreating 0 2m  <none> ocp-app-01b.lab1.somedom.local      <none>
ns9 pod9   0/1 ContainerCreating 0 2m  <none> ocp-app-01b.lab1.somedom.local      <none>
ns1 pod1   0/1 ContainerCreating 0 45s <none> ocp-app-01i.lab1.somedom.local      <none>
ns2 pod2   0/1 ContainerCreating 0 58s <none> ocp-app-01i.lab1.somedom.local      <none>
ns3 pod3   0/1 ContainerCreating 0 58s <none> ocp-app-01i.lab1.somedom.local      <none>
ns5 pod5   0/1 ContainerCreating 0 1m  <none> ocp-app-01i.lab1.somedom.local      <none>
ns6 pod6   0/1 ContainerCreating 0 2m  <none> ocp-app-01g.lab1.somedom.local      <none>
ns4 pod4   0/1 ContainerCreating 0 1m  <none> ocp-app-01i.lab1.somedom.local      <none>
ns7 pod7   0/1 ContainerCreating 0 2m  <none> ocp-app-01b.lab1.somedom.local      <none>

You could use the following to select the 6th field, $6, and then perform the operation that was shown in the other answer. The method is simple yet effective.

You're basically taking the column of interest, $6 in this example, and performing a calculation on it using the appropriate conversion factor based on the units present.

Once converted, the calculation is added as part of the output, used to sort the output correctly based on that column's expanded form value, and then deleted in the final output of column data.

$ oc get po -o wide --all-namespaces | \
    grep ContainerCreat | \
    awk '{ unitvalue=$6; }; \
            $6 ~ /[0-9]+s/ { m=1 }; \
            $6 ~ /[0-9]+m/ { m=60 }; \
            $6 ~ /[0-9]+h/ { m=3600 }; \
            $6 ~ /[0-9]+d/ { m=86400 }; \
            { \
                sub("[smhd]","",unitvalue); \
                unitvalue=unitvalue*m; \
                print unitvalue, $1, $4, $6, $8; \
            }' | \
    sort -rn -k1,1 | \
    awk '{print $2,$3,$4,$5}' | column -t

Example

$ oc get po -o wide --all-namespaces | \
        grep ContainerCreat | \
        awk '{ unitvalue=$6; }; \
                $6 ~ /[0-9]+s/ { m=1 }; \
                $6 ~ /[0-9]+m/ { m=60 }; \
                $6 ~ /[0-9]+h/ { m=3600 }; \
                $6 ~ /[0-9]+d/ { m=86400 }; \
                { \
                    sub("[smhd]","",unitvalue); \
                    unitvalue=unitvalue*m; \
                    print unitvalue, $1, $4, $6, $8; \
                }' | \
        sort -rn -k1,1 | \
        awk '{print $2,$3,$4,$5}' | column -t

 ns1 ContainerCreating 45s ocp-app-01i.lab1.somedom.local
 ns2 ContainerCreating 58s ocp-app-01i.lab1.somedom.local
 ns3 ContainerCreating 58s ocp-app-01i.lab1.somedom.local
 ns4 ContainerCreating 1m  ocp-app-01i.lab1.somedom.local
 ns5 ContainerCreating 1m  ocp-app-01i.lab1.somedom.local
 ns6 ContainerCreating 2m  ocp-app-01g.lab1.somedom.local
 ns7 ContainerCreating 2m  ocp-app-01b.lab1.somedom.local
 ns8 ContainerCreating 2m  ocp-app-01b.lab1.somedom.local
 ns9 ContainerCreating 2m  ocp-app-01b.lab1.somedom.local

Notice in the final output that the data is sorted by the original $6 column in numeric order based on time.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.