0

Got some valuable help here earlier with grep so hopefully I can get this sorted out too.

This is from Rclone log

Transferred:      577.080M / 577.080 MBytes, 100%, 12.660 MBytes/s, ETA 0s
Errors:                 0
Checks:                 2 / 2, 100%
Transferred:            2 / 2, 100%
Elapsed time:       45.5s

What I am trying to do is create a email notification with custom text. Something like "Transferred 577 MBytes, 2 files with 0 errors in 45.5 seconds @ 12,660 MBytes/s"

So for this to work I need to print the values. I tried the same way I did before without any luck. Transferred it two times in the log, how to split them to get TRF= 577.080Mbytes and TRS= 12.660 MBytes/s

TRF=$(grep -o 'Transferred:.*' $logfile| cut -d\   -f4)
ERR=$(grep -o 'Errors:.*' $logfile | cut -d\   -f4)
TIM=$(grep -o 'Elapsed time:.*' $logfile | cut -d\   -f3-)
TRS=$(grep -o 'Transferred:.*' $logfile | cut -d\   -f4)
2
  • Do you really have two lines that both start with Transferred: but give different information in different formats?
    – John1024
    Jul 23, 2019 at 1:18
  • Yeah, unfortunately Jul 23, 2019 at 7:56

2 Answers 2

0

TRF:

$ grep -Pom1 'Transferred:.*/ \K[^,]*' "$logfile"
577.080 MBytes
  • -P enable perl-compatible regular expressions and \K to not include the things before \K in the matched output
  • -o only output matched parts of the line
  • -m1 quit after the first matching line to exclude the second Transferred: line

TRS:

$ grep -Po 'Transferred:[^,]*,[^,]*, \K[^,]*' "$logfile"
12.660 MBytes/s
  • similar to TRF: get everything following the second comma + space which is not a comma
  • we don't need -m1 here as the pattern contains more than one comma

ERR:

$ grep -Po 'Errors:[[:space:]]*\K.*' "$logfile"
0
  • only get the digits at the end of the line, strip space characters

TIM:

$ grep -Po 'Elapsed time:[[:space:]]*\K.*' "$logfile"
45.5s

Number of transferred files: (it's the first of the two numbers)

$ grep -Po 'Transferred:[[:space:]]*\K[0-9]+(?= /)' "$logfile"
2
  • (?= /) is a positive lookahead (PCRE) where we're looking for a space and /. This part is like \K not included in the matched output and needed to not match the first Transferred line.
1
  • PERFECT! Thank you so much, just what I needed Jul 23, 2019 at 9:59
0

This approach creates the whole output line with only a single process. Consequently, it should be much faster than using a series of grep and cut. Also, awk is quite powerful (it does floating point math, for example) which might be useful if you later do enhancements to the output.

Try:

$ awk '/ETA/{mb=$2; speed=$7" "$8} /Errors/{err=$2} /Transferred/{n=$2} /Elapsed/{printf "Transferred %s, %s files with %s errors in %s @ %s\n",mb,n,err,$3,speed}' rclone.log 
Transferred 577.080M, 2 files with 0 errors in 45.5s @ 12.660 MBytes/s,

How it works

  • /ETA/{mb=$2; speed=$7" "$8}

    If the current line contains ETA (meaning that it is the first Transferred line, then save the second field in mb and the seventh and eighth fields in speed.

  • /Errors/{err=$2}

    If this line contains Errors, then save the second field in err.

  • /Transferred/{n=$2}

    If this line contains Transferred, save the second field in variable n.

  • /Elapsed/{printf "Transferred %s, %s files with %s errors in %s @ %s\n",mb,n,err,$3,speed}'

    Finally, when we reach a line containing Elapsed, print out our message.

Multiline Version

For those who prefer their code spread over multiple lines:

awk '
/ETA/{
    mb=$2
    speed=$7" "$8
}
/Errors/{
    err=$2
}
/Transferred/{
    n=$2
}
/Elapsed/{
    printf "Transferred %s, %s files with %s errors in %s @ %s\n",mb,n,err,$3,speed
}' rclone.log 
1
  • Thank you, will test this in a script I am trying to create :) Jul 24, 2019 at 18:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .