3

I have a log file which contain lots of line like below:

2017-07-16 01:06:07 | 8801624874139 | http://192.168.5.1:2020/credit/purchase/4 | XpressLoan | {"resultCode":0,"resultMessage":"OK","amount":100000,"serviceFee":24400,"totalOutstandingdebt":124400,"msisdn":8801624874139}  
2017-07-16 01:06:24 | 8801628666938 | http://192.168.5.1:2020/credit/purchase/5 | XpressLoan | {"resultCode":0,"resultMessage":"OK","amount":50000,"serviceFee":12180,"totalOutstandingdebt":62180,"msisdn":8801628666938}

From the above log how can I get the amount value?

Expected output:

100000
50000
3
  • Pipe this to sed -n 's/".*amount":\(.*\),".*/\1/p'
    – eins6180
    Jul 15, 2017 at 19:16
  • 1
    That looks suspiciously like json in that field; consider a json parser.
    – Jeff Schaller
    Jul 15, 2017 at 19:21
  • @eins6180 On my machine this doesn't work. However I have a similar approach. In your approach you refer to a regex group that matches everything instead of numbers.
    – John Goofy
    Jul 15, 2017 at 20:10

3 Answers 3

4

Use combination of awk + jq (JSON manipulation tool):

awk '{ print $10 }' logfile | jq -r '.amount'

The output:

100000
50000

With such kind of approach you'll be able to extract any/multiple keys/values from your JSON-encoded field

1
  • 1
    That's pretty neat, I wasn't aware of jq, cheers.
    – fduff
    Jul 15, 2017 at 20:27
1

Try this:

$ awk -F\" '/amount/ {print $9}' file | sed 's/[:|,]//g'
100000
50000

OR:

$ sed 's/^.*amount\":\([0-9]*\),\".*$/\1/' file
100000
50000

EDIT

You can summarize the output of the first command if you pipe it again to awk like this example:

$ awk -F\" '/amount/ {print $9}' file | sed 's/[:|,]//g' |\
awk '{sum += $1} END {print sum}'
150000
5
  • (1) The first approach is very sensitive to possible future changes in the file format.  If some record is missing “resultCode” or “resultMessage”, or has some other field before “amount”, it will break.  (2) The second approach is somewhat sensitive to possible future changes in the file format.  If “amount” is ever the last field (so the amount value is followed by } instead of ,), it will break.  But I’m nit-picking — if the file could vary that much, it would be the OP’s responsibility to tell us.  (3) You don’t need to escape the quotes.  … (Cont’d) Jul 15, 2017 at 21:33
  • (Cont’d) …  (4) If the “amount” value is ever anything other than a sequence of digits — such as “100tons”, “−100”, or even “50.0” — your second answer will fail.  It’s probably safer to say \([^,]*\).  (5) On any lines that don’t satisfy your substitution command (including those mentioned in paragraph #4), your second command will output the entire line.  The question doesn’t specify what to do in such a case; it might be better to say sed -n and then put a p at the end of the s command: sed -n 's/…/…/p'. Jul 15, 2017 at 21:33
  • @Scott To both comments: I wasn't aware that JSon output is so volatile.
    – John Goofy
    Jul 15, 2017 at 23:04
  • @JohnGoofy Can I get the summation from those returned values in a single command? I used your first command.
    – Rezuan
    Jul 17, 2017 at 11:37
  • 1
    @Rezuan Yes, please review my edit.
    – John Goofy
    Jul 18, 2017 at 11:04
1

Could use multiple awk to get the needed data.

awk -F"|" {'print $5'}| awk -F"," '{print $3}'| awk -F":" '{print $2}'
  • the first awk will get you to the {} data
  • the second awk will give you the key:value
  • the third awk will give you the value of the key

even if format will change then the patter should be identified by the separators and adjust it accordingly.

e.g.:

my_var="2017-07-16 01:06:07 | 8801624874139 | http://192.168.5.1:2020/credit/purchase/4 | XpressLoan | {"resultCode":0,"resultMessage":"OK","amount":100000,"serviceFee":24400,"totalOutstandingdebt":124400,"msisdn":8801624874139}"

$ echo $my_var | awk -F"|" {'print $5'}
 {resultCode:0,resultMessage:OK,amount:100000,serviceFee:24400,totalOutstandingdebt:124400,msisdn:8801624874139}

$ echo $my_var | awk -F"|" {'print $5'}| awk -F"," '{print $4}'
serviceFee:24400

$ echo $my_var | awk -F"|" {'print $5'}| awk -F"," '{print $3}'| awk -F":" '{print $2}'
100000

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .