3

I would like to compare the two files, and only print this line and append it to "source.txt":

01.02.70 08h00,4.4.4.4,443

Here are my files:

source.txt

DATETIME,IPSOURCE,PORT
01.01.70 08h00,0.0.0.0,443
01.01.70 08h00,2.2.2.2,443

events.txt

DATETIME,IPSOURCE,PORT
01.02.70 09h00,0.0.0.0,443
01.02.70 09h00,2.2.2.2,443
01.02.70 08h00,4.4.4.4,443

I don't care about the DATETIME field in the comparison, I only want to add the new IP that appears in the log file "events.txt" to "source.txt" (addresses 0.0.0.0 and 2.2.2.2 appear in the two files).

So I want to use

grep -vxFf source.txt events.txt

without taking the first field into account, I look only for differences in the IPSOURCE field (second column).

1

Use awk when you need field based comparison

$ awk -F, 'NR==FNR{ip[$2]=$0; next} $2 in ip{delete ip[$2]} END{for(k in ip) print ip[k]}' events.txt source.txt
01.02.70 08h00,4.4.4.4,443
  • -F, set , as input field separator
  • NR==FNR{ip[$2]=$0; next} use second field as key and save entire line in associative array. This block of code will execute only for first file input (i.e events.txt)
  • $2 in ip{delete ip[$2]} when second file (i.e source.txt) is processing, delete elements in ip array if second field was already present
  • END{for(k in ip) print ip[k]} after all input is processed, print lines remaining in ip array


Or, change the order of input files and print lines from events.txt if second field was not present in source.txt

$ awk -F, 'NR==FNR{ip[$2]; next} !($2 in ip)' source.txt events.txt
01.02.70 08h00,4.4.4.4,443
1

Simply do:

awk -F, 'NR==FNR{arr[$2]=$0;next} !($2 in arr)' source event #>> source

If you want have appended difference line based on second column with comma separated fields, just Uncomment >> source in above command.

NO, if you want to compare based on IP:Port part. then you can use as following.

awk -F, 'NR==FNR{arr[$2FS$3]=$0;next} !($2FS$3 in arr)' source event #>> source
1

According to your conditions:

  • append it to "source.txt"

  • I only want to add the new IP that appears in the log file "events.txt" to "source.txt"


Here's a complete solution based on combination of GNU sed, cut and grep:

sed -i "\$ a $(cut -d, -f2 source.txt | grep -vf - events.txt)" source.txt

The resulting source.txt file contents:

DATETIME,IPSOURCE,PORT 
01.01.70 08h00,0.0.0.0,443 
01.01.70 08h00,2.2.2.2,443
01.02.70 08h00,4.4.4.4,443

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.