# Cat 2 files and avoid duplication of lines

I have the file1 with the following content

{"name":"clio5", "value":"13"}
{"name":"citroen_c4", "value":"23"}
{"name":"citroen_c3", "value":"12"}
{"name":"golf4", "value":"16"}
{"name":"golf3", "value":"8"}


And I have the file2 with the following content

{"name":"clio5", "value":"14"}
{"name":"citroen_c4", "value":"25"}
{"name":"golf4", "value":"18"}


I want to execute a shell command in order to display the content of the file1 and the file2. if a name exist in both file1 and file2 so I want to display only the related line of the file2.

So the output should look like this:

$command taking account file1 file2 {"name":"clio5", "value":"14"} {"name":"citroen_c4", "value":"25"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"18"} {"name":"golf3", "value":"8"}  The command should not edit file1 neither file2 The command should be as simple as possible The command could contains grep, sed, awk • Is this JSON? (Your examples are quite close, but not exact; the commas separating the value sets are missing.) If you only wanted to avoid printing duplicate lines, uniq would do fine, but this appears to be more than that. – a CVn Jul 22 '13 at 11:19 • @MichaelKjörling And if it is json, there would be much more useful tools to operate on the date probably. – Bernhard Jul 22 '13 at 13:04 ## 3 Answers This does not preserve the order of the lines, but that was not a requirement awk '{line[$1]=$0} END {for (key in line) print line[key]}' file1 file2  If order is not important, these commands should do it: cat file2 file1 > files cut -f2 -d: files | cut -f1 -d, | sort | uniq > pat for i in$(cat pat); do grep $i files -m 1; done  The first command joins the input files (order is important; file2 goes first because its values will override the ones in file1). The second command cuts the car brands from the input files, sorts and removes duplicates, and stores them in the pat file. The third command greps each pattern on the combined input file and stops on the first match, so if the same car brand appears on both files, only the one in file2 is printed. If order is important and all the patterns are in file1, you can use just file1 in the second command (instead of files) and omit the sort and uniq commands. • Instead of making a temporary file, you can pipe to a loop: ...sort | uniq | while read i ; do.... Also, you can pipe the output of cat to cut: cat file2 file1 | cut -f2 -d: | .... – user26112 Jul 22 '13 at 13:32 • @EvanTeitelman, Thanks. I imagined there would be a more compact way, but I feared it would be less readable. – Paulo Almeida Jul 22 '13 at 13:49 $ awk -F, 'NR==FNR{a[$1]=$0;next};$1 in a {$0=a[$1]}; 1' file2 file1 {"name":"clio5", "value":"14"} {"name":"citroen_c4", "value":"25"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"18"} {"name":"golf3", "value":"8"}  That assumes that file2 only updates entries that are already found in file1. If file2 may insert new entries, you may want to change it to: awk -F, ' NR==FNR{a[$1]=$0;next}$1 in a {$0=a[$1];delete a[\$1]}
1
END{for (i in a) print a[i]}' file2 file1


The order in which the inserted entries are displayed is unspecified. The above assumes there's only one entry per name in each file.