1

I have a very large hosts file on my Linux machine:

1.1.1.1  hostA hosta
1.1.1.2  hostB hostb
1.1.1.3  hostC hostc
1.1.1.4  hostD hostd
1.1.1.5  hostE hoste
1.1.1.6  hostF hostf
1.1.1.7  hostG hostg
.......

And I have the lines I want to modify in file2:

2.1.1.1 hostA hosta
2.1.1.2 hostB hostb 
2.1.1.3 hostC hostc 

So, the desired output is:

2.1.1.1  hostA hosta
2.1.1.2  hostB hostb
2.1.1.3  hostC hostc
1.1.1.4  hostD hostd
1.1.1.5  hostE hoste
1.1.1.6  hostF hostf
1.1.1.7  hostG hostg
.......

I know how to use like the one below:

sed '/hostA/ s/1.1.1.1/2.1.1.1/' hosts

But I don't know how to use it in a for loop. Or there is another, faster way to modify hosts?

3
  • 1
    how large is "very large", as in, number of lines according to wc -l /etc/hosts? ("very large" is a very relative term. Few people agree on what it means, and it does make a difference here on how to solve your problem.) Dec 5, 2022 at 14:03
  • 1
    Thanks for your reply Muller. wc -l /etc/hosts 1832 /etc/hosts
    – test good1
    Dec 5, 2022 at 14:11
  • Please don't multi-post: stackoverflow.com/q/74687950/1745001
    – Ed Morton
    Dec 5, 2022 at 22:30

5 Answers 5

2

The following is using Miller (mlr) to read the files as header-less "pretty-printed" data and then perform a left-join operation between the hosts file and the new file, using the 2nd and 3rd fields on each line as the join key.

$ mlr --pprint -N join -j 2,3 -f hosts --ul then reorder -f 1,2,3 new
2.1.1.1 hostA hosta
2.1.1.2 hostB hostb
2.1.1.3 hostC hostc
1.1.1.4 hostD hostd
1.1.1.5 hostE hoste
1.1.1.6 hostF hostf
1.1.1.7 hostG hostg

The reorder operation at the end reorders the fields in the same order as they were found in the input (we would otherwise get the last two fields as fields 1 and 2 for parts of the data).

0

You could do this with 2 GNU sed invocations, e.g.:

sed -E 's:([^ ]+) +(.*):/\2/ s/^[^ ]+/\1/:' file2 |
sed -Ef - hosts

The first sed generates a new sed-script:

/hostA hosta/ s/^[^ ]+/2.1.1.1/
/hostB hostb/ s/^[^ ]+/2.1.1.2/
/hostC hostc/ s/^[^ ]+/2.1.1.3/
0

Using any awk:

$ awk 'NR==FNR{ip[$2]=$1; next} $2 in ip{$1=ip[$2]} 1' file2 hosts
2.1.1.1 hostA hosta
2.1.1.2 hostB hostb
2.1.1.3 hostC hostc
1.1.1.4 hostD hostd
1.1.1.5 hostE hoste
1.1.1.6 hostF hostf
1.1.1.7 hostG hostg

or if you care abut preserving white space between fields:

awk 'NR==FNR{ip[$2]=$1; next} $2 in ip{sub(/[0-9.]+/,ip[$2])} 1' file2 hosts

If you actually need to use both host names from each line as a key rather than just $2 as in your example then just change every $2 to ($2,$3).

0

Doing it the lazy way with a single sed run:

sed -E '1,/^$/{H;d;};G;s/^([0-9.]*)([[:space:]]*)([[:print:]]*)(\n.*\n)([0-9.]*)([[:space:]]*)\3/\5\2\3\4/;P;d' file2 <(echo) /etc/hosts

This is collecting file2 lines in hold space, appending them to each line of the hosts file as lookup table, and using backreferences to replace the IP.

Probably easier to write than to read, as often with regular expressions.

-3

As a variant:

$ cut -d' ' -f2 file2 > hostids
$ grep -v -f hostids hosts > newhosts
$ cat file2 >> newhosts
$ sort -t' ' -k2 newhosts > hosts
1
  • 1
    This will consider hostABC as a match for hostA.
    – terdon
    Dec 5, 2022 at 15:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .