0

I have a file that looks like this

File 1

   3 123456789
   3 00000     
   2 123456789
   2 abcde
   1    
   4 abcdefgh

The lines in the first column correlate with the line number of another file, which looks something like this:

File 2

$a&a(md
( l 0 p a$1
=2 3 x5 x4
&a”s?m!a

I would like some way to fuse the files, replacing the numbers in the first column of the first file with the content of the corresponding line of the second file. So the final file would look something like this:

Output

=2 3 x5 x4    123456789
=2 3 x5 x4    00000  
( l 0 p a$1   123456789
( l 0 p a$1   abcde
$a&a(md          
&a”s?m!a      abcdefgh

The formatting doesn't matter much, as long as I can tell which column is which.

I'm using Linux.

Any help is greatly appreciated.

1

You could use awk, constructing an array from file2 indexed by its line (record) number, and then doing a lookup based on the first field of file1

$ awk 'NR==FNR {a[FNR]=$0;next} {printf "%s\t%s\n", a[$1], $2}' file2 file1
=2 3 x5 x4  123456789
=2 3 x5 x4  00000
( l 0 p a$1 123456789
( l 0 p a$1 abcde
$a&a(md 
&a”s?m!a    abcdefgh

This tab-separates the two parts: you could comma-separate them instead if you prefer.

  • Great, this works! Just one thing - the output in the terminal looks the way it does in your answer. When redirecting the output into a new text file, it looks as if the second column is in the next row. It actually doesn't matter to me, but can you recommend how it can best be formatted if I want to keep working on the file with awk? – user146854 Dec 17 '15 at 13:50
  • @user146854 what text editor are you opening the new file with? Perhaps its tab stops or line wrapping options are set funny? Really there's no "best formatting" I can recommend without knowing what you want to do with the result. – steeldriver Dec 17 '15 at 14:12
1

You can use the join command for that. It merges two files based on a common field. Here is an example using bash (I broke the lines for readability):

join -t , \
    <(awk '{ print NR","$0; }' file2.txt | sort -k 1b,1 -t ,) \
    <(awk '{ print $1","$2; }' file1.txt | sort -k 1b,1 -t ,) \
    | cut -d , -f 2- \
    | column -t -s ,

Output:

( l 0 p a$1  123456789
( l 0 p a$1  abcde
=2 3 x5 x4   00000
=2 3 x5 x4   123456789
&a”s?m!a     abcdefgh

Some explanations:

  • To simplify the way fields and columns are handled, the first step is to normalize the separator character. Here, , is used for that.
  • awk is used to normalize the file1.txt separator.
  • Since file2.txt doesn't have an explicit field to join the lines, awk is used to add the line number to each line.
  • By default, join uses the first field of each file to join the files. So there's no need to specify which field should be used. Note that -t is given indicating that fields are separated by ,.
  • join expects that both input files are sorted, so sort is used to ensure that the default ordering expected by join is used.
  • <( some_command ) creates anonymous named pipes. It basically allows us to use command outputs as files.
  • cut is used to remove the line number field from the output.
  • And last, column is used to format the output as a table (-t). To avoid messing with the data from file2.txt, , is used as separator (-s ,).
  • I'll test this as soon as I can, just one quick question I just thought of - if the output format is like this, will further processing steps be able to easily recognize the two different columns (since there are spaces in some parts of the first column)? – user146854 Dec 17 '15 at 12:47
  • I updated my answer with that. – Marcelo Cerri Dec 17 '15 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.