4

I have several text files, and each file's content is like

attribute1,attribute2,...
1,2,...

The first lines which describe the names of the columns are the same in all the files.

I want to concatenate the second lines of the files, add a new column with filename for each line for which file it comes from, and put one of the first lines as the first line in the new file. For example, the desired new file looks like:

file,attribute1,attribute2,...
file1,1,2,...
file2,11,12,...
file3,21,22,...

How should I do that?

2
  • So each file has just two lines ? – don_crissti Nov 13 '15 at 19:39
  • yes. @don_crissti – Tim Nov 13 '15 at 19:40
4

To add the file name, with awk:

awk -v OFS=, 'NR==1{print "file", $0};FNR==2{print FILENAME, $0}' file1 file2 ... fileN >outfile

which prints the updated header if it's the first line of input (NR==1) or (||) if it's the second line of each file(FNR==2) it prints the FILENAME and the line ($0).


Initial answer before your edit.
You could do that with sed:

sed -n '1p;n;p' file1 file2 ... fileN >outfile

the -n suppresses printing, 1p prints the first line and then for each (odd) line it empties the pattern space and reads in the next line (the even lines that is) and prints it.

9
  • thanks. how do you do the same with awk? – Tim Nov 13 '15 at 19:48
  • I forgot that I would like to add a new column, which consists of the filename for each line. See my update. – Tim Nov 13 '15 at 19:56
  • I realized that after I commented. this is a bit more clear: 'NR==1 || FNR==2' – glenn jackman Nov 13 '15 at 19:56
  • @tim, update your question with the new requirements. Don't bury them in comments. – glenn jackman Nov 13 '15 at 19:57
  • @glenn: exactly. – Tim Nov 13 '15 at 20:01
2
{   printf file,; head -n1 <file
    grep -n '' ./file* /dev/null|
    sed -ne'/^[^:]*:2:/s/:.:/,/p'
}   >./new_file

grep will get your filenames at the head of each line, and can assist in selecting by line -number as well. this will probably not work very well, though, if you've got colons in your filenames.

2
  • 1
    Mike, I think you could ditch the /dev/null in this case as we're talking about at least two files. Also, I think this post could be updated (with zgrep) per the latest OP requirements (see comments under my post) and would in fact be much better for what OP actually wants to do as it would work with gz files. – don_crissti Nov 13 '15 at 20:54
  • 1
    @don_crissti - but i like the /dev/null - it's reassuring, habitual. and i just prefer to cover those kinds of bases, and i feel like thinking about them keeps me sharp. – mikeserv Nov 14 '15 at 0:56
1

With tail

cat file1.txt && tail -q -n +2 file2.txt file3.txt ..

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.