4

I have several text files, and each file's content is like

attribute1,attribute2,...
1,2,...

The first lines which describe the names of the columns are the same in all the files.

I want to concatenate the second lines of the files, add a new column with filename for each line for which file it comes from, and put one of the first lines as the first line in the new file. For example, the desired new file looks like:

file,attribute1,attribute2,...
file1,1,2,...
file2,11,12,...
file3,21,22,...

How should I do that?

2
  • So each file has just two lines ? Nov 13, 2015 at 19:39
  • yes. @don_crissti
    – Tim
    Nov 13, 2015 at 19:40

3 Answers 3

4

To add the file name, with awk:

awk -v OFS=, 'NR==1{print "file", $0};FNR==2{print FILENAME, $0}' file1 file2 ... fileN >outfile

which prints the updated header if it's the first line of input (NR==1) or (||) if it's the second line of each file(FNR==2) it prints the FILENAME and the line ($0).


Initial answer before your edit.
You could do that with sed:

sed -n '1p;n;p' file1 file2 ... fileN >outfile

the -n suppresses printing, 1p prints the first line and then for each (odd) line it empties the pattern space and reads in the next line (the even lines that is) and prints it.

9
  • thanks. how do you do the same with awk?
    – Tim
    Nov 13, 2015 at 19:48
  • I forgot that I would like to add a new column, which consists of the filename for each line. See my update.
    – Tim
    Nov 13, 2015 at 19:56
  • I realized that after I commented. this is a bit more clear: 'NR==1 || FNR==2' Nov 13, 2015 at 19:56
  • @tim, update your question with the new requirements. Don't bury them in comments. Nov 13, 2015 at 19:57
  • @glenn: exactly.
    – Tim
    Nov 13, 2015 at 20:01
2
{   printf file,; head -n1 <file
    grep -n '' ./file* /dev/null|
    sed -ne'/^[^:]*:2:/s/:.:/,/p'
}   >./new_file

grep will get your filenames at the head of each line, and can assist in selecting by line -number as well. this will probably not work very well, though, if you've got colons in your filenames.

2
  • 1
    Mike, I think you could ditch the /dev/null in this case as we're talking about at least two files. Also, I think this post could be updated (with zgrep) per the latest OP requirements (see comments under my post) and would in fact be much better for what OP actually wants to do as it would work with gz files. Nov 13, 2015 at 20:54
  • 1
    @don_crissti - but i like the /dev/null - it's reassuring, habitual. and i just prefer to cover those kinds of bases, and i feel like thinking about them keeps me sharp.
    – mikeserv
    Nov 14, 2015 at 0:56
1

With tail

cat file1.txt && tail -q -n +2 file2.txt file3.txt ..

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.