3

I would like to concatenate several tables and just keeping the header in the first line (and thus not having a new header each time that a new table has been concatenate). Does anyone know how to do that?

I am currently using:

find "/dir/folder" name "*.dat" -exec cat {} + >> "/dir/folder/table.txt"

but I can get rid of the header with that.

3

With a BSD/GNU sed:

find "/dir/folder" -name "*.dat" -exec sed -se1d {} + >> "/dir/folder/table.txt"

...which instructs to treat all input files separately, and for each to delete the 1st line.

If the header isn't already in table.txt, you should first put it there, though:

set -- /dir/folder/*.dat
head -n1 <"$1" >>/dir/folder/table.txt
find ...

If all of the *.dat matches are in child directories, that won't work though.

{   find /dir/folder -name \*.dat -exec \
         sh -c 'head -n1 "$0"; kill "$PPID"' {} \;
    find /dir/folder -name \*.dat -exec sed -se1d {} +
}   >>/dir/folder/table.txt

That should.

  • This doesn't keep the first header as OP want. – cuonglm Aug 3 '15 at 13:25
  • @cuonglm, I was just putting that in. – mikeserv Aug 3 '15 at 13:27
  • That´s close to be ok, just the header is now gone! Even so I have put the set -- /dir/folder/*.dat head -n1 <"$1" >>/dir/folder/table.txt find "/dir/folder" name "*.dat" -exec sed -se1d {} + >> "/dir/folder/table.txt" Do you have any idea why? – steve Aug 3 '15 at 14:49
  • @steve - Nope, not really - it should work as long as there is at least one file which matches *.dat and the first of these contains your header on the first line in the folder /dir/folder/. If all if the *.dat files are in child directories, though, you'll need to glob one of those, I guess. – mikeserv Aug 3 '15 at 14:57
  • @steve - that edit should work. – mikeserv Aug 3 '15 at 15:04
2

If the header is on one line, with GNU tail:

find "/dir/folder" -name "*.dat" -exec tail -qn +2 {} +

POSIXly, you'd need to run one tail per file:

To preserve the header of the first file, GNUly:

find "/dir/folder" -name "*.dat" -print0 | {
   IFS= read -rd '' first &&
     cat "$first" &&
     xargs -r0 tail -qn +2
}
  • I think I have been unclear because none of those answer work. In fact I have thousand of tables in several subfolder within a directory from the root directory I want to access those tables and create a new one grouping all the tables from the subfolder and just having one header in the first line. Thank you very much for your answers! – steve Aug 3 '15 at 13:52
  • @steve, that solution outputs files starting at their second line. You can then append that to a file that already contains the header. – Stéphane Chazelas Aug 3 '15 at 14:14
1

If all your files can be fit in to one awk invocation:

find "/dir/folder" name "*.dat" -exec awk 'FNR == 1 && NR != 1 {next};1' {} + 
  • It´s not really working. Could you learn me what´s the meaning of ['FNR == 1 && NR != 1 {next};1'] ? – steve Aug 3 '15 at 13:55
  • @steve: Yes, like I said, it only works when all your files fit in one awk invocation. FNR == 1 mean the first line of file, NR != 1 mean we aren't at the first line of first file. FNR == 1 && NR != 1 is true when we are at the first line of each files start at second one, if we are here, next skip that line. – cuonglm Aug 3 '15 at 14:01
  • Why downvote here? – cuonglm Aug 3 '15 at 14:24
0

tail allow you to skip given number of lines:

find "/dir/folder" name "*.dat" -exec tail +2 {} + >> "/dir/folder/table.txt"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.