8

I have three file with n number of lines as given below

sample1.txt

------------------------------
Date        Time    Name    
------------------------------
2013/10/12  12:09:09    Krish
2013/10/12  13:12:01    Ramb
2013/10/12  15:28:39    Likha
2013/10/12  15:56:12    Mat
.
.
.

Sample2.txt

------------------------------
Date        Time    Name    
------------------------------
2013/10/12  11:19:21    Jack
2013/10/12  12:11:09    Rob
2013/10/12  15:45:12    Rick
2013/10/12  22:11:10    Phil
.
.
.

Sample3.txt

------------------------------
Date        Time    Name    
------------------------------
2013/10/12  12:09:09    Eric
2013/10/12  13:12:01    Bob
2013/10/12  15:28:39    Mike
2013/10/12  15:56:12    Nick
.
.
.

I need to merge these three files in a single file(Master.txt) by excluding the headers (First 3 lines) in Sample2.txt and Sample3.txt as given below

Desired Output

$cat Master.txt

------------------------------
Date        Time    Name    
------------------------------
2013/10/12  12:09:09    Krish
2013/10/12  13:12:01    Ramb
2013/10/12  15:28:39    Likha
2013/10/12  15:56:12    Mat
2013/10/12  11:19:21    Jack
2013/10/12  12:11:09    Rob
2013/10/12  15:45:12    Rick
2013/10/12  22:11:10    Phil
2013/10/12  12:09:09    Eric
2013/10/12  13:12:01    Bob
2013/10/12  15:28:39    Mike
2013/10/12  15:56:12    Nick

Note : In AIX machine with Ksh 88

5
  • 4
    This is probably best done programmatically. Possibly using Python and Perl. If you are doing such things frequently, you could consider using R. R will let you import the individual files as data frames, and then you can merge the data frames together. I believe Python's pandas for example offers similar functionality, but I have not used it. Nov 7, 2013 at 11:20
  • @FaheemMitha Perl/Python is certainly overkill for this. Plus Python on AIX might be slightly problematic.
    – peterph
    Nov 7, 2013 at 12:01
  • @peterph Well, maybe. But if he has to do such manipulations frequently, then the R approach has much to recommend it. Though I have no idea about the current status of R on AIX. Though R of pretty much any vintage should able to do these kinds of manipulations. Nov 7, 2013 at 12:08
  • 1
    @FaheemMitha why on earth would you recommend R for this? All the OP needs is to remove topmost 3 lines from all but the first file. Given UNIX strongly text-based interface it is work for the standard utilities.
    – peterph
    Nov 7, 2013 at 12:13
  • 1
    @peterph Well, it is not necessary for this particular manipulation, but it seems the poster has need to manipulate text tables, and in general R is a good way to do so, and probably scales better to more complex situations. An alternative approach of using unix tools is certainly feasible. I'm not suggesting that is a bad way to go. Nov 7, 2013 at 12:17

2 Answers 2

18
{ cat sample1.txt; tail -n +4 sample2.txt; tail -n +4 sample3.txt; } > out.txt
7
  • @Stephane Chazelas: Regrets for the earlier comment. Thanks it works perfectly as expected..!!!
    – Ram
    Nov 7, 2013 at 12:26
  • 1
    @ram the tail -n +4 means that, regardless of the size of the file the top 3 lines are excluded. I just tried it with a 10 line fie. Don't mix up the "4" here with the fact that your sample has 4 lines. That's basically a coincidence. Nov 7, 2013 at 12:27
  • @ram :) for completeness, head has a similar option: -n -X will print all but the last X lines.
    – peterph
    Nov 7, 2013 at 12:34
  • 2
    @peterph, not on AIX, not in the POSIX specification (only positive numbers allowed) Nov 7, 2013 at 12:47
  • Hm, my bad... this is a GNU extension.
    – peterph
    Nov 7, 2013 at 18:20
7
sed '4,${/^---/d;/^Date/d;}' sample1.txt sample2.txt sample3.txt > out.txt
4
  • 2
    +1 but could you add a bit of an explanation? What does the ${} syntax do in sed?
    – terdon
    Nov 7, 2013 at 12:55
  • That's not ${}, but "do {...} on lines 4 to last"; it removes matching lines, but only after the first header has gone by. Using tail is more general and more robust, though.
    – alexis
    Nov 7, 2013 at 15:06
  • @terdon as said, {} is grouping of commands.
    – peterph
    Nov 7, 2013 at 18:15
  • @alexis depends on what you want in the end. One of the advantages (not that important these days though) is that it only spawns one process. Plus you can easily make it a script with #!/bin/sed being the interpreter.
    – peterph
    Nov 7, 2013 at 18:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .