Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

For example : 01.txt, 02.txt, 03.txt and 04.txt has following values.

HDR|A|B|C|D|F|||||||
DTL|1|2|3|4|5|||||||||||
DTL|5|6|7|8|9|||||||||||
TLR|A|B|C|D|F|||||||

I have to remove complete lines based on tags like HDR, TLR.

Once HDR and TLR lines are removed from four files, it should merge into single file .

we should merge in such way , should maintain files orders after removing HDR , TLR lines.(01_latest.txt, 02_latest_txt, 03_latest.txt and 04_latest.txt)

How can we achieve in UNIX Script?

share|improve this question
add comment

2 Answers

You can remove those lines using grep too:

$ grep -vE "HDR|TLR" *.txt > merged.txt

This will skip any lines that contain a "HDR" or "TLR". The | is an or operator so you can expand it as needed too, "STR1|STR2|STR3", for example. Looping through the files is guaranteed so long as the names are consistently 01.txt, 02.txt, etc.

You can convince yourself of this with the following example.

$ ls -1
01.txt
02.txt
03.txt
04.txt
05.txt
06.txt
07.txt
08.txt
09.txt
11.txt
12.txt
13.txt
14.txt
15.txt
16.txt
17.txt
18.txt
19.txt

$ echo *.txt
01.txt 02.txt 03.txt 04.txt 05.txt 06.txt 07.txt 08.txt 09.txt 11.txt 12.txt 13.txt 14.txt 15.txt 16.txt 17.txt 18.txt 19.txt

Here you can see that the shell expanded *.txt in the sequential order that you're looking for.

share|improve this answer
add comment

You can remove those lines and merge everything using

sed '/^\(HDR\|TLR\)|/d' [0-9]*.txt > merged.txt

Here it is assumed, that the first entry of the line that should be removed is exactly HDR or TLR.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.