Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have multiple files that contain ascii text information in the first 5-10 lines, followed by well-tabulated matrix information. In a shell script, I want to remove these first few lines of text so that I can use the pure matrix information in another program. How can I use bash shell commands to do this?

If it's any help, I'm using RedHat and an Ubuntu linux systems.

share|improve this question
add comment

2 Answers 2

up vote 14 down vote accepted

As long as the file is not a symlink or hardlink, you can use sed, tail, or awk. Example below.

$ cat t.txt
12
34
56
78
90

sed

$ sed -e '1,3d' < t.txt
78
90

You can also use sed in-place without a temp file: sed -i -e 1,3d yourfile. This won't echo anything, it will just modify the file in-place. If you don't need to pipe the result to another command, this is easier.

tail

$ tail -n +4 t.txt
78
90

awk

$ awk 'NR > 3 { print }' < t.txt
78
90
share|improve this answer
3  
You can also use sed in-place without a temp file: sed -i -e 1,3d yourfile. This won't echo anything, it will just modify the file in-place. If you don't need to pipe the result to another command, this is easier. –  Yanick Girouard May 2 '12 at 23:46
    
As long as the file is not a symlink or hardlink. –  jw013 May 3 '12 at 0:26
1  
Thanks @YanickGirouard, @IgnacioVazquezAbrams! You two have just saved me a ton of manual labor on my research! :) –  Paul Sep 6 '12 at 5:22
    
@jw013 What is the symlink limitation about? Sed? Awk? Tail? All of them? –  user13764 Jan 7 at 22:19
    
@Svetlana sed -i specifically. Most implementations just delete the file and replace it with a new one, which doesn't work for links since you end up leaving the original at its other location. –  jw013 Jan 7 at 22:21
add comment

If the tabulated lines are the ones that have a tab character:

grep '␉' <input_file >output_file

( being a literal tab character) or equivalently

sed -n '/␉/p' <input_file >output_file

In a bash/ksh/zsh script, you can write $'\t' for a tab, e.g. grep $'\t' or sed -n $'/\t/p'.

If you want to eliminate 10 lines at the beginning of the file:

tail -n +11 <input_file >output_file

(note that it's +11 to eliminate 10 lines, because +11 means “start from line 11” and tail numbers lines from 1) or

sed '1,10d' <input_file >output_file

On Linux, you can take advantage of GNU sed's -i option to modify files in place:

sed -i -n '/\t/p' *.txt

Or you can use a shell loop and temporary files:

for x in *.txt; do
  tail -n +11 <"$x" >"$x.tmp"
  mv "$x.tmp" "$x"
done

Or if you don't want to modify the files in place, but instead give them a different name:

for x in *.txt; do
  tail -n +11 <"$x" >"${x%.txt}.data"
done
share|improve this answer
1  
"tabulated" usually means "pretty-printed in a table", not "indented with tab characters". –  Ignacio Vazquez-Abrams May 3 '12 at 2:14
    
@IgnacioVazquez-Abrams I know. The pretty-printed table sometimes uses tab characters, that's easier to spot than aligned columns. Of course, if Paul gave a sample input, I could give a better matcher. –  Gilles May 3 '12 at 10:04
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.