5

This question already has an answer here:

The files are named file00 through file99 (hundred files). They are all in the same directory. How do I grab all the hundred files and delete the first 10 lines from each file? I am using CentOS 7.

marked as duplicate by Scott, Stephen Harris, sam, Kusalananda, PersianGulf Jul 31 '16 at 20:59

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

4

This will delete the first ten lines from all files whose names range from file00 to file99:

sed -i.bak -n '11,$p' file[0-9][0-9]

This might require GNU sed, which is known as gsed on OSX. For other seds, try:

find file[0-9][0-9] -maxdepth 0 -type f -exec sed -i.bak -n '11,$p' {} \;

How it works:

  1. -i.bak tells sed to modify the files in-place, creating a backup file with the .bak extension

  2. -n tells sed not to print any lines unless we explicitly ask it to.

  3. 11,$p tells sed to print lines starting with line number 11 and continuing to the last line (denoted $) in the file.

  4. file[0-9][0-9] is a shell glob that will expand to match all files whose names consist of file followed by two digits.

Alternative

The logic can be inverted. Instead of explicitly printing the lines 11 and after, we could (hat tip: JigglyNaga) delete the first ten lines:

sed -i.bak '1,10d' file[0-9][0-9]
  • 1
    Incrediblly fast :) and probably the best solution.. – sjsam Jul 31 '16 at 7:32
  • 1
    Works if sed resets the line numbers inbetween files. (not on OS X) – Kusalananda Jul 31 '16 at 7:36
  • You could avoid needing the -n flag with 1,10d. – JigglyNaga Jul 31 '16 at 7:36
  • @JigglyNaga Yes, that works. Answer updated to add the 1,10d approach. – John1024 Jul 31 '16 at 7:41
  • But, this might have a disadvantage if say file file34 is a link. The op has not specified any requirements about this – sjsam Jul 31 '16 at 7:42
4

Using a simple loop in bash (or any Bourne-like shell) and GNU or BSD sed:

for f in file[0-9][0-9]; do
  sed -i '.bak' '1,10d' "$f"
done

The sed editing script 1,10d will delete lines 1 through to 10 from the input stream. The other lines will be outputted. With -i '.bak' we ask sed to do the editing in-line in the file, but to back up the original with a .bak suffix.

Alternatives with explicit looping (allows for sub-intervals, ksh93, bash or zsh syntax):

for (( i = 0; i < 100; ++i )); do
  f="$( printf "file%02d" "$i" )"
  sed -i '.bak' '1,10d' "$f"
done

Using ksh93 or zsh, another way:

typeset -Z 2 j
for (( i = 0; i < 100; ++i )); do
  j=$i
  sed -i '.bak' '1,10d' "file$j"
done

The typeset -Z 2 j will force $j into a zero-filled number of width 2. We don't want to typeset i in this way as its value would turn into 00 once it reached 100, effectively turning the loop into an infinite loop. That's for ksh93, zsh seems able to handle it as it apparently uses the value before submitting it to the formatting.

  • 1
    Good catch about ksh93 and i=100 becoming as opposed to expanding to 00. Compare zsh -c 'typeset -Z2 i; i=100; echo "$i $((i))"' with the same with ksh93. – Stéphane Chazelas Jul 31 '16 at 16:22
  • @StéphaneChazelas It's verges on being a bug in ksh93 IMHO. – Kusalananda Jul 31 '16 at 16:24
  • Another way to do ranges / steps: for i in {0..9}{2,4,6,8,0} 100;do ..., or seq For a one-off by hand, it's generally not a problem to generate a filename that you know won't exist (like 00). – Peter Cordes Jul 31 '16 at 17:20
3

To avoid creating temporary files or affecting the permissions, ownership or other attributes of the files, with ksh93:

for f in file{2}([0-9]); do
  tail -n +11 < "$f" 1<>; "$f"
done

The <>; redirection operator acts like <> (open in read+write mode), except that the shell truncates the file upon the command exiting successfully at the position the command left it.

So, we're basically overwriting the file over itself.

2

If you prefer a bash loop:

for f in file[0-9][0-9]; do
    tail -n +11 "$f" > tmp &&
    mv -f tmp "$f"
done

Not the answer you're looking for? Browse other questions tagged or ask your own question.