Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way.

I want to delete the first 31 bytes of each file in /foo/. Each file is long enough. Well, I'm sure somebody will deliver me a suprisingly easy solution I just can't imagine. Maybe awk?

share|improve this question
1  
Any awk/sed/ed solution will be line-oriented, so if you don't know the first line will be at least 31 characters then complications ensue. –  glenn jackman May 27 '11 at 15:51

3 Answers 3

up vote 9 down vote accepted
for file in /foo/*
do
  if [ -f "$file" ]
  then
    dd if="$file" of="$file.truncated" bs=31 skip=1 && mv "$file.truncated" "$file"
  fi
done

or the faster, thanks to Gilles' suggestion:

for file in /foo/*
    do
      if [ -f $file ]
      then
        tail +32c $file > $file.truncated && mv $file.truncated $file
      fi
    done

Note: Posix tail specify "-c +32" instead of "+32c" but Solaris default tail doesn't like it:

   $ /usr/bin/tail -c +32 /tmp/foo > /tmp/foo1
    tail: cannot open input

/usr/xpg4/bin/tail is fine with both syntaxes.

share|improve this answer
1  
Suggesting dd here is overkill, tail is more appropriate (simpler, less risk of a killer typo, no spurious messages on stderr). –  Gilles May 27 '11 at 22:16
    
You are right. I usually avoid commands intended to process text files when processing possibly binary ones but "tail +32c" will work here. –  jlliagre May 28 '11 at 8:26
1  
@jlliagre: You have written cut (shouldn't that be tail? ... asis, it doesn't work for me... –  Peter.O May 28 '11 at 9:21
    
Of course, it's tail. Sorry about the mismatch. –  jlliagre May 28 '11 at 9:23
    
@jlliagre: On Solaris, you should have /usr/xpg4/bin ahead of /usr/bin on your PATH, or you'll be stuck in the early 1990s. Many unices (e.g. GNU, BusyBox) no longer support the historical +32c syntax, and take it to mean a file called +32c (as POSIX requires). –  Gilles May 28 '11 at 12:31

tail -c +32 outputs its input minus the first 31 bytes. (Yes, the argument is off by one.) To edit a file in place, use sponge in a loop, or if you don't have it and don't want to bother, do its job in the shell:

for x in /foo/*; do tail -c +32 "$x" | sponge "$x"; done
for x in /foo/*; do tail -c +32 "$x" >"$x.new" && mv "$x.new" "$x"; done

If the commands are interrupted for whatever reason (e.g. a power failure), it might be hard to figure out where you left off. Writing the new files to a separate directory would make things easier.

mkdir /foo.tmp
cd /foo
for x in *; do tail -c +42 -- "$x" >"/foo.tmp/$x" && rm -- "$x"; done
mv /foo.tmp/* /foo
rmdir /foo.tmp

If the files are really large (as in, large enough that having two copies of even a single one is a problem), you can use one of the techniques mentioned in this thread.

share|improve this answer

The following commands cut first 31 bytes from $file (using $file~ as a temp. copy):

dd if="$file" of="$file~" bs=1 skip=31
mv "$file~" "$file"

You only need to list or find all files under /foo/ and execute the two above for every $file found.

share|improve this answer
1  
Swapping bs and skip values will increase the performance. –  jlliagre May 27 '11 at 16:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.