I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way.

I want to delete the first 31 bytes of each file in /foo/. Each file is long enough. Well, I'm sure somebody will deliver me a suprisingly easy solution I just can't imagine. Maybe awk?

  • 2
    Any awk/sed/ed solution will be line-oriented, so if you don't know the first line will be at least 31 characters then complications ensue. – glenn jackman May 27 '11 at 15:51
for file in /foo/*
  if [ -f "$file" ]
    dd if="$file" of="$file.truncated" bs=31 skip=1 && mv "$file.truncated" "$file"

or the faster, thanks to Gilles' suggestion:

for file in /foo/*
      if [ -f $file ]
        tail +32c $file > $file.truncated && mv $file.truncated $file

Note: Posix tail specify "-c +32" instead of "+32c" but Solaris default tail doesn't like it:

   $ /usr/bin/tail -c +32 /tmp/foo > /tmp/foo1
    tail: cannot open input

/usr/xpg4/bin/tail is fine with both syntaxes.

  • 1
    Suggesting dd here is overkill, tail is more appropriate (simpler, less risk of a killer typo, no spurious messages on stderr). – Gilles May 27 '11 at 22:16
  • You are right. I usually avoid commands intended to process text files when processing possibly binary ones but "tail +32c" will work here. – jlliagre May 28 '11 at 8:26
  • 1
    @jlliagre: You have written cut (shouldn't that be tail? ... asis, it doesn't work for me... – Peter.O May 28 '11 at 9:21
  • Of course, it's tail. Sorry about the mismatch. – jlliagre May 28 '11 at 9:23
  • @jlliagre: On Solaris, you should have /usr/xpg4/bin ahead of /usr/bin on your PATH, or you'll be stuck in the early 1990s. Many unices (e.g. GNU, BusyBox) no longer support the historical +32c syntax, and take it to mean a file called +32c (as POSIX requires). – Gilles May 28 '11 at 12:31

The following commands cut first 31 bytes from $file (using $file~ as a temp. copy):

dd if="$file" of="$file~" bs=1 skip=31
mv "$file~" "$file"

You only need to list or find all files under /foo/ and execute the two above for every $file found.

  • 1
    Swapping bs and skip values will increase the performance. – jlliagre May 27 '11 at 16:07

tail -c +32 outputs its input minus the first 31 bytes. (Yes, the argument is off by one.) To edit a file in place, use sponge in a loop, or if you don't have it and don't want to bother, do its job in the shell:

for x in /foo/*; do tail -c +32 "$x" | sponge "$x"; done
for x in /foo/*; do tail -c +32 "$x" >"$x.new" && mv "$x.new" "$x"; done

If the commands are interrupted for whatever reason (e.g. a power failure), it might be hard to figure out where you left off. Writing the new files to a separate directory would make things easier.

mkdir /foo.tmp
cd /foo
for x in *; do tail -c +42 -- "$x" >"/foo.tmp/$x" && rm -- "$x"; done
mv /foo.tmp/* /foo
rmdir /foo.tmp

If the files are really large (as in, large enough that having two copies of even a single one is a problem), you can use one of the techniques mentioned in this thread.


You can use Vim in Ex mode:

for each in /foo/*
  ex -sc '%!tail -c+32' -cx "$each"
  1. % select all lines

  2. ! run command

  3. x save and close

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.