7

I have a file, that has trash (binary header and footer) at the beginning and end of the file. I would like to know how to nuke these bytes. For an example, let's assume 25 bytes from the beginning. And, 2 bytes from the end.

I know I can use truncate and dd, but truncate doesn't work with a stream and it seems kind of cludgey to run two commands on the hard file. It would be nicer if truncate, knowing how big the file was, could cat the file to dd. Or, if there was a nicer way to do this?

12

You can combine GNU tail and head:

tail -c +26 file | head -c -2

will output the contents of file starting at byte 26, and stopping two bytes (minus two -2) before the end. (-c operates on bytes, not characters.)

  • Note that tail -c +26 is standard (head -c -2 is not). – Stéphane Chazelas May 24 '17 at 21:47
  • @StéphaneChazelas is there a POSIX compliant version of the head -c -2 command? – cat May 25 '17 at 1:30
4

dd will do both for you in a single command. Set the block size to 1 byte, skip the 25 first bytes, count to the size of file minus skip and end bytes.

100 byte file
file.img

dd if=./file.img of=./trimed_file.img bs=1 skip=25 count=73

Double check numbers cause it might count from 0.

  • isn't too slow? – Ravexina May 24 '17 at 21:00
  • Too slow? Only if the file is very large like many GB. Then you could adjust the block size. – jc__ May 24 '17 at 21:01
  • it took 0m8.319s for a 1.3 MB file, For the count he can use something like $(( stat -c %s test` - 5 ))` I guess. – Ravexina May 24 '17 at 21:06
  • Remember that skip and count are blocks not bytes. Pick the largest multiple for the block size. The closer the block size is to the HD buffer size the more efficient and faster the command is. – jc__ May 24 '17 at 21:19
  • 4
    with bs=1 dd makes two system calls for each byte, that's horribly slow. GNU dd has the count_bytes and skip_bytes that can help, do something like dd bs=8192 iflag=count_bytes,skip_bytes skip=25 count=73 < infile > outfile – ilkkachu May 24 '17 at 21:25
2

With ksh93:

{ head -c "$n"; } < file <#((n = EOF - 25 - 2 , 25))

Or to do it in-place:

{ head -c "$n"; } < file <#((n = EOF - 25 - 2 , 25)) 1<>; file

If you have /opt/ast/bin ahead of your $PATH, you'll get the head builtin.

  • <#((...)) is a lseek() operator. ... is interpreted as an arithmetic expression where EOF is the length of the file. So above, we're assigning the length of the portion to display to $n and seeking to 25 bytes within the file.
  • <>; is a open-in-read+write-mode-and-truncate-if-the-command-is-successful redirection operator.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.