2

I find xxd to be nice and convenient for simple manipulation of binary data in the terminal. However it has a limitation in that all addresses outputted by the program is limited to 32 bits. E.g.

root:/# xxd -s 0x5baae0000 /dev/sda
baae0000: 2d7c 6176 6976 6172 7c61 7469 7a61 720a  -|avivar|atizar.

Notice the 5 at the beginning is gone. This means that using this directly with xxd -r will end up patching a region over 10 billion bytes earlier than what I want. In other words I want to e.g. do this,

root:/# echo HELLO | xxd -o 0x5baae0000 | xxd -r - /dev/sdb

but I end up having to work around this by doing something like this,

root:/# echo HELLO | xxd -o 0x5baae0000 | xxd -r -seek 0x500000000 - /dev/sdb

which adds back the missing 0x5000000 bytes offset into the target file when parsing the patch.


In any case I want to know if there is either a unknown/clever way to have xxd use 64 bit addresses or an alternative program that operates like xxd without these limitations?

2

xxd uses longs for the address but explicitly truncates it to 32bit and there doesn't seem to be any way around that. That's vim issue #3791 (thanks for reporting it), now fixed in patch 8.1.0854.

It doesn't however seem it ever had that limitation on input (with -r), so you can use standard od to print the hex dump:

od -j 0x5baae0000 -Ax -vtx1 -N16 /dev/sda

Which will output something like:

5baae0000 2d 7c 61 76 69 76 61 72 7c 61 74 69 7a 61 72 0a
5baae0010

However note that several od implementations, including GNU od¹ don't seek to reach the offset specified by -j but read and skip all the data which makes it impractical for a large block device like in your case.

Or if you care about the ASCII side, using BSD hexdump's

$ hexdump -ve '"%_ax:" 16/1 " %02x"' -e '"  " 16 "%_p" "\n"' -s 0x5baae0000 -n 16 /dev/sda
5baae0000: 2d 7c 61 76 69 76 61 72 7c 61 74 69 7a 61 72 0a  -|avivar|atizar

Both of which you can feed to xxd -r

You can actually reproduce xxd's output format (without the 32 bit address limitation) with hexdump with:

hexdump -ve '"%08_ax: " 2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"" "
                        2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"
            ' -e '"  " 16/1 "%_p" "\n"'

Neither od nor hexdump support an equivalent of xxd's -o to offset the address, but you could always post-process their output to add the offset to the address fields with for instance:

perl -pe 's/^\w+/sprintf "%08x", 0xabcdef + hex$&/e'

In any case, to write data at specific offset into a file, you don't need xxd, you can use dd:

echo HELLO | dd bs=1 seek="$((0x5baae0000))" of=/dev/sda

Or ksh93 >#((...)) seeking operator:

echo HELLO 1<> /dev/sda >#((0x5baae0000))

or zsh's sysseek builtin:

zmodload zsh/system
{
  sysseek -u 1 $((0x5baae0000)) &&
    echo HELLO
} 1<> /dev/sda

You can also use xxd with a 0 offset and use dd/ksh93/zsh to do the seeking:

echo HELLO | xxd | { sysseek -u 1 $((0x5baae0000)) && xxd -r; } 1<> /dev/sda

or:

echo HELLO | xxd | { dd bs=1 seek="$((0x5baae0000))" count=0 && xxd -r; } 1<> /dev/sda

¹ Looking at the source, GNU od uses fstat() to get the size of the input (instead of lseek(SEEK_END)) so as not seek past the end of the file, which on systems like Linux where st_size doesn't reflect the size of the block device doesn't work for block devices

  • This doesn't capture the problem at all, sorry. It's the intermediate format of xxd I need (save it, manipulate it, use it). It's not actually writing e.g. «HELLO» to a specific location in a file that is the problem. I even gave an example myself how I could do it. Thanks for pointing out relevant code though. – Christer Jan 4 at 21:15
  • @Christer, see edit. – Stéphane Chazelas Jan 4 at 21:59
  • od does not support seek and took 50 seconds to read those 16 bytes. hexdump on the other hand is almost perfect, though please also add -v to your answer. Sadly doesn't seem hexdump have an analogue to xxd -o which it a big bummer. Still going to mark this as accepted for now, thank you. :) – Christer Jan 4 at 23:20
  • @Christer, see edit. – Stéphane Chazelas Jan 5 at 9:39
1

A patch with a fix for the issue has now been officially included as a part of vim as patch 854. So either try to update to a newer version of vim (v8.1.0854+) that includes the patch or compile from source yourself if not yet available.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.