6

Given a binary file, how do you convert it to a hex string and back, using only standard tools like sed and cut, on a minimal system with busybox installed?

These tools are not available:

  • perl
  • python
  • xxd (comes with vim)
  • gcc

A hexdump command comes with busybox, but it is different from the one that comes with util-linux.

I'm looking for a script or command to convert a file to a hex string, and a corresponding one for converting it back to binary. The intermediate format doesn't have to be hex, it can be base64 or something else.

This is for an embedded device with limited disk space.

  • Out of interest, why do you need to? What constraints are there on the string, does it just have to be printable? – Useless Mar 20 '17 at 10:35
  • It's for an old system, for copying a file over telnet in an expect script, for debugging. The string does not have to be printable, but it's an advantage that it doesn't have to be escaped when using expect. (I know, I know, telnet is pretty bad. Luckily, it was removed in all later versions). – Alexander Mar 20 '17 at 12:05
  • OK, so mostly need to avoid control characters. Do you know what version of busybox you have? – Useless Mar 20 '17 at 12:39
  • Yes, 1.18.4 (quite old) or 1.26.2 (the latest version). – Alexander Mar 20 '17 at 12:47
  • From the source, it looks like 1.18.4 should have both uuencode and makemime. Are they not available? – Useless Mar 20 '17 at 13:04
8

Here's what I came up with (based on several online sources and some experimentation).

Converting from hex to bin (hex2bin):

#!/bin/sh
sed 's/\([0-9A-F]\{2\}\)/\\\\\\x\1/gI' "$1" | xargs printf

Converting from bin to hex (bin2hex):

#!/bin/sh
hexdump -C "$1" | cut -b9- | cut -d"|" -f1 | tr -d ' \t\n\r'

Example use:

./bin2hex binary_file_1 | ./hex2bin - > binary_file_2
diff -s binary_file_1 binary_file_2

This works with busybox, but hex2bin is unfortunately limited by the maximum length of the argument given to xargs, so this method will only work for small files (less than 32 KiB on my desktop system).

  • 1
    Rumors are there is such a thing as useless use of cat. – Satō Katsura Mar 20 '17 at 9:45
  • Thanks, removed cat frrom bin2hex. When using sed -i instead of cat and sed, however, sed complained that it could not read - (while cat had no such complaint). – Alexander Mar 20 '17 at 9:53
  • 1
    @Alexander, -i tries to edit in place, which is hard to do for a stream. You can still do without the cat and use sed < "$1" 's/...'" | ... – ilkkachu Mar 20 '17 at 9:56
  • 1
    @Alexander, ah yes, I missed you wanted that. Though of course you could explicitly test $1 against a dash but of course it's easier since cat has the logic built-in. – ilkkachu Mar 20 '17 at 10:15
  • 2
    Couldn't bin2hex simply be hexdump -v -e '1/1 "%02x"' "$1" – jbo5112 Apr 28 '17 at 20:55
4

POSIXly (and only using a common subset compatible with busybox (or at least the busybox as built for the current busybox Debian package):

  • bin2hex:

    (
      export LC_ALL=C
        od -An -vtx1 |
        tr -s ' \t\n' '\n\n\n' |
        grep .
    )
    

    (one hex per line)

  • hex2bin:

    (
      export LC_ALL=C
        awk '
          BEGIN{
            for (i = 0; i < 256; i++)
              c[sprintf("%02x", i)] = sprintf("%o", i)
          }
          NR % 200 == 1 {printf "%s", end "printf '\''"; end = "'\''\n"}
          {printf "\\%s", c[$0]}
          END {print end}' |
        sh
    )
    

If your busybox, contrary to Debian's one has been built without the DESKTOP option, then the -An and -tx1 option to od won't be available. You can use od -b instead which will give a one-byte octal dump with octal offsets. od -b is Unix but not POSIX however so won't work on every Unix-like system.

bin2hex would become:

(
  export LC_ALL=C
    od -b |
    awk '
      BEGIN{
        for(i = 0; i < 256; i++)
          hex[sprintf("%03o", i)] = sprintf("%02x", i)
      }
      NF > 1 {for (i = 2; i<= NF; i++) print hex[$i]}'
)

Again, tested only with Debian's busybox, I can't tell how much of that is dependant on some busybox compile-time option or another. You'd have to test on the target system.

  • When running this bin2hex script on the hardware device, I get: ``` od: invalid option -- 'A' BusyBox v1.26.2 (2017-03-13 14:50:51 CET) multi-call binary. Usage: od [-aBbcDdeFfHhIiLlOovXx] [FILE] Print FILE (or stdin) unambiguously, as octal bytes by default ``` I get the same result on my desktop system. – Alexander Mar 20 '17 at 12:13
  • 1
    @Alexander, yes, as always with busybox, how much of the POSIX standard is implemented depends very much on the version and the selected configuration options at build time. Here, for it to support od -A, it seems you need to tick the DESKTOP option ("Enable options for full-blown desktop systems") at build time (as is done with my Debian busybox it would seem). You can always leave out the -An and filter out the address with sed, but it looks like it would still not support -tx1, so you're probably out of luck with your busybox od. Or you'd need awk again to translate oct to hex – Stéphane Chazelas Mar 20 '17 at 12:26
  • @Alexander, see edit if the alternative works on your target system. – Stéphane Chazelas Mar 20 '17 at 12:49
  • The new version seems to freeze on my desktop system (64-bit Arch Linux, busybox 1.26.2). On the test hardware with the new version of busybox (1.26.2) it also times out and produces no output. I'm not well versed enough in awk to see why. od -b works as expected. busybox awk is present. Same result with busybox 1.18.4. – Alexander Mar 20 '17 at 13:04
  • @Alexander, are you feeding the output on stdin (of that subshell, as in (...) < file)? – Stéphane Chazelas Mar 20 '17 at 13:14
2

Your busybox may have been built with the applet base64 so you could do

busybox base64 <bin >txt
busybox base64 -d <txt >bin

Or if not you may have the venerable uuencode and uudecode, which may also have the -m option to encode in hex. Eg

echo -n abcABC | busybox uuencode -m - | busybox uudecode - 

List the built-in applets with busybox --list.

  • Thanks! base64 and uuencode are unfortunately not available on the hardware device, but it works great on my desktop system. – Alexander Mar 20 '17 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.