how to read a nul-terminated-string from a binary file

Question

I have a binary file that is filled with FF values. I filled its start with many \000. Then, I padded its start with 10 \000, in order to get some kind of offset, and then I wrote a shorter string, also terminated with \000

I used this printf:

printf \000\000\000\000\000\000\000\000\000\000MAC_ADDRESS=12:34:56:78:90,PCB_MAIN_ID=m/SF-1V/MAIN/0.0,PCB_PIGGY1_ID=n/SF-1V/PS/0.0,CSL_HW_VARIANT=D\000' > eeprom

This is how it looks like when I show the hexdump of the file

Now what I want to know is how can I read the string. I can use MY_STR=${eeprom:OFFSET} (eeprom is the file name) which will give me the string, but will also give me the rest of the file which I don't want. How can I stop it when it first encounters \000?

• Can't use MY_STR=${eeprom:OFFSET:LENGTH} because string's length is unknown
• Another thing - how can I fill it up again with FF?
• Using sh (busybox)

EDIT

Trying to do some minor example of this... I have one file input with this values (after xxd -c 1 input):

0000000: 68  h
0000001: 65  e
0000002: 6c  l
0000003: 6c  l
0000004: 6f  o
0000005: 2c  ,
0000006: 20
0000007: 00  .
0000008: 69  i
0000009: 74  t
000000a: 27  '
000000b: 73  s
000000c: 20
000000d: 6d  m
000000e: 65  e
000000f: 2c  ,
0000010: 00  .

and I have this script s.sh:

BUF=""
for c in $(xxd -p input); do
    if [ "${c}" != 00 ]; then
        BUF="$BUFc";
    else
        break;
    fi
done

echo $BUF

and I expected it to echo "hello", however, nothing is printed

Answer 1

Solution 1: Direct Variable Assignment

If all you're worried about are the null bytes then you should just be able to directly read the data from the file into a variable using whichever standard method you prefer, i.e. you should be able to just ignore the null bytes and read the data from the file. Here's an example using the cat command and command substitution:

$ data="$(cat eeprom)"
$ echo "${data}"
MAC_ADDRESS=12:34:56:78:90,PCB_MAIN_ID=m/SF-1V/MAIN/0.0,PCB_PIGGY1_ID=n/SF-1V/PS/0.0,CSL_HW_VARIANT=D

This worked for me inside of a BusyBox Docker container.

Solution 2: Using xxd and a for loop

If you want more control than you can use xxd to convert the bytes to hexadecimal strings and iterate over these strings. Then, while iterating over these string, you can apply whatever logic you'd like, e.g. you could explicitly skip over the initial null values and print the rest of the data until you reach some break condition.

Here's a script that specifies a "white-list" of valid characters (ASCII 32 through 127), treats any subsequence of other characters as a separator, and extracts all valid substrings:

#!/bin/sh
# get_hex_substrings.sh

# Get the path to the data-file as a command-line argument
datafile="$1"

# Keep track of state using environment variables
inside_padding_block="true"
inside_bad_block="false"

# NOTE: The '-p' flag is for "plain" output (no additional formatting)
# and the '-c 1' option specifies that the representation of each byte
# will be printed on a separate line
for h in $(xxd -p -c 1 "${datafile}"); do

    # Convert the hex character to standard decimal
    d="$((0x${h}))"

    # Case where we're still inside the initial padding block
    if [ "${inside_padding_block}" == "true" ]; then
        if [ "${d}" -ge 32 ] && [ "${d}" -le 127 ]; then
            inside_padding_block="false";
            printf '\x'"${h}";
        fi

    # Case where we're passed the initial padding, but inside another
    # block of non-printable characters
    elif [ "${inside_bad_block}" == "true" ]; then
        if [ "${d}" -ge 32 ] && [ "${d}" -le 127 ]; then
            inside_bad_block="false";
            printf '\x'"${h}";
        fi

    # Case where we're inside of a substring that we want to extract
    else
        if [ "${d}" -ge 32 ] && [ "${d}" -le 127 ]; then
            printf '\x'"${h}";
        else
            inside_bad_block="true";
            echo
        fi
    fi
done

if [ "${inside_bad_block}" == "false" ]; then
    echo
fi

Now we can test this out by creating an example file which has both \x00 and \xff subsequences separating substrings:

printf '\x00\x00\x00string1\xff\xff\xffstring2\x00\x00\x00string3\x00\x00\x00' > data.hex

And here's the output we get when running the script:

$ sh get_hex_substrings.sh data.hex
string1
string2
string3

Solution 3: Using the tr and cut commands

You could also try using the tr and cut commands to deal with the null bytes. Here's an example of extracting the first null-terminated string from a list of null-terminated strings by squeezing/collapsing adjacent null-characters and converting them to newlines:

$ printf '\000\000\000string1\000\000\000string2\000\000\000string3\000\000\000' > file.dat
$ tr -s '\000' '\n' < file.dat | cut -d$'\n' -f2
string1