I have data produced by file, which is identifying the mime-types of all the files in a directory (or directories). The result is a list, with commands on the left and mime-types on the right. By default, these two values (or strings?) are delimited by a colon (:). I want to instead delimit the commands from their mime-types with the NUL character. So,

files=( ${(f)"$(file --print0 --mime-type **/*)"} )

This seems to work.

print -l ${files[@]} gives:

cs:           text/x-shellscript
da:           text/x-shellscript
dns-test:     text/x-shellscript

Or, print -l ${(V)files[@]} gives:

cs^@:           text/x-shellscript
da^@:           text/x-shellscript
dns-test^@:     text/x-shellscript

But, when using subscripting to match the first word, the output is the first character instead. So,

for f in "${files[@]}"; do
  print "${f[(pws:\0:)1]}"

This only prints the first character of each array element (each line outputted by file):


If I instead omit the specification of (ps:\0:), thus using the default of separating on white-space, then I do get the first word of each element. So,

for f in "${files[@]}"; do
  print "${f[(w)1]}"

This works:


But I don't want to rely on white-space. I want to use a safer delimiter like NUL. Note that I also tried the subscript [(pws:\000:)1], but this also only produced the first character.

How can I identify words delimited by NUL when working with subscripts?


1 Answer 1


File paths can contain newline characters. That's why several utilities that report files or things about files offer you to delimit records with NULs instead of newline as 0 is the only byte value that cannot occur in a file path.

If you're splitting the output on newline as you do with ${(f)"$(file...)"}, you're defeating the purpose.

file is a bit of an odd one in that it doesn't use NUL as its record delimiter, but uses it to delimit the file path in its output record (which remains loosely defined).

$ ls
':\0:'$'\n'':\1:'  'a'$'\n''b'
$ file --print0 --mime-type -- ./* | sed -n l
:\\1:\000: application/x-xz$
b\000:       application/zip$

So its record is <file-path><NUL><colon><some-spaces><mime-type><newline> which makes it unnecessarily more difficult to parse (even impossible if the text contains newlines which in the case of --mime-type thankfully should not be the case).

Here, you could do:

typeset -A mime_type
file --print0 --mime-type --no-pad --separator '' ./**/* |
    IFS= read -rd $'\0' file &&
      IFS=' ' read -r type

Where the first read with NUL as -delimiter reads the NUL-delimited file path and the second reads the newline-delimited mime-type (trimming the leading whitespace with IFS processing).

The --no-pad turns the <some-spaces> into a single space, --separator '' removes the colon, so we have <file-path><NUL><space><mime-type><newline>

In the loop, we fill the $mime_type associative array, so you can thereafter get the type of a given file with $mime_type[./given/file] or list the files of a given type with print -rC1 -- ${(k)mime_type[(R)text/plain]}.

As to why $scalar[(pws[\0])1] doesn't split on NULs, that's a bug in current versions of zsh. The behaviour is the same as in $array[(pws[])1] suggesting zsh fails to escape the NUL as it otherwise usually does so you end up splitting on the empty string. That missing escaping affects other separators like all those containing byte values 0x83 to 0xa2 (such as á which in UTF-8 is encoded as 0xc3 0xa1).

Without s[separator], I find it splits on $IFS characters (not whitespace, another bug, a documentation one this time), NUL being in the default value of $IFS, so you could do: IFS=$'\0'; first_non_empty_NUL_separated_field=$scalar[(w)1], but you could also use the 0 (short for ps[\0]) parameter expansion flag:

non_empty_NUL_separated_fields=( ${(0)scalar} )
NUL_separated_fields=( "${(0@)scalar}" )

(and then first=$NUL_separated_fields[1]).

Or in one go first=${${(0)scalar}[1]}.

Or you can use the ${scalar%%pattern} Korn shell operator:


Or IFS-splitting:

NUL_separated_fields=( $=scalar )

Or ksh93-style ${param/pattern[/replacement]}:


Some comments on your code:

  • in file --print0 --mime-type **/*, if any of the file paths starts with -, that would be taken as an option to file. You could change it to file --print0 --mime-type -- **/* to avoid that but that still doesn't work properly if there's a file called - in the current working directory. Using ./**/* avoids both problems (and further problems down the line).
  • @, whether as a parameter expansion flag or as [@] or $@ only really makes sense when quoted, to prevent removal of empty elements. ${files[@]} in list context is the same as $files or $files[@] or $files[*] and expands to the non-empty elements in the array. Note that in Korn-like shells, you do also need the quotes, to prevent empty-removal but also to prevent split+glob there. Korn-like shells (contrary to most other shells or languages) also require the braces, and $files alone expand to the element of indice 0, not all the elements.
  • With print, you generally want to use the -r option without which print does some backslash processing, and you almost always want to use the -- or - option delimiter without which you introduce command injection vulnerabilities. print $var is an ACE vulnerability. Like in the Korn shell where that builtin comes from, you want print -r - $var or print -r -- $var (though in ksh, you'd need print -r - "$var").
  • print -rC1 -- $list is generally better than print -rl -- $list to print a list raw on 1 Column as the latter prints an empty line if passed no argument.
  • This is a great workaround, but is it possible to use \0 as the delimiter in "${f[(pws:\0:)1]}" as the OP wanted? I had assumed that it isn't, since bash at least can't deal with NUL in variables as far as I know, but you said zsh has no issue with NUL. Does that mean it could do something like this, somehow?
    – terdon
    Nov 12, 2022 at 19:21
  • 2
    @terdon, it's not a work around. The OP's approach of splitting on newline and then split each resulting line on NULs is wrong. Now, why $scalar[(pws:\0:)1] doesn't split on NULs looks like a bug. You could always use IFS=$'\0' and $scalar[(w)1] or ${${(0)scalar}[1]} as a work around. Nov 12, 2022 at 19:38
  • I don't think this works for me. In my program, I want both the filename (on the left) and its type. This way I can run a loop, checking if the right side contains a specific string (text, audio, image...), and if so, then storing/printing the filename on the left. From there, I can rm/trash all the files of a specific mime type in the directory.
    – Pound Hash
    Nov 12, 2022 at 21:41
  • 1
    @PoundHash, that's exactly what it does: loop over file's output with the file in $file and the type in $type. In the body of the loop here we set the $mime_type associative array, but you can do anything else you want. With the types in an associative array, you can do things like rm -f ${(k)mime_type[(R)application/x-pie-executable]} to delete files of a given type. Nov 13, 2022 at 7:14
  • 1
    @PoundHash, you use NUL instead of NL as delimiters for file paths because NL is valid in a file path but NUL is not. That's why you have find -print0, du --files0-from and why file --files-from is broken by design. If you split the output of file on NLs, you're breaking appart multi-line file paths. "record" here is the unit of information returned by file. That can't be a "line" because a file path can be made of several line, hence the NUL's. file's record however is the contenation of a NUL delimited record and a line ( a newline delimited record) which makes it a bit awkward. Nov 13, 2022 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.