4

GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)

Idea is to set a variable to a NUL delimited data set. Here $samples

This, however, result in:

warning: command substitution: ignored null byte in input

when doing:

samples="$(find . -type d -iregex './sample[0-9][0-9]' -printf "%f\0" | sort -z)"

Thought I could re-use this variable as I need to iterate the same values multiple times:

while IFS= read -rd '' sample; do
    echo $sample
done<<< "$samples"

I could use \n over \0 in the find command in this exact case, but would like to know how, if possible, to do it with NUL delimiter generally speaking.

Optionally I could do:

while IFS= read -rd '' sample; do
    echo $sample
done< <(find . -type d -iregex './E[0-9][0-9]' -printf "%f\0" | sort -z)

but - as I need to loop it several times it makes for some very redundant code - and would have to run the find and sort command each time.

Convert the result into an array perhaps?


  • Is this possible?
  • Why can not NUL delimited data be used as is?
8

It is a fact that you can't store \0 null bytes in a bash string context, because of the underlying C implementation. See Why $'\0' or $'\x0' is an empty string? Should be the null-character, isn't it?.

One option would be strip off the null bytes after the sort command, at the end of the pipeline using tr and store the result to solve the immediate problem of the warning message thrown. But that would still leave your logic flawed as the filenames with newlines would still be broken.

Use an array, use the mapfile or readarray command (on bash 4.4+) to directly slurp in the results from the find command

IFS= readarray -t -d '' samples < <(find . -type d -iregex './sample[0-9][0-9]' -printf "%f\0" | sort -z)
  • Thank you. But would it not then be the same as using -printf "%f\n"? If one are separating with newline in the result one are assuming there are no newlines in the data anyway (if you get what I mean). The whole point of using zero delimiter is to be sure one do not split names with newlines in them … – user3342816 Jun 9 at 16:24
  • @user3342816: I thought your intention was to run the find command once, so after sorting the results, you wouldn't need the null delimited output anymore, so store the result in the variable without it – Inian Jun 9 at 16:27
  • From the link you posted (tried to find something like that first) I read it as it is not possible to store zero in a var, so that much set array to be the "only" option. – user3342816 Jun 9 at 16:27
  • Yes. That was the intention. But if a value can have new-line, the result would be ambiguous even if sorted correctly. – user3342816 Jun 9 at 16:29
  • @user3342816 : Then I suppose the readarray option is the one you want – Inian Jun 9 at 16:33
6

The bash shell does not support what you want to do. The zsh shell does out of the box.

% mkdir sample11 SAMple12 sample21 sample22 dir1
% ll
total 20
drwxrwxr-x 2 fpm fpm 4096 Jun  9 13:46 dir1
drwxrwxr-x 2 fpm fpm 4096 Jun  9 13:46 sample11
drwxrwxr-x 2 fpm fpm 4096 Jun  9 13:46 SAMple12
drwxrwxr-x 2 fpm fpm 4096 Jun  9 13:46 sample21
drwxrwxr-x 2 fpm fpm 4096 Jun  9 13:46 sample22
% samples=$(find . -type d -iregex './sample[0-9][0-9]' -print0 | sort -z)
% echo $samples
./sample11./SAMple12./sample21./sample22
% echo $samples | od -a
0000000   .   /   s   a   m   p   l   e   1   1 nul   .   /   S   A   M
0000020   p   l   e   1   2 nul   .   /   s   a   m   p   l   e   2   1
0000040 nul   .   /   s   a   m   p   l   e   2   2 nul  nl
0000055
%
  • 1
    ... however, please beware that the value of the environment variable samples will be truncated at the nul byte when being passed to child processes: If you perform export samples and start a child instance of zsh, $samples will only be ./sample11 in the child instance. – Martin Rosenau Jun 10 at 8:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.