3

I've looked this up and can't find an answer, I apologise in advance if it's been previously asked.

I'm using shell on FreeBSD (/bin/sh) and I want to dump to stdout all shell (not environment) variables starting with _myvar_. The closest I can get is set | grep '^_myvar_' but that only dumps the first line of multiline variables (some will be multiline and I need them in full), and it could be error prone in pathological edge cases such as a string containing "myvar" that happens to line-break just before that part.

if I could list just variable names (not values), then I could filter within a do...read...while and get the values one at a time just for vars with matching names, but I can't find a way to do this. I also can't filter the full output, because there is no deterministic way to tell whether an output line contains a continuation or a new variable, that doesn't have edge-case issues with strings containing _myvar_, = or newline (\n) characters, or possibly, trailing spaces. I don't want to modify the environment, because the code is included in other code and the environment has to be stable for it.

It isn't a problem for the output/list to include any matching environment variables if that helps (if any exist - it's extremely unlikely they will)

Is there a way to do this?

  • 1
  • Shell. They show up in set but not in printenv when using /bin/sh, and cease to exist when the script finally exits. Question updated to be clearer. – Stilez Aug 13 '18 at 17:53
  • In ksh or Bash, you could use "${!_myvar_@}", though that would still include exported variables – ilkkachu Aug 13 '18 at 18:18
  • @ilkkachu -But the shell in the question explicitly isn't ksh or bash. Do you know a way to do it in a more traditional type of shell like the one involved in the question? – Stilez Aug 13 '18 at 18:31
  • @Stilez, yep. You could pick the variable names from the set output, you might just get some extras if some variables had values that would look like something set would output for another variable, but you could check those later. Though indirect references to the variables would also be easier in Bash or ksh, as you wouldn't need to use any eval trickery. So yeah, I'd suggest at least looking at using some other shell, one that would be more suited to that job... – ilkkachu Aug 13 '18 at 19:40
3

As the set builtin of FreeBSD sh outputs in a format that is suitable for reinput to the shell, you can do:

out() case $1 in (_myvar_*) printf '%s\n' "${1%%=*}"; esac
eval "$(set | sed 's/^/out /')"

That is prefix each line of the output of set with "out " and have that evaluated as shell code (where out is a function that prints the substring of its first argument up to the first =).

sed would also insert "out " in the content of multiline variable, but that would still be included in the argument that our out function receives and past the first =, so in the part we're not displaying.

For instance, on a set output like:

TERM=xterm
USER=stephane
_myvar_foo='line1
line2
line3'

We would be evaluating:

out TERM=xterm
out USER=stephane
out _myvar_foo='line1
out line2
out line3'

But that's still fine as out is only called 3 times for those 3 variables.

To print both variable name and value:

out() case $1 in (_myvar_*)
  eval 'printf "name: \"%s\" value: \"%s\"\n" "${1%%=*}" "${'"${1%%=*}"'}"'
esac
eval "$(set | sed 's/^/out /')"

Note that it only outputs variables, not other types of parameters like $-, positional parameters...

That approach only works for sh implementations where set only outputs scalar variables (won't work for arrays or associative arrays or compound variables, where out var=(x) becomes a syntax error). Those shells that have other variable types often also have better introspection features.

In zsh:

typeset -pm '_myvar_*'

or for the names only

echo $parameters[(I)_myvar_*]

In bash:

v=("${!_myvar_@}"); ((${#v[@]})) && typeset -p -- "${v[@]}"
echo "${!_myvar_@}"
|improve this answer|||||
  • Interesting. I'm assuming out() is a normal function definition without the more common out() {\n ...CODE...\n}\n around the function's code? (The rest makes sense). Also how robust is this against pathological values/newlines, or is that purely down to the function executed and any escaping/substitutions done within eval? – Stilez Aug 14 '18 at 6:35
  • @Stilez, yes definition a function is sticking fname() in front of a command, though with some shells like bash or yash that only works for compound commands (some other shells also have issues when the command is not a compound one and has redirection which is probably POSIX only requires compound commands to be supported). In any case, a case ... esac like {...;} are compound commands. – Stéphane Chazelas Aug 14 '18 at 6:59
  • @Stilez, it should be robust as long as set is correct, as long as it outputs something that can indeed be used as input to the shell as it's meant to. You can't do that with any sh though. Not with bash for instance where the arrays would break that. – Stéphane Chazelas Aug 14 '18 at 7:01
  • The values should in theory be strings only - although one can't prevent a user from defining additional variables otherwise (but it's unlikely and someone who defines pathological vars with names matching internal/private ones, probably deserves the outcome ;-) ). If I have to switch to bash at any time, there are probably solutions for that shell. But at the moment, it looks like it has to be the traditional /bin/sh not bash. I'll check if it works nicely when home :) thanks – Stilez Aug 14 '18 at 7:14
  • 1
    both case $1 in _myvar_*) and case $1 in (_myvar_*) are standard sh. The latter was not supported by the Bourne shell. I prefer the former as I find it more legible and it helps with vi's parens matching. I don't see that shellcheck complains about it (tried with case $1 in (a) echo a; esac with a #! /bin/sh - shebang at shellcheck.net. Possibly you have an older version of shellcheck. – Stéphane Chazelas Aug 14 '18 at 10:23
1

I ended up with the following simplification of @StéphaneChazelas solution, because I didn't need more. Posting it below in case it helps anyone. But Stéphane found the answer, this is just a modded version of it, nothing more.

#!/bin/sh

get_vars() {
  if printf '%s' "$1" | grep -qE '^_myvar_'; then
    printf '%s\n' "$1" | sed 's/^get_vars //'
  fi
}

eval "$( set | sed 's/^/get_vars /' )"

The eval call prefixes each line, but because of the way set outputs its values, it actually creates a single command get_vars VARNAME=VALUE for each variable. Evaluating this executes the commands. Then each time it's called, get_vars checks if the variable which is captured in the current line matches the required regex, and if so, removes the "get_vars " prefix from any second and further lines, and displays the result.

|improve this answer|||||
  • 1
    Note that it may look simpler, but it's a lot more expensive as it forks 4 processes and runs two external utilities for each variable (compared to none in my approach). It has however the benefit of giving an output that is suitable for reinput to the shell. replacing grep with a case statement would improve performance. – Stéphane Chazelas Aug 14 '18 at 9:44
  • Learn every day. Thank you for an excellent answer + comments! – Stilez Aug 14 '18 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.