In a shell script how can I get the same output from a variable for piping like a command does (for example: cat data.dat | awk ...)?

list=`cat data.dat`

then update list with running it through awk.

closed as unclear what you're asking by Kusalananda, Jeff Schaller, Isaac, RalfFriedl, schily Nov 3 '18 at 17:20

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • It's unclear what the purpose of your loop is. – Kusalananda Nov 2 '18 at 13:49
  • The purpose of it is to filter continuosly by the parameters. I chose this way because i also have to filter only by the 4th field of the input. – peter Nov 2 '18 at 13:53
  • You can simplify your awk to awk -F ";" -v keyword="key" '$4~keyword', since "print" is the default action and specifying conditions as awk "patterns" is cleaner. – filbranden Nov 2 '18 at 14:10
  • It's still unclear what you want to do. Where is the keyword read from? Is it coming from the same file that you run the awk program is running over? Can you describe with words tho overall problem that you are trying to solve by writing this script? – Kusalananda Nov 2 '18 at 14:10
  • Sorry if it was unclear I just wanted to give a little context. The problem was that I wanted to substitute the awk piped input from cat data.dat with a variable (the same list variable). The accepted answer works with list="$(echo "$list" | awk -F ";" -v keyword="$i" '{ if ($4~keyword) { print }}')" – peter Nov 2 '18 at 14:17

did you try as

export list

and in your list-assignment, use

echo $list | awk - - - -- 
  • 1
    Thank you, it almost did work, just needed extra quotes around $list so it had the line feeds too (like the cat data.dat had). – peter Nov 2 '18 at 13:51
  • 1
    Using export is not needed here. The variable is not used by an external command. – filbranden Nov 2 '18 at 14:08
  • 1
    @peter a) you should use printf %s "$list" instead of echo b) you should use "$@" instead of $* c) if you just want to filter all lines whose 4th field matches one of the keywords in argv, you can simply do it with list=$(IFS='|'; awk -F';' -v kw="$*" '$4~kw' data.dat), no need for any loop. – mosvy Nov 3 '18 at 10:07
  • Thanks @mosvy this is more elegant, but it's not filtering to include all the keywords, gives back rows which contains at least one. – peter Nov 3 '18 at 22:04
  • @peter if you have a file where the lines are fields separated with ;, and you want to find all lines in which the 4th field contains all the keywords from a list, no matter the order they're in (as it was in your original question that you trashed, why?), you can do it with this: mgrep(){ r=; for a; do r="$r(?=.*\b$a\b)"; done; grep -P "([^;]*;){3}$r"; } then mgrep <file "$@" or mgrep <file kw1 kw2 .... Please make another question with your actual problem, since your approach was deficient and could probably be done much easier even without GNU extensions. – mosvy Nov 8 '18 at 13:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.