2

I have an awk script that receives some parameters and filters data from given csv file according to those parameters.

there are 2 outputs:

  • First, it writes a .csv file with all the records that match parameters.
  • Then, it prints only the field $2 which refers to name of records. But it should be a random subset of 20 records.

So far, I have done this:

I call the script this way: ./Script.awk ARG1=20 ARG2="AAA" ARG3=1900 data.csv

#! /usr/bin/awk -f

# Define FS
BEGIN {FS=OFS = ","}

$4 > ARG1 && $8 == ARG2 && $20 > ARG3 { print  > "filtered_data.csv" ; print $2 }

So, it creates correctly the filtered data file and print $2, but there are many records, therefore I would like to print only a random subset. So, any idea how can it be done?

Thanks!!

Improve this question
1
  • 2
    Don't use a shebang to call awk. Use a shebang to define your shell and then simply call awk. That way you can use the shell to do what it does best, and awk to do what it does best. Also don't name variables things like ARG1 as that's obfuscating your code by looking like ARGV[1], using all upper case letters which could clash with builtins, and completely missing the point of variable names - they should identify what the value they hold means, not what position it occurs in or anything else.
    – Ed Morton
    Commented Jan 2, 2021 at 22:34

2 Answers 2

Reset to default
3 
#!/usr/bin/env bash

out='filtered_data.csv'
awk -F, '$4 > ARG1 && $8 == ARG2 && $20 > ARG3' "$@" > "$out" &&
cut -d, -f2 "$out" | shuf -n20

but, again, pick meaningful variable names rather than ARG1, etc.

Improve this answer
1
  • 1
    thank you so much for all your comments! as you could have deduced I'm a newbie with scripting... Actually the ARG1, ARG2, etc.. are not the names I've used, I just wrote a reduced version of the script here, but something more I have learned! Thanks :D
    – guillem
    Commented Jan 2, 2021 at 22:44
1

EDIT: I posted that before Ed Morton's answer, which is clearly better.

I leave it here in case it can be useful to someone else.

Ok, actually was pretty easy... I'm going to answer to myself in case someone needs it! The working script working as desired:

#! /usr/bin/awk -f

# Define FS
BEGIN {FS=OFS = ","}

$4 > ARG1 && $8 == ARG2 && $20 > ARG3 { print  > "filtered_data.csv" ; print $2 | "shuf -n20"}

Just needed to pipe "shuf -n20" with print :)

Improve this answer
2
  • 2
    Your use of a shebang is forcing you to have awk create a file, spawn a subshell, and create a pipe, and call a Unix tool - 4 things a shell exists to do.
    – Ed Morton
    Commented Jan 2, 2021 at 22:42
  • 2
    @Quasímodo My concerns aren't so much about performance in this case, they're about having each tool do what it's designed to do for all the usual reasons that's desirable. The OP already had awk doing 1 thing the shell is best at (create a file) and then had to add another to solve their new requirement (pipe to a shell command). It's just always simpler, cleaner and easier to enhance not to call awk with a shebang.
    – Ed Morton
    Commented Jan 2, 2021 at 23:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .