0

I have a file named data.txt with the following content:

 1           aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 2           bFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 3           cFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 4           dFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 5           eFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 6           fFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 7           gFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 8           hFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
 9           iFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
10           jFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234
11           kFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf        IT524234

Please note that the first field is the line number.

Now I want to construct a shell script such that I could call that script with some line number arguments and it should print out the 1st and 2nd field of the corresponding line numbers in data.txt. For example: get.sh 1 3 5 should print:

1           aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
3           cFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
5           eFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf

I think awk can be used for printing only 1st and 2nd field but I am stuck at filtering only specific lines based on the arguments passed to the shell script. Thanks in advance.

0
5

In awk, you could collect the line numbers to an array and read through the file once, printing the lines that are mentioned in the array:

#!/bin/sh
awk -v lines="$*" 'BEGIN { split(lines, a, "[, ]"); 
                           for (i in a) b[a[i]] = 1;}
                   NR in b {print $1, $2}' < data.txt

The split() splits the variable lines along spaces and commas to array a, and the for loop builds the array b such that the keys of that array contain the lines we're interested in. Then NR in b just checks if the key matching the current line number exists.

Note that that will print each line only once, regardless of how many times it exists in the input, and the lines will be printed input numeric order, not the order given by the argument:

$ bash get.sh 7 3 3
3 cFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
7 gFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf

(get.sh 7,3,3 works too)

1
  • +1. nice enhancement with the space or comma delimiter. stolen for my perl RE-based version. – cas May 5 at 6:03
1

Put the below lines in a text file and name it as get.sh. then make it executable .

#!/bin/sh
## this is GNU sed
sed -En "
  $(printf '%sbp\n' "$@" "d;")
  :p;s/\S+/&\n/2;P
" data.txt

Now invoke the script as shown:

chmod +x ./get.sh
./get.sh 1 3 5
1
#!/bin/bash

perl -le '
for (@ARGV) {
  # separate command line args into filename(s) and line-number(s)
  # line-numbers can be space and/or comma separated.
  if (-e $_) { push @files, $_ } else { push @lines, split /,/};
};

@ARGV = @files;
$re = join("|",@lines);

while(<>) {
  print join("\t",(split)[0..1]) if ($. =~ m/^($re)$/);
  close(ARGV) if eof;
}' "$@"

This builds a regex from the non-filename arguments, which it later uses to match against each file's line-numbers. On a match, it splits the input line by whitespace and prints the first two fields separated by a tab.

The close(ARGV) is only needed because we care about the line number of the current file, not the line-number of all input seen so far. perl only resets the $. (aka $NR or $INPUT_LINE_NUMBER) variable when a file handle is closed, but file handles aren't normally closed in a while(<>) loop. This just explicitly closes the filehandle so that $. is reset. See perldoc -f eof.

$ ./get.sh 1 3,5 data.txt 
1       aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
3       cFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
5       eFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf

This script, should, of course be a perl script instead of a pointless bash wrapper around a perl "one-liner". But people seem to think that one-liners are "correct", while scripts that use anything except #!/bin/bash or #!/bin/sh as the interpreter are somehow wrong.

#!/usr/bin/perl -l

for (@ARGV) {
  # separate command line args into filename(s) and line-number(s)
  # line-numbers can be space and/or comma separated.
  if (-e $_) { push @files, $_ } else { push @lines, split /,/ };
};

@ARGV = @files;
$re = join('|',@lines);

while(<>) {
  print join("\t",(split)[0..1]) if ($. =~ m/^($re)$/);
  close(ARGV) if eof;
};
$ ./get.pl 1 3,5 data.txt  
1       aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
3       cFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
5       eFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf

This does exactly the same thing without wasting miniscule amounts of time and memory forking a shell interpreter which does nothing but fork the perl interpreter.

More importantly, it avoids issues with shell quoting because the shell is not involved. Also, syntax highlighting works correctly in your editor because the script isn't just a single quoted string inside a shell script. And line-numbers are correct in warning/error messages when debugging a script because they refer to the absolute line-number of the script file, rather than the relative line number inside the one-liner.

1
  • +1 Not only for providing the 'hows' but also the 'whys' , for they fill in an important backgrounder information too. It's all like not only providing green veggies for lunch but also listing out their many pros n cons. (Me just got up from lunch so this analogy was uppermost ) – guest_7 May 5 at 9:48
1

The following bash script builds a sed script that will modify and display the lines by the numbers provided on the command line:

#!/bin/bash

sed_script=()
for lineno do
        sed_script+=( -e "$lineno ba" )
done

sed "${sed_script[@]}" \
        -e 'd' -e ':a' -e 's/[[:blank:]]*[^[:blank:]]*$//' <data.txt

or, for /bin/sh:

#!/bin/sh

for lineno do
    set -- "$@" -e "$lineno ba"
    shift
done

sed "$@" -e 'd' -e ':a' -e 's/[[:blank:]]*[^[:blank:]]*$//' <data.txt

If executed as ./get.sh 1 3 5, both of these scripts would end up executing the command

sed -e '1 ba' -e '3 ba' -e '5 ba' -e d -e :a -e 's/[[:blank:]]*[^[:blank:]]*$//' <data.txt

This corresponds to the sed script

1 ba
3 ba
5 ba
d
:a
s/[[:blank:]]*[^[:blank:]]*$//

The substitution s/[[:blank:]]*[^[:blank:]]*$// would delete the last column in the data (the last series of spaces or tabs followed a series of non-spaces and non-tabs at the end of the line). The rest of the sed code makes sure that the execution branches to the :a label if we're on one of the lines that we want to output (this is what ba does), and that the current line is otherwise removed with d.

You could modify this and delete the hard-coded redirection in from data.txt in the script. Instead, you could redirect any data stream you wished to process:

$ ./get.sh 1 2 9 <data.txt
 1           aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
 2           bFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
 9           iFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf

A similar approach in awk: Give the awk code a bunch of space-delimited line numbers from the script's command line (the code below gets these from the lineno environment variable set up by the calling shell). Unpack these into keys in a lookup table (lines) in the BEGIN block, and then simply test whether FNR (the current line number in the current file) is a key in this lookup table.

#!/bin/sh

lineno="$*" awk '
        BEGIN {
                n = split(ENVIRON["lineno"], a, " ")
                for (i = 1; i <= n; ++i)
                        lines[a[i]] = 1
        }

        FNR in lines { print $1, $2 }'

(The FNR in lines could be changed to lines[FNR] so that the 1's that we store as values actually become useful.)

Testing:

$ ./get.sh 3 4 8 <data.txt
 3           cFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
 4           dFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
 8           hFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf
1
  • I wish my profs explained so verbosely n thoroughly, I would have maxed my grades. – guest_7 May 5 at 9:51
0
#! /bin/bash
# get.sh
IFS=$'\n'
args=(`sort -nu <<<$*`)
unset IFS

awk -v lines="${args[*]}" 'BEGIN{split(lines, ar, " ");}{ for (i in ar) { if (NR == ar[i]) print $1,$2} }' data.txt

In this first an array args is created that contains sorted and unique values.We have used options -n and -u for this. See for more detail.

Then split creates an array ar from variable lines. Now for loop print the required output if element in ar equals to record number(NR).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.