I have two files one with 17k lines and another one with 4k lines. I wanted to compare position 115 to position 125 with each line in the second file and if there is a match, write the entire line from the first file into a new file. I had come up with a solution where i read the file using 'cat $filename | while read LINE'. but it's taking around 8 mins to complete. is there any other way like using 'awk' to reduce this process time.
cat $filename | while read LINE do #read 115 to 125 and then remove trailing spaces and leading zeroes vid=`echo "$LINE" | cut -c 115-125 | sed 's,^ *,,; s, *$,,' | sed 's/^*//'` exist=0 #match vid with entire line in id.txt exist=`grep -x "$vid" $file_dir/id.txt | wc -l` if [[ $exist -gt 0 ]]; then echo "$LINE" >> $dest_dir/id.txt fi done