0

Here is a sample of my input data:

$9.99,Titan the Power,www.example.com,565654
$15.99,Conan The Barbarian,www.sure.com,565438
$1.99,Julia Cesar,www.bfg.com,69722

I have written this code so that I create an array from the input file, and then isolate the title with the $f2 variable.

#!/bin/bash
input="/home/test/Documents/Scripts/test.csv"

readarray myarray < $input    

# Sends all of arrays into while loop which read each line by line
echo "${myarray[@]}" | while IFS=',' read -r f1 f2 f3 f4
do
  # echo field 2 of each line
  echo $f2 #This displays the title of each product (just for testing)
done

Now I would like to compare each title ($f2) with another file ($csv2) to see if there are any positive matches.

csv2:

$1.99,The Power of Now,www.dvd.com,45674
$9.99,Titan the Power,www.otherwebsite.com,13357
$2.99,The incredible Hulk,www.purchase.com,13956

I know that I can compare files with something like this:

if [ "$f2" == "$csv2" ]; then
  echo "match"
fi

The above code matches the whole thing, and lines in csv2 are likely to be in a different order and contain other stuff which I'm not interested in. I want the script to inform me of only the lines from $f2 which have matching titles in csv2. So the output might look something like this if only the first title was present in csv2:

Matching lines:

$9.99,Titan the Power,www.otherwebsite.com,13357
$9.99,Titan the Power,www.example.com,565654

I would like the original line and the matching line to be displayed as the output so I can compare them, like above (notice other field values are slightly different between $input and $csv2 but title is the same).

7
  • 2
    Unless you're doing this for strictly pedagogical reasons, I'd suggest using awk/perl/python (or a standard utility such as join) for this - rather than pure shell programming May 12, 2016 at 12:21
  • He's mentioned that the lines won't necessarily be in the same order, which means join might miss some matches unless he sorts both files first, adding complexity. What's wrong with shell script? This can be done with relatively simple grep.
    – smokes2345
    May 12, 2016 at 12:50
  • csv2 data added
    – neilH
    May 12, 2016 at 13:32
  • This kind of question has been asked and answered so many times (I don't have the time to search for the duplicates but there should be about a dozen of them)... Here's a very simple awk one liner that should do: awk -F, 'NR==FNR{x[$2]=$0;next};$2 in x{print;print x[$2]}' csv1 csv2 May 12, 2016 at 13:57
  • good solution don, that is working
    – neilH
    May 12, 2016 at 15:57

1 Answer 1

0

I would get all the titles from your first file with something like

interesting_titles=$(cat $input |cut -d, -f2)

then use that to grep your second file for those titles

grep -F "$interesting_titles" $csv2

Anything returned by grep would be a match.

You can shorten this to a one-liner with

grep -F "$(cat $input |cut -d, -f2)" $csv2

If you want the output from both files side-by-side you probably want a for loop, like....

cat $input |cut -d, -f2 | while read t; do
  grep -i "$t" $csv2
  if [ $? -eq 0 ];then
    grep -i "$t" $input
  fi
done

This will loop over each line in $input, check for and print that record in $csv2 in $input

7
  • Thanks smokes2345, that's working nicely, I just need one more thing which I forgot, and I hope is an easy amendment- I need the original line from $input and the matching title line form $csv2 to be displayed as the output- I've edited my question.
    – neilH
    May 12, 2016 at 13:11
  • Hi smokes, i've run the script and the output is longer than expected and contains some of the data which doesn't match, this is the output:
    – neilH
    May 12, 2016 at 14:34
  • $9.99,Titan the Power,www.otherwebsite.com,13357 $9.99,Titan the Power,www.example.com,565654 $9.99,Titan the Power,www.otherwebsite.com,13357 $9.99,Titan the Power,www.example.com,565654 $1.99,The Power of Now,www.dvd.com,45674 $9.99,Titan the Power,www.otherwebsite.com,13357 $9.99,Titan the Power,www.example.com,565654 $1.99,The Power of Now,www.dvd.com,45674 $2.99,The incredible Hulk,www.purchase.com,13956 $15.99,Conan The Barbarian,www.sure.com,565438
    – neilH
    May 12, 2016 at 14:34
  • effing spaces, fixed the script
    – smokes2345
    May 13, 2016 at 2:37
  • brilliant, that's worked! Could you explain what the if [ $? -eq 0 ] bit does? cheers for your help
    – neilH
    May 13, 2016 at 9:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .