I have 2 csv files whose contents are-





and impo1.csv:


Basically expo1.csv is a file on the server, and impo1.csv is a file I created to update expo1.csv. A script makes the changes in expo1.csv as specified in impo1.csv after performing slight processing in the impo1 data (eg. The line 102,p2,yes from impo1.csv is processed and then an update is made to expo1.csv - p2_1,yes.)

expo1.csv after changes:




Now after the script makes the changes, we need to validate if the changes are done properly by comparing the impo1 and expo1 files. This is where i'm stuck.

So far I could isolate the data between the commas in impo1.csv separately into variables using awk:

Sno=104 102 104
Posw=p1 p2 p10
cho=no yes no

Now the question is, how do I check this? The impo1.csv files contains around 3000 updates. If I grep p1 expo1.csv|grep no expo1.csv, obviously it will not return the correct result as the file has many 'no' strings. I have tried using a for loop to separate the data using awk into separate variables and then grep using a wildcard - grep sno expo1.csv|grep '/<$posw.*,$cho>/' expo1.csv - but it doesn't work.

Using GNU bash 4.1.2.

EDIT - Should have mentioned this earlier, my bad - There are no clear patterns in the impo1.csv file which I can use to check the expo1 file. I have made corrections to the sample file contents which illustrate my point.

2 Answers 2


The solution is rather simple. You just need to create a pattern from each line of impo1.csv and then grep it from expo1.csv after updated

validate() {
    # $1 ~ impo1.csv
    # $2 ~ expo1.csv after changes
    while read pattern; do
        grep -q "^$pattern" $2 || return 1
    done < <(sed "s/,/,.*/" $1 )
  • Thanks, but I wasn't clear enough in my explanation, sorry. Getting patterns from the impo1.csv file is very difficult. I have added the necessary explanation to the question. Apr 25, 2017 at 11:00
  • @ShashankKR Does each line of impo1.csv contains only one update?
    – linerd
    Apr 25, 2017 at 11:09
  • Yes@linerd one line contains one update. Apr 25, 2017 at 11:12
  • Thanks @linerd, I've decided to use a while loop to do this, but instead of using read pattern i'm using read line. Apr 26, 2017 at 4:52
awk -F, '
            DATA[$1 SUBSEP $++i] = $++i;
    DATA[$1 SUBSEP $2] != $3
    ' expo1.csv impo1.csv

will print lines in impo1.csv which are different to data in expo1.csv

  • Note this shows in LQP review queue, probably because of the lack of explanation.
    – fedorqui
    Apr 25, 2017 at 10:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .