Removal of extra spaces but not the newline from csv files

Question

I have to compare 2 files by using comm command like the below

comm -23 csv1 csv2 > csv3

I seem to face the problem that both the csv are generated from different sources (csv1 is extracted from another csv containing many columns and first column is extracted from it and csv 2 is generated from DB and consists of some extra spaces)

csv2 contains data like below

Vp1234▒▒▒▒
vp4567▒▒▒▒
vp6789-v2▒▒▒

(where ▒ indicates a space). While csv1 contains similar data (which is why I have to compare to find the data only existing in csv1) but without spaces.

Hence its difficult to compare both.

I have tried sed but it didn't work or tried wrongly.

Answer 1

If the only issue are the spaces, you can try using diff instead of comm because it has the -w flag:

   -w, --ignore-all-space
          ignore all white space

You can then parse the output to mimic comm -23. For example:

diff -w  csv1 csv2 | grep -Po "^< \K.*" > csv3

Alternatively, you can remove all leading and trailing spaces from your file:

sed -i 's/ *$//; s/^ *//' csv2