Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a CSV file (say temp.csv) with strings and numbers. I need to delete the lines which contain a particular pattern of string at the beginning of the lines in the file. For example, here is my file -

req1,incl_patt1,excl_patt1,2,ind1
req1,incl_patt2,excl_patt2,1,ind1
req1,incl_patt3,excl_patt3,4,ind1
req2,inc_patt1,exc_patt1,1,ind2
req2,inc_patt2,exc_patt2,2,ind2
req2,inc_patt3,exc_patt3,3,ind2
req3,pattern3,expatt3,1,ind3
req4,pattern4,expatt4,1,ind4

I am writing a pattern to req_file_curr in a command ~ $ req_file_curr=req1

echo "${req_file_curr}" gives output as req1

However when using it in a sed command, like below:

sed '/\"${req_file_curr}\"/d' temp.csv

gives output as

req1,incl_patt1,excl_patt1,2,ind1
req1,incl_patt2,excl_patt2,1,ind1
req1,incl_patt3,excl_patt3,4,ind1
req2,inc_patt1,exc_patt1,1,ind2
req2,inc_patt2,exc_patt2,2,ind2
req2,inc_patt3,exc_patt3,3,ind2
req3,pattern3,expatt3,1,ind3
req4,pattern4,expatt4,1,ind4

I am expeceting the output as

req2,inc_patt1,exc_patt1,1,ind2
req2,inc_patt2,exc_patt2,2,ind2
req2,inc_patt3,exc_patt3,3,ind2
req3,pattern3,expatt3,1,ind3
req4,pattern4,expatt4,1,ind4

How can I use the variable from a shell as a pattern to look for in sed or awk?

share|improve this question

3 Answers 3

up vote 4 down vote accepted

You need to enclose the sed pattern in double quotes so that the shell can expand the variable before passing it to sed:

sed "/^${req_file}/d" temp.csv

You also need to use the ^ anchor to indicate to sed that this pattern needs to occur at the beginning of the line.

share|improve this answer
    
+1 and Thank you for explaining the use of ^ anchor. –  Dhruuv May 12 at 19:37

No need for awk or sed here:

grep -v "^$req_file_curr," file

Note that (like for sed or awk though to a lesser extent), that assumes that $req_file_curr does not contain regexp operators (like ., *...).

If it does, you'd have to escape those characters.

share|improve this answer
    
I tried to append .req to the contents of first columns to check if sed or awk works, it actually works fine with the code provided by @joseph. req_file_curr=req1.req. Using the sed command as provided above sed "/^${req_file}/d" temp.csv gives exactly what I need. Am I missing anything here? –  Dhruuv May 13 at 16:36
    
@Dhruuv, . matches any character, so req1.req would match req1.req but also req12req or req1,req. –  Stéphane Chazelas May 13 at 17:05
    
I do have . in the first column of my file. I am leaning more towards sed because I can use inplace edit option sed -i (which I learned recently) instead of streaming the output into a file and then renaming that file to the original file name. Is there a way to handle the regexp operators while using sed? Otherwise, I will have to go ahead with grep –  Dhruuv May 13 at 17:55
1  
@Dhruuv See unix.stackexchange.com/a/129063/22565 (escape as per the LHS) –  Stéphane Chazelas May 13 at 19:14

You have set req_file_curr but use req_file. And your sed command won't work, since shell variable can not be expanded in your sed command.

You can try this:

$ sed '/'"${req_file_curr}"'/d' file
req2,inc_patt1,exc_patt1,1,ind2
req2,inc_patt2,exc_patt2,2,ind2
req2,inc_patt3,exc_patt3,3,ind2
req3,pattern3,expatt3,1,ind3
req4,pattern4,expatt4,1,ind4

or using sed with double quote:

sed "/${req_file_curr}/d" file

and using awk:

$ awk -v req=$req_file_curr '$0 !~ req' file
req2,inc_patt1,exc_patt1,1,ind2
req2,inc_patt2,exc_patt2,2,ind2
req2,inc_patt3,exc_patt3,3,ind2
req3,pattern3,expatt3,1,ind3
req4,pattern4,expatt4,1,ind4
share|improve this answer
    
Thank you for pointing the mistake in my question, I edited my question. –  Dhruuv May 12 at 19:32
    
Thanks @StephaneChazelas, fixed. –  cuonglm May 12 at 20:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.