I have file "report.csv" . This report is generated based on few parameters
which can range from 1-10. So the report.csv has all the parameters printed in
it first and then , the main header and the data. Now I need to process
this file with the header and data and exclude all the parameters. The file looks like this:

para_1: abc
para_2: def
para_3: ghi
para_n: xyz

Customer_Name, Customer_Number, .....
Jon Doe,19923
Adam Plak,19985

I used below command to get the position of the Column header

This gives me the position

grep -n "Customer_Name" report.csv | cut -d : -f1

(let's say) 13. Then I use this command to copy to another file and use the new file

sed '1,13d' report.csv >> report_test.csv

however i need to make it dynamic , like

$var1 = grep -n "Customer_Name" report.csv | cut -d : -f1
$var2 = d
sed '1,$var1$var2' report.csv >> report_test.csv

can you experts please help achieve this in Shell Script?

  • Actually, why can't you do everything in sed like sed '1,/Customer_Name/' report.csv >> report_test.csv? – Philippos Jun 24 '19 at 8:49
var1=`grep -n "Customer_Name" report.csv | cut -d : -f1`
sed "1,${var1}${var2}"  report.csv  >> report_test.csv


var1=$(grep -n "Customer_Name" report.csv | cut -d : -f1)
sed "1,${var1}${var2}"  report.csv  >> report_test.csv


What we needed here is command substitution

The command substitution can be done in two ways one is using

  • $(…)
  • `…`

The purpose of command substitution is to evaluate the command which is placed inside the backtick and provide its result as an argument to the actual command.

Difference between the two command substitution methods

The old-style backquotes ` ` treat backslashes and nesting a bit different. The new-style $() interprets everything in between ( ) as a command. The new-style $() applies to all POSIX-conformant shells.

These are good reads to understand command substitution better:

  • when i Do this var1=$(grep -n "Customer Invoice Delivery" b | cut -d : -f1) i get below error cm01-dev:@/tmp>sh test_sh.sh test_sh.sh: syntax error at line 1: `var1=$' unexpected content of test_sh.sh is var1=$(grep -n "Customer_Name" b | cut -d : -f1) echo $var1 var2='d' echo $var2 sed "1,${var1}${var2}" report.csv >> report_test.csv – Praveen Jun 24 '19 at 6:05
  • Can you try again, in my case file name was b , I have edited back my answer with your file name report.csv ? So it should be var1=$(grep -n "Customer_Name" report.csv | cut -d : -f1) – Arushix Jun 24 '19 at 6:11
  • it still gives the same error test_sh.sh: syntax error at line 2: `var1=$' unexpected – Praveen Jun 24 '19 at 6:31
  • problem is assignment of the value of grep -n "Customer_Name" report.csv | cut -d : -f1 in variable var1 – Praveen Jun 24 '19 at 6:38
  • you can try var1=`grep -n "Customer_Name" report.csv | cut -d : -f1` Use ` sign to enclose the statement – Arushix Jun 24 '19 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.