I have following file:

 $less dummyKeyAndValue.txt

I want the output as the following :

 $less dummyNewFile.txt
   apiKey=key1 term=abc
   apiKey=key2 term=def
   apiKey=key1 term=pqr
   apiKey=key3 term=def

Mainly,I want to extract 'apiKey' and 'term' from dummyKeyAndValue.txt file,they both can appear in different order in file.I tried following command :

   $cat dummyKeyAndValue.txt | tee >(egrep -o 'apiKey=[a-zA-Z0-9]+')  |   
   egrep -o 'term=[a-zA-Z]+' | less

I get output as :


Can someone help me with the command for getting desired output?


As an alternative a very efficient but slightly more complicated solution

sed 'G;s/;/\n/' | awk -F= '
$1~/apiKey/ {key=$2}
$1~/term/ {term=$2}
/^$/ {printf("  apiKey=%s term=%s\n", key, term)

First sed is used to do two things: The "G" command will effectively add an open line after every "record set", and secondly the "substitute" command (s/;/\n/) will effectively expand each record set to be one-per-line by replacing every ; with a new-line character. What comes out of sed is they key-value pairs one per line, with an open line designating the end of each record.

Then awk needs only to look at the first field to find the attributes you are interested in, and the second field for the value, which eliminates the need for index and substr. Once awk encounters an "open line" it prints the values that it has found. For resilience' sake you can "clear" the values at the end of each record. Note the use of -F= to instruct awk to split the line into fields based on the =-sign.

$1 ~ /.../ means "When the first field matches the value /.../

It then assigns a value to a variable (key or term)

The /^$/ means "when awk encounters an open line"


This awk based solution can help because it is easier to to read/maintain. awk is often the preferred tool for parsing column-like values from a text file.

/tmp$ cat a.awk
   keypart=substr($0, index($0, "apiKey=")+7)
   keyvalue=substr(keypart, 1, index(keypart, ";")-1)

   termpart=substr($0, index($0, "term=")+5)
   termvalue=substr(termpart, 1, index(termpart, ";")-1)

# If the attribute is last on the input line there will be no ; to mark the end so use the whole part
   if(keyvalue=="") {keyvalue=keypart}
   if(termvalue=="") {termvalue=termpart}
   printf ("  apikey=%s term=%s\n", keyvalue, termvalue)

The Awk script (named a.awk above but any file name the makse sense can be used) can be used like this:

awk -f a.awk inputfile

As you can see I handle the case of an input field that ends at the end of the line specially with an if statement for each. I would enhance this script as follow to automatically handled those cases:

/tmp$ cat a.awk  
   LINE=$0 ";"

   keypart=substr(LINE, index(LINE, "apiKey=")+7)
   keyvalue=substr(keypart, 1, index(keypart, ";")-1)

   termpart=substr(LINE, index(LINE, "term=")+5)
   termvalue=substr(termpart, 1, index(termpart, ";")-1)

   printf ("  apikey=%s term=%s\n", keyvalue, termvalue)

The benefit of this becomes clearer when you add more cases!


It's probably not very efficient, but if you want to pursue your 'two greps' approach, you could do it using paste:

$ paste <(grep -o 'apiKey=[^;]*' dummyKeyAndValue.txt) <(grep -o 'term=[^;]*' dummyKeyAndValue.txt)
apiKey=key1     term=abc
apiKey=key2     term=def
apiKey=key1     term=pqr
apiKey=key3     term=def

Or the KISS approach, using GNU sed:

sed -nE -e 's/(apiKey=[^;]*).*(term=[^;]*)/\1 \2/p' \
  -e 's/(term=[^;]*).*(apiKey=[^;]*)/\2 \1/p' dummyKeyAndValue.txt
  • It is working.How it is inefficient? I mean in which cases it will fail? – Gunjan Aswani Jan 2 '17 at 13:38
  • @GunjanAswani I was just referring to the fact that it processes the file twice – steeldriver Jan 2 '17 at 14:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.