3

We want to add the following properties:

-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M

in the two first lines that contain the word - HBASE_MASTER_OPTS

Example before change:

more ams-hbase-env-template.txt

export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly"    
export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=71 -XX:+UseCMSInitiatingOccupancyOnly"
export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=72 -XX:+UseCMSInitiatingOccupancyOnly"
export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=73 -XX:+UseCMSInitiatingOccupancyOnly"

Example after change (expected results):

export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M"    
export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=71 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M"
export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=72 -XX:+UseCMSInitiatingOccupancyOnly"
export HBASE_MASTER_OPTS=" -XX:CMSInitiatingOccupancyFraction=73 -XX:+UseCMSInitiatingOccupancyOnly"

What I did until now is that:

sed -i '/HBASE_MASTER_OPTS/ s/$/-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M/' ams-hbase-env-template.txt

But this isn't right because:

  1. We want the properties before the end of double quote.

  2. Properties must be only on the first two lines that variable is HBASE_MASTER_OPTS.

  3. In case properties are already in line, then it will not append again on any next time!

Improve this question
3
  • 1
    Your point 3 is not fully clear to me: Do you mean that if a line contains HBASE_MASTER_OPTS, but also already contains the properties, you do not want to add them again? And, if such a line is encountered, would it still count as being one of the first two lines? – Dirk Herrmann Aug 13 '18 at 22:41
  • 1
    regarding to Your point 3 is not fully clear to me: Do you mean that if a line contains HBASE_MASTER_OPTS, but also already contains the properties, you do not want to add them again - YES I MEAN THAT – yael Aug 13 '18 at 22:49
  • @yael Can you update your example (before + after) so that your point 3 is covered? It would be helpful if your example would reflect all the points that you need on what you're asking... Thanks! – filbranden Aug 15 '18 at 5:46
2

Extending on Steve's answer:

awk '/HBASE_MASTER_OPTS/&&/ -XX:[+]UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M/{x++;print;next}/HBASE_MASTER_OPTS/&&x<2{gsub(/"$/," -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M\"");x++}1' ams-hbase-env-template.txt

Update: Here's a more maintenance friendly version, where every string only occurs once.

awk 'BEGIN{p="-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M"}{m=0}/HBASE_MASTER_OPTS/{m=1;++x}m&&index($0,p){print;next}m&&x<=2{gsub(/"$/," " p "\"")}1' ams-hbase-env-template.txt

And some explanation:

  • In the first version in the pattern matching, [+] is used for the literal + character, because otherwise + would be interpreted as special character.

The remaining points are about the second version:

  • In the BEGIN clause, the properties string p is defined. All the following clauses apply to the lines of the input.
  • {m=0}: This clause has no condition, which means it applies to every line, such that for each line first m (for match) will be set to zero.
  • /HBASE_MASTER_OPTS/{m=1;++x}: If a line matches the HBASE_MASTER_OPTS string, m is set and our line counter is incremented.
  • m&&index($0,p){print;next}: this clause checks if, in addition to a match indicated by m, also the properties string is contained in the input line. In such a case index will return an non-zero value. If so, we print the line as it is (not adding the properties again) and with next skip the remaining clauses and start processing the next line. Note that the following clauses were originally provided by Steve and only adapted by me:
  • m&&x<=2{gsub(/"$/," " p "\"")}: if we reach this point and we have a matching line and our counter has not exceeded 2, we modify $0 (the input line) by prepending the terminal " with a blank and our properties string. Apart from that, no output is produced, which will be done by the next clause.
  • 1: This is a condition (always true) without an action, which means the action defaults to printing the input line.
Improve this answer
3
  • it still append the properties even if they are exist in the lines – yael Aug 13 '18 at 22:56
  • ok I will wait for the fix – yael Aug 13 '18 at 23:03
  • what to say , wonderful really , – yael Aug 13 '18 at 23:23
3

How about this awk solution. Creates the desired output in the question.

awk '/HBASE_MASTER_OPTS/&&x<2{gsub(/"$/," -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M\"");x++}1' ams-hbase-env-template.txt

Try it online!

Improve this answer
2
  • one problem is that when we run your CLI , let say we put the output to out.txt , on the next time awk will append the lines , so actually we get duplicate properties – yael Aug 13 '18 at 22:48
  • so can you fix the awk syntax , so on the next awk running it will not append the properties because they are already exists – yael Aug 13 '18 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.