-1

How to delete from configuration file the variable/s hadoop.tasklog.totalLogFileSize that are not equal to 100

example

more file.conf

hadoop.tasklog.totalLogFileSize=107
hadoop.tasklog.totalLogFileSize=10
hadoop.tasklog.totalLogFileSize=100
hadoop.tasklog.totalLogFileSize=189
hadoop.tasklog.totalLogFileSize=104
hadoop.tasklog.purgeLogSplits=true
hadoop.tasklog.logsRetainHours=12
.
.

expected output

more file.conf

hadoop.tasklog.totalLogFileSize=100
hadoop.tasklog.purgeLogSplits=true
hadoop.tasklog.logsRetainHours=12
.
.
  • 1
    Why do you make config file changes using scripts to start with? – Kusalananda Aug 8 '18 at 8:50
  • we are working on log4j configuration , and some old conf comes with old values – yael Aug 8 '18 at 8:51
  • 2
    when asking question, please also add what you have tried yourself to solve this.. this is not a free coding service site.. you have asked and received lots of answers on text processing questions, by now you should be able to at least make an attempt to solve it.. – Sundeep Aug 8 '18 at 8:56
  • @yael - you're clearly working on Hadoop, is this vanilla apache, Hortonworks or Cloudera? There are better methods to manage these than editing the files with sed. – slm Aug 8 '18 at 11:24
1
$ awk -F= '/hadoop\.tasklog\.totalLogFileSize/ && $2 != 100 { next } 1' file
hadoop.tasklog.totalLogFileSize=100
hadoop.tasklog.purgeLogSplits=true
hadoop.tasklog.logsRetainHours=12

This treats the file as =-delimited fields. If a line matches the given pattern, and if the number after = (i.e. the second field) is not 100, then the line is skipped and not outputted. All other lines are outputted by the trailing 1 at the end of the program (which could be replaced by { print }).

Using sed:

$ sed '/hadoop\.tasklog\.totalLogFileSize/{ /=100$/!d; }' file
hadoop.tasklog.totalLogFileSize=100
hadoop.tasklog.purgeLogSplits=true
hadoop.tasklog.logsRetainHours=12

This outputs all lines, but when a line matches the given pattern, and then does not match =100 at the end of the line, it is deleted from the input.

  • this is very great idea to do it with awk , but actually I tag sed and perl one liner since we want the option "sed -i" or "perl -i" to edit the file , with awk we need to copy the file that created to the source file , and we want to avoid that ( +1 for your effort ) – yael Aug 8 '18 at 8:49
  • @yael See updated answer. – Kusalananda Aug 8 '18 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.