2

A script generates log files with tens of kilolines, and most lines are very long. It would be great to make this log human-friendly with a few edits.

At run time, there are helpful environment variables. Say, ${X} corresponds to /usr/projects/draco/vendors/spack.tt.developmental/spack.tt.2017-12-06.with-flags/opt/spack/cray-CNL-haswell/intel-17.0.4/gsl-2.4-zagfxq3p5iorh7viw4gc24bwzlx2tyk7

My goal is to use the environment variables to find lines like

Leaving directory /usr/projects/draco/vendors/spack.tt.developmental/spack.tt.2017-12-06.with-flags/opt/spack/cray-CNL-haswell/intel-17.0.4/gsl-2.4-zagfxq3p5iorh7viw4gc24bwzlx2tyk7

and supplant them with a convenient placeholder, here gsl-install

Leaving directory gsl-install

My efforts include an unsuccessful syntax like this

sed -i -e 's/${X}/gsl-install/g' tedious.log

How can I provide sed an environment variable and have corresponding strings replaced?


$ uname -a
Darwin ***.***.*** 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
  • Where did that gsl-install come from? I don't see it in the origin string. – igal Dec 7 '17 at 0:42
  • Not sure where you are getting gsl-install from, but you could quickly convert it to Leaving directory gsl-2.4-zagfxq3p5iorh7viw4gc24bwzlx2tyk7 – Tigger Dec 7 '17 at 0:42
  • @Tigger: Great observation. There is an issue left out of the problem statement that precludes using your insight. I want to get rid off the darn hashes to allow me to use diff across these log files. – dantopa Dec 7 '17 at 0:54
3

You have two problems

  • your environment variable contains '/' characters
  • you are containing your sed argument in single quotes so ${X} is passed unsubstituted

You can use any character as a sed command delimiter. Choose something not likely to be in your search string. And use double quotes around your sed command..
For example:

sed -i -e "s;${X};gsl-install;g" tedious.log
  • I have seen ; used as the delimiter as file names can still contain _ might be something to add to the answer. – thebtm Dec 7 '17 at 1:02
  • @PiedPiper: great answer and explanation. Works as promised. – dantopa Dec 7 '17 at 4:52
1

To use bash variables in a sed script, you need to break out of single quote, add the variable inside double quotes and go back into the single quote for sed.

Example below:

DAY1=$(date -d "+1days" +%m%d%Y)
DAY2=$(date -d "+2days" +%m%d%Y)
DAY3=$(date -d "+3days" +%m%d%Y)
sed -n -E -e '/\s\s\s*'"($DAY1|$DAY2|$DAY3)"'/p' $file
#or
sed -n -E -e ';\s\s\s*'"($DAY1|$DAY2|$DAY3)"';p' $file
  • 2
    . . . but see What characters do I need to escape when using sed in a sh script? - especially since the variable in the OP's case includes forward slash path delimiters – steeldriver Dec 7 '17 at 0:51
  • 1
    I was trying to point the in the right direction. – thebtm Dec 7 '17 at 0:55
  • @steeldriver: Yes. Those forward slash delimiters are the crux of the problem. The post is basically: use an environment variable for a path to edit a log file. – dantopa Dec 7 '17 at 0:56
  • 1
    @thebtm: Good post. It adds to my knowledge. Yet it is a solution for a slightly different problem. – dantopa Dec 7 '17 at 0:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.