I have a sample text file called result.txt just to demonstrate:


but in real application, the content of the result.txt can have any characters line by line like:


and I want to delete a line of string via argument in a bash script. So, in my bash script called test.sh, I have the following code:

sed -i "/$del/d" $result
cat result.txt

Example 1

So to delete "{HEX}.A.Bs", I run the following script with an argument:

./test.sh "{HEX}.A.Bs"

this should delete a string called "{HEX}.A.Bs" inside result.txt but all the contents inside result.txt were deleted. The expected output should be:


Example 2

If I want to delete "{HEX}.A.Bss":

./test.sh "{HEX}.A.Bss"

The above example left only a string "{HEX}.A.Bs" inside result.txt. It should delete only the string "{HEX}.A.Bss" . So, the expected output from the this example 2 should yield this:


May I know what is wrong with the code here?


7 Answers 7


If you want to delete lines that match a string then use a tool that understands literal strings, e.g. with GNU awk for inplace editing (just like GNU sed):

del="$1" awk -i inplace '($0"") != ENVIRON["del"]' "$result"
  • 2
    Note that it assumes $result doesn't contain = characters. del="$1" awk -i inplace -e '$0 != ENVIRON["del"]' -E /dev/null "$result" to work around it. Jun 7, 2020 at 6:16
  • wrt $result containing =, right that's always the case and if you ever have that situation then the common, idiomatic fix is just to specify the full path, e.g. ./foo=bar instead of foo=bar - IMHO that doesn't have to be addressed in every answer. wrt $del containing a number - you're right, corrrected thanks.
    – Ed Morton
    Jun 7, 2020 at 14:21
  • That's a limitation still worth mentioning. It's not comp.lang.awk, not everybody here is aware of the limitations of the awk API. Here, since we're already assuming GNU awk, we might as well use that more reliable API. Prefixing strings with ./ is more cumbersome (like case $1 in ([!/]*=*) file=./$file;; esac) and then, you potentially need to strip it back in awk if you need FILENAME to expand to the string provided by the user. Jun 7, 2020 at 15:32
  • Understood, IMHO its just not useful to discus that in every awk answer or even in random awk answers that aren't about that topic.
    – Ed Morton
    Jun 7, 2020 at 16:39

With perl (that -i non-standard option supported by a few modern sed implementations was actually borrowed from perl):

LINE_TO_REMOVE='{HEX}.A.Bs' perl -i -nle '
  print if $_ ne $ENV{LINE_TO_REMOVE}' -- "$file"

To use sed (here assuming GNU sed or compatible), you'd need first to escape the regular expression operators in the string to match (that includes the . used in your example which unescaped would match any single character) and anchor it at beginning and end with ^ and $ respectively.

regexp='^'$(printf '%s\n' "$string" | sed 's:[][\\/.^$*]:\\&:g')'$'
sed -i -e "/$regexp/d" -- "$file"

(it also assumes $string and the file contain valid text).

  • FWIW for escaping metachars I find sed 's/[^^]/[&]/g; s/\^/\\^/g' easier to remember (put every char except ^ inside a bracket expression and escape the ^s) and it'll work whether the subsequent use is in sed or some other tool, with almost all possible sed delimiters, and whether it's being used in a BRE or ERE context.
    – Ed Morton
    Jun 7, 2020 at 14:34
  • @EdMorton, note that it won't work if that other tool is perl (or any tool working with PCREs) or most awk implementations where [\][x] doesn't match on \x, but on one of \[]x characters. Jun 7, 2020 at 15:21
  • Yup, you're right about awk (or any other tool that allows [\]] to mean a literal ] within a bracket expression). Next time I have to use this I'll have to rethink how to write that sed expression, thanks.
    – Ed Morton
    Jun 7, 2020 at 16:28
  • I finally got around to thinking about this as I had a problem to solve that required it and I think sed 's/[^^\\]/[&]/g; s/\^/\\^/g; s/\\/\\\\/g will output a regexp that works (i.e. causes every char to be treated literally) in any version of sed, grep, awk, or perl. Do you know of any case where that would fail with those tools?
    – Ed Morton
    Aug 15, 2020 at 13:01
  • 1
    That should work, though there's always the problem of locales that use GBK, BIG5, BIG5-HKSCS, GB18030 multibyte charsets where some multibyte characters contain bytes that also are the encoding of [, ] or backslash. Then you'll have trouble if the escaping is done by a multi-byte aware sed but the result is used by a tool that is not, or the other way round. There can't be a solution that works for all there though. Aug 15, 2020 at 15:27

What's wrong with your code is that you're successfully matching the substring {HEX}.A.Bss on every line.

Assuming your string contains no characters that need escaping, you could instead use

sed "/^$string\$/d" filename

I.e., anchor the pattern at both the start and end of the line.

Since the pattern does contain a character that needs escaping (., which matches a single character), it would be safer to use grep -F, or awk with a string comparison.

If you still want to use sed, make sure that all dots in the string are matched as dot and nothing else:

pattern=$( printf '%s\n' "$string" | sed 's/[.]/[.]/g' )
sed "/^$pattern\$/d" filename

To delete lines matching a string exactly, use

grep -v -xF -e "$string" filename >newfile && mv newfile filename

The options used with grep here are

  • -v: Negates the sense of the test, i.e. all lines not matching the pattern are extracted.
  • -x: Require that the whole line matches the given pattern.
  • -F: Treat the pattern as a string rather than as a regular expression.
  • -e: "The next argument is the pattern". This is needed in case the value $string starts with a dash.

To retain as much meta data of the original file as possible (e.g. ownership, permissions etc.), make a copy of it, apply the grep operation to the copy, and redirect the result to the original file:

cp filename filename.tmp &&
grep -v -xF -e "$string" filename.tmp >filename
rm -f filename.tmp
  • grep -v -xF -e "$string" filename >newfile && mv newfile filename <- using this one, it doesn't remove the last entry in result.txt
    – Kalib Zen
    Jun 7, 2020 at 0:17

Assuming your result.txt file content is exactly as provided by you, and your search strings are as per your examples...

$ cat test.sh
sed '/^'"$del"'$/d' "$result"

$ ./test.sh '{HEX}.A.Bs'
$ ./test.sh '{HEX}.A.Bss'
$ ./test.sh '{HEX}.A.Bsss'
$ ./test.sh '{HEX}.A.Bssss'

I believe \b is for word boundry , and just in case if the line contains "{HEX}.A.Bs anotherword" or "{HEX}.A.Bs filename" ( which is string contains space and "filename" as a filename ) , it will also delete that.

so if you want to delete only specific "{HEX}.A.Bs" its better to use:

$ sed '|^'"$del"'$|d' $result  

"$del" contains the string exactly you passed and single quotes ' in between them tell to match it exactly as it is and not as a pattern.


perl -pe "s|^\Q$del\E$||g;" $result | sed '/^$/d'   

\Q => quote (disable) pattern metacharacters until \E

and if you want in-place edit ( change/edit the file directly ) you could use :

$ perl -pe "s|^\Q$del\E$||g;" $result | sed '/^$/d' > tmpfile && mv tmpfile $result  

And if you want to use awk :

$ awk -v d=$del  '$0 != d' $result > tmp && mv tmp $result

If you want to use grep:

$ grep -vxF "$del" $result > tmp && mv tmp $result

-F, --fixed-strings, --fixed-regexp Interpret PATTERN as a list of fixed strings
-v, --invert-match Invert the sense of matching, to select non-matching lines. (-v is specified by POSIX.)
-x, --line-regexp Select only those matches that exactly match the whole line. (-x is specified by POSIX.)

  • That would delete a line line {HEX}BARB.
    – Ed Morton
    Jun 6, 2020 at 18:20
  • @EdMorton only if he use -E or regex in sed to escape chars Jun 6, 2020 at 18:23
  • if he used -E things would get worse as then {..} would be interpreted as a RE interval. I'm referring to the .s in .A.B being RE metachars meaning "any character".
    – Ed Morton
    Jun 6, 2020 at 18:25
  • @EdMorton : I got it ... now updated the answer...it should work for him Jun 6, 2020 at 19:01
  • 1st example on using $ sed '/^'"$del"'$/d' $result , if I have the following in result.txt -> /var/log/ and /backup/ , then when I run ./test.sh /var/log, I got the following error: sed: -e expression #1, char 12: expected newer version of sed . and if I run ./test.sh /backup/, I got this error: sed: can't find label for jump to ackup/$/d' . and nothing is deleted. || 2nd example on using , perl -pe "s/^\Q$del\E$//g;" $result | sed '/^$/d' ` I got the following error, `Backslash found where operator expected at -e line 1, near "s/^\Q/var/log\" syntax error at -e line 1, near "s...
    – Kalib Zen
    Jun 7, 2020 at 6:38

Here is one way if you don't want to be bothered with all this escaping business:

$ printf '%s\n' "$del" |
  sed -Ene '1h;G;/^(.*)\n\1$/!P' \
- "$result" > tmp && mv tmp "$result"

Assuming tmp file is not the same as result file. And no newline(s) in del variable. Gnu sed is being used.

We have essentially transferred the del variable in the hold and then we make a strict check on the current line with this hold content. For no match we print. Repeat for the whole file.


Sorry, it seems like I got this:

sed -i "/^$del\b/d" $result
  • 3
    Try that if/when someone calls your script as ./test.sh '.*' to make sure you're happy with the results. I'd remove the -i from your sed command line first though.
    – Ed Morton
    Jun 6, 2020 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.