1

I have a sample text file called result.txt just to demonstrate:

{HEX}.A.Bs
{HEX}.A.Bss
{HEX}.A.Bsss
{HEX}.A.Bssss

but in real application, the content of the result.txt can have any characters line by line like:

/usr/local/abc.txt
/var/tmp/
png
OIPOP()_+(_)&*#)@IOJDNU*@Utest

and I want to delete a line of string via argument in a bash script. So, in my bash script called test.sh, I have the following code:

!/bin/sh
#test.sh
result=result.txt
del="$1"
sed -i "/$del/d" $result
cat result.txt

Example 1

So to delete "{HEX}.A.Bs", I run the following script with an argument:

./test.sh "{HEX}.A.Bs"

this should delete a string called "{HEX}.A.Bs" inside result.txt but all the contents inside result.txt were deleted. The expected output should be:

{HEX}.A.Bss
{HEX}.A.Bsss
{HEX}.A.Bssss

Example 2

If I want to delete "{HEX}.A.Bss":

./test.sh "{HEX}.A.Bss"

The above example left only a string "{HEX}.A.Bs" inside result.txt. It should delete only the string "{HEX}.A.Bss" . So, the expected output from the this example 2 should yield this:

{HEX}.A.Bs
{HEX}.A.Bsss
{HEX}.A.Bssss

May I know what is wrong with the code here?

3

If you want to delete lines that match a string then use a tool that understands literal strings, e.g. with GNU awk for inplace editing (just like GNU sed):

del="$1" awk -i inplace '($0"") != ENVIRON["del"]' "$result"
| improve this answer | |
  • 2
    Note that it assumes $result doesn't contain = characters. del="$1" awk -i inplace -e '$0 != ENVIRON["del"]' -E /dev/null "$result" to work around it. – Stéphane Chazelas Jun 7 at 6:16
  • wrt $result containing =, right that's always the case and if you ever have that situation then the common, idiomatic fix is just to specify the full path, e.g. ./foo=bar instead of foo=bar - IMHO that doesn't have to be addressed in every answer. wrt $del containing a number - you're right, corrrected thanks. – Ed Morton Jun 7 at 14:21
  • That's a limitation still worth mentioning. It's not comp.lang.awk, not everybody here is aware of the limitations of the awk API. Here, since we're already assuming GNU awk, we might as well use that more reliable API. Prefixing strings with ./ is more cumbersome (like case $1 in ([!/]*=*) file=./$file;; esac) and then, you potentially need to strip it back in awk if you need FILENAME to expand to the string provided by the user. – Stéphane Chazelas Jun 7 at 15:32
  • Understood, IMHO its just not useful to discus that in every awk answer or even in random awk answers that aren't about that topic. – Ed Morton Jun 7 at 16:39
2

With perl (that -i non-standard option supported by a few modern sed implementations was actually borrowed from perl):

LINE_TO_REMOVE='{HEX}.A.Bs' perl -i -nle '
  print if $_ ne $ENV{LINE_TO_REMOVE}' -- "$file"

To use sed (here assuming GNU sed or compatible), you'd need first to escape the regular expression operators in the string to match (that includes the . used in your example which unescaped would match any single character) and anchor it at beginning and end with ^ and $ respectively.

string='{HEX}.A.Bs'
regexp='^'$(printf '%s\n' "$string" | sed 's:[][\\/.^$*]:\\&:g')'$'
sed -i -e "/$regexp/d" -- "$file"

(it also assumes $string and the file contain valid text).

| improve this answer | |
  • FWIW for escaping metachars I find sed 's/[^^]/[&]/g; s/\^/\\^/g' easier to remember (put every char except ^ inside a bracket expression and escape the ^s) and it'll work whether the subsequent use is in sed or some other tool, with almost all possible sed delimiters, and whether it's being used in a BRE or ERE context. – Ed Morton Jun 7 at 14:34
  • @EdMorton, note that it won't work if that other tool is perl (or any tool working with PCREs) or most awk implementations where [\][x] doesn't match on \x, but on one of \[]x characters. – Stéphane Chazelas Jun 7 at 15:21
  • Yup, you're right about awk (or any other tool that allows [\]] to mean a literal ] within a bracket expression). Next time I have to use this I'll have to rethink how to write that sed expression, thanks. – Ed Morton Jun 7 at 16:28
2

What's wrong with your code is that you're successfully matching the substring {HEX}.A.Bss on every line.

Assuming your string contains no characters that need escaping, you could instead use

sed "/^$string\$/d" filename

I.e., anchor the pattern at both the start and end of the line.

Since the pattern does contain a character that needs escaping (., which matches a single character), it would be safer to use grep -F, or awk with a string comparison.

If you still want to use sed, make sure that all dots in the string are matched as dot and nothing else:

pattern=$( printf '%s\n' "$string" | sed 's/[.]/[.]/g' )
sed "/^$pattern\$/d" filename

To delete lines matching a string exactly, use

grep -v -xF -e "$string" filename >newfile && mv newfile filename

The options used with grep here are

  • -v: Negates the sense of the test, i.e. all lines not matching the pattern are extracted.
  • -x: Require that the whole line matches the given pattern.
  • -F: Treat the pattern as a string rather than as a regular expression.
  • -e: "The next argument is the pattern". This is needed in case the value $string starts with a dash.

To retain as much meta data of the original file as possible (e.g. ownership, permissions etc.), make a copy of it, apply the grep operation to the copy, and redirect the result to the original file:

cp filename filename.tmp &&
grep -v -xF -e "$string" filename.tmp >filename
rm -f filename.tmp
| improve this answer | |
  • grep -v -xF -e "$string" filename >newfile && mv newfile filename <- using this one, it doesn't remove the last entry in result.txt – Kalib Hozany Jun 7 at 0:17
1

Assuming your result.txt file content is exactly as provided by you, and your search strings are as per your examples...

$ cat test.sh
#!/bin/sh
result="result.txt"
del="$1"
sed '/^'"$del"'$/d' "$result"
$

$ ./test.sh '{HEX}.A.Bs'
{HEX}.A.Bss
{HEX}.A.Bsss
{HEX}.A.Bssss
$ ./test.sh '{HEX}.A.Bss'
{HEX}.A.Bs
{HEX}.A.Bsss
{HEX}.A.Bssss
$ ./test.sh '{HEX}.A.Bsss'
{HEX}.A.Bs
{HEX}.A.Bss
{HEX}.A.Bssss
$ ./test.sh '{HEX}.A.Bssss'
{HEX}.A.Bs
{HEX}.A.Bss
{HEX}.A.Bsss
$
| improve this answer | |
1

I believe \b is for word boundry , and just in case if the line contains "{HEX}.A.Bs anotherword" or "{HEX}.A.Bs filename" ( which is string contains space and "filename" as a filename ) , it will also delete that.

so if you want to delete only specific "{HEX}.A.Bs" its better to use:

$ sed '|^'"$del"'$|d' $result  

"$del" contains the string exactly you passed and single quotes ' in between them tell to match it exactly as it is and not as a pattern.

Perl:

perl -pe "s|^\Q$del\E$||g;" $result | sed '/^$/d'   

\Q => quote (disable) pattern metacharacters until \E

and if you want in-place edit ( change/edit the file directly ) you could use :

$ perl -pe "s|^\Q$del\E$||g;" $result | sed '/^$/d' > tmpfile && mv tmpfile $result  

And if you want to use awk :

$ awk -v d=$del  '$0 != d' $result > tmp && mv tmp $result

If you want to use grep:

$ grep -vxF "$del" $result > tmp && mv tmp $result

-F, --fixed-strings, --fixed-regexp Interpret PATTERN as a list of fixed strings
-v, --invert-match Invert the sense of matching, to select non-matching lines. (-v is specified by POSIX.)
-x, --line-regexp Select only those matches that exactly match the whole line. (-x is specified by POSIX.)

| improve this answer | |
  • That would delete a line line {HEX}BARB. – Ed Morton Jun 6 at 18:20
  • @EdMorton only if he use -E or regex in sed to escape chars – Stalin Vignesh Kumar Jun 6 at 18:23
  • if he used -E things would get worse as then {..} would be interpreted as a RE interval. I'm referring to the .s in .A.B being RE metachars meaning "any character". – Ed Morton Jun 6 at 18:25
  • @EdMorton : I got it ... now updated the answer...it should work for him – Stalin Vignesh Kumar Jun 6 at 19:01
  • 1st example on using $ sed '/^'"$del"'$/d' $result , if I have the following in result.txt -> /var/log/ and /backup/ , then when I run ./test.sh /var/log, I got the following error: sed: -e expression #1, char 12: expected newer version of sed . and if I run ./test.sh /backup/, I got this error: sed: can't find label for jump to ackup/$/d' . and nothing is deleted. || 2nd example on using , perl -pe "s/^\Q$del\E$//g;" $result | sed '/^$/d' ` I got the following error, `Backslash found where operator expected at -e line 1, near "s/^\Q/var/log\" syntax error at -e line 1, near "s... – Kalib Hozany Jun 7 at 6:38
1

Here is one way if you don't want to be bothered with all this escaping business:

$ printf '%s\n' "$del" |
  sed -Ene '1h;G;/^(.*)\n\1$/!P' \
- "$result" > tmp && mv tmp "$result"

Assuming tmp file is not the same as result file. And no newline(s) in del variable. Gnu sed is being used.

We have essentially transferred the del variable in the hold and then we make a strict check on the current line with this hold content. For no match we print. Repeat for the whole file.

| improve this answer | |
0

Sorry, it seems like I got this:

sed -i "/^$del\b/d" $result
| improve this answer | |
  • 3
    Try that if/when someone calls your script as ./test.sh '.*' to make sure you're happy with the results. I'd remove the -i from your sed command line first though. – Ed Morton Jun 6 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.