0

I want to delete a specific line of a file in bash.

What I am currently doing is to get the line number and pass it so sed to delete this line:

awk '/qr/{ print NR; exit }' test | sed -i "${1}d" test

The awk part works well, but in this state, the sed part deletes all the content of the file (named test).

However, when I do it without the variable :

sed -i '1d' test

It works fine.

What am I doing wrong ?

8
  • sed does not read $1 from a pipeline (if it gets a filename arg, it won't read stdin at all). The $1 is expanded by the shell even before either of the processes in the pipeline are executed. So the sed expression is just "d", and it deleted every line. May 25, 2020 at 8:36
  • Ok, good to know. But do I have to store the return of the awk in a variable in order to use it with sed ? @Paul_Pedant May 25, 2020 at 8:37
  • 1
    What speaks against doing the job completely in awk, as in awk '/qr/{if (!i++) next}1' test (assuming you only want to remove the first occurence)? If you need "inplace" function, GNU awk > 4.1.0 supports the -i inplace command-line option for that.
    – AdminBee
    May 25, 2020 at 8:45
  • For non-huge files, I store the whole file in an array X[NR], mark lines for deletion in D[NR], and write the lot back to FILENAME in the END block. That means you can do stuff like mark multiple lines, paragraphs containing any number of specific words, etc. So a generic solution, but it's not for everybody. May 25, 2020 at 8:46
  • I didn't know about "inplace". My way does not use tmp files, and I can abort it cleanly if no change is needed, or I find a data error. I need to check what inplace does if I exit (1) somewhere. May 25, 2020 at 8:52

4 Answers 4

4

${1} is replaced by the first argument in the current context; it doesn’t read anything from awk’s output in your example.

If you want to keep both awk and sed, one way to go about this is to store the output of awk in a variable:

line=$(awk '/qr/{ print NR; exit }' test); sed -i "${line}d" test

but that won’t work well if the file doesn’t have any line containing “qr”.

A better approach might be to use sed only:

sed -i "/qr/d" test

but that will delete all matching lines, not just the first one.

The AWK (or rather, gawk, the GNU implementation, starting with version 4.1.0) equivalent of the above is

gawk -i inplace '!/qr/' test

or, replacing only the first instance,

gawk -i inplace '/qr/ && !i { i++; next; } 1' test
3
  • 1
    That first command would completely wipe out your input file if it didn't contain qr, you'd need to make it line=$(awk '/qr/{ print NR; f=1; exit } END{exit !f}' test) && sed -i "${line}d" test if you were really going to do something like that.
    – Ed Morton
    May 25, 2020 at 16:54
  • Right, that’s why there’s a “but” ;-). Thanks for the working version! May 25, 2020 at 17:05
  • Yeah at first I didn't see the but and then once I did I thought won’t work well was kinda understating the impact so decided to comment :-). You're welcome!
    – Ed Morton
    May 25, 2020 at 17:06
2

Given this input file:

$ cat file
foo
qr
bar

If you were really going to use that approach then it'd be (without -i so you can see the effect of the sed):

$ awk '/qr/{print NR; exit}' file | xargs -n 1 -I {} sed '{}d' file
foo
bar

but of course that's a pointless pipe to xargs+sed and what you should do instead is just:

awk '!f && /qr/{f=1; next} 1' file > tmp && mv tmp file

or with a recent version of gawk that supports it:

awk -i inplace '!f && /qr/{f=1; next} 1' file
0

Assuming you only want to remove the first occurence of the pattern, you can use awk entirely:

awk '/qr/{if (!i++) next}1' test

This will print all lines (1) except for the first line matching /qr/ (where i is still zero, and hence the next command will be issued before the "print" action can become effective).

If you need the "inplace" editing function of sed -i, and you have GNU awk > 4.1.0, you can make use of the "inplace" extension:

awk -i inplace '/qr/{if (!i++) next}1' test
0

If you are looking to just delete the first matching line then

sed  "0,/qr/{/qr/d;}" file 

Just drop -inline in after testing

This only processes the addresses from the first line to the first match with qr

0,/qr/ 

and the deletes any line in that address range that contains qr

{/qr/d;}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.