I have various text files I need to modify


I'm trying to automate the process of removing "test5232.lop", including the proceeding the pipe, like this:


Without the need to generate a temp file if possible

  • If any of the answers solved your problem, please accept it by clicking the checkmark next to it. Thank you! – Jeff Schaller Nov 26 '17 at 15:39

This looks like a job for cut. Tell it the delimiter is |, that we want to specify fields to drop, not fields to keep (--complement) and that the we want to select field 3 (to drop in this case).


 cut -d '|' --complement -f 3


$ echo 'test.xyz|test3.abc|test5232.lop|filename.test|file.text|qwerty.bat|x' | cut -d '|' --complement -f 3
  • Nice! I mostly use POSIX features only, so I wasn't aware of the --complement option. Notably, this is GNU-specific. – Wildcard Apr 1 '17 at 1:04
  • I was looking at a more specific option, because its in text file and "test5232.lop" isn't always in the same place. – John Doe6262 Apr 1 '17 at 1:10
  • @JohnDoe6262, that's actually a much less interesting use case. – Wildcard Apr 1 '17 at 1:13
  • @JohnDoe6262, maybe @wildcard will modify his sed answer, because if just want to remove a specific string sed is super easy. – Stephen Rauch Apr 1 '17 at 1:14
  • 1
    If your cut doesn't support --complement you can use a field list instead: cut -d '|' -f 1-2,4- – David Foerster Apr 1 '17 at 12:48

Check also this awk simple solution. Will remove the string no matter where it is and should be portable:

$ a="test.xyz|test3.abc|test5232.lop|filename.test|file.text|qwerty.bat"
$ awk -F"test5232.lop." '{printf("%s%s\n",$1,$2)}' <<<"$a"

About your request for in-place editing , GNU AWK version > 4.1 also can make inplace edits according to gawk manual:

awk -i inplace -v INPLACE_SUFFIX=.bak '{...}'

But in any case, neither awk nor sed nor perl can achieve a real inplace editing. GNU sed Info Pages clarify this issue for us:

     This option specifies that files are to be edited in-place.  GNU
     'sed' does this by creating a temporary file and sending output to
     this file rather than to the standard output.(1).

Meaning that you can use any solution in-here by appending at the end something like this:

awk/sed/perl/whatever oldfile >tmpfile && mvtmpfile oldfile && rm -f tmpfile 
  • @JohnDoe6262 Clarification about in-place editing added – George Vasiliou Apr 1 '17 at 10:09

Just use Sed:

sed 's/|test5232\.lop//' file.txt

Original answer, before request was clarified:

POSIX features only, using Sed:

sed 's/|[^|]*//2' file.txt

If you know that all lines have at least three | symbols, you can use the more intuitive form:

sed 's/[^|]*|//3' file.txt
  • This works well, can sed input/output to the same filename without needing to create a temp filename? – John Doe6262 Apr 1 '17 at 1:18
  • Not portably. Use -i option if your Sed supports it. (Check the man page for whether it requires an argument or not; that depends on what implementation you use.) – Wildcard Apr 1 '17 at 1:21

To replace the 3rd field (where a "field" is "anything except a pipe, zero or more times, followed by a pipe") with nothing:

awk '{$0=gensub(/[^|]*\|/, "", 3); print $0}' input

Apparently you want to remove "test5232.lop" anywhere in the line:

sed -i 's/|test5232\.lop//' input

(although any solution, including sed -i creates a temporary file)

perl -F'[|]' -pale '$_ = join "|", grep $_ ne "test5232.lop", @F' yourfile

perl -F'[|]' -pale '1 while s/(^|\|)\Ktest5232\.lop(\||$)//g'     yourfile

sed -e '

' yourfile

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.