6

I would like to use bash or shell script and replace anything between the two parentheses with an empty space. The text between the two parentheses could be in multiple lines, such as:

myFunction (line0

line1

line2

line3

line4) 

that I would like to convert to:

myFunction ( )
  • 2
    Did you want to replace line with newline or prepend/prefix all non-empty lines inside the parentheses with new? – phk Jan 29 '17 at 12:35
  • No, indeed I want to replace some text which is sitting between two parentheses in multiple lines with some new text that I am going to read from a second file. – Armin Jan 29 '17 at 18:32
  • You want to convert to myFunction ( ) or to replace the text with new text that comes from a second file as per your comment above? You talk about two different things. – George Vasiliou Jan 29 '17 at 21:58
  • Eventually I would like to replace the text within the two parentheses with a new text. However, the final replacement is easy. That's why I simplified my question. – Armin Jan 29 '17 at 22:06
  • This has turned into an meta.stackexchange.com/questions/66377/what-is-the-xy-problem/… - tell us what you actually want to do. By all means tell us what you can do yourself. – icarus Jan 30 '17 at 3:19
5

AWK

AWK allows executing code-block {} on range of conditions. In this case, we want to execute gsub() on every line in range from the one that contains ( to the one that contains ).

$ awk '$0~/[(]/,$0~/[)]/{gsub(/line/,"newline")};1' input.txt                                                     
another line
something else

myFunction (newline0

newline1

whatever

newline2

newline3

newline4)

some other line

Python (original answer)

Here's a quick python script that does the job:

#!/usr/bin/env python3
from __future__ import print_function
import sys

with open(sys.argv[1]) as fp:
    flag = None
    for line in fp:
        clean_line = line.strip()
        if "(" in clean_line: flag = True
        if flag:
           clean_line = clean_line.replace("line","newline")
        print(clean_line) 
        if ")" in clean_line: flag = False

Test run:

$ cat input.txt                                                                                                          
another line
something else

myFunction (line0

line1

lilne2

line3

line4)

some other line
$ ./edit_function_args.py input.txt                                                                                      
another line
something else

myFunction (newline0

newline1

newline2

newline3

line4)

some other line

BASH version

The same script, except rewritten in bash with sed

#!/bin/bash
flag=false
while IFS= read -r line
do

    if grep -q '('  <<< "$line"
    then
        flag=true 
    fi


    if $flag
    then
        line=$(sed 's/line/newline/'   <<< "$line") 
    fi

    printf "%s\n" "$line"


    if grep -q ')'  <<< "$line"
    then
        flag=false     
    fi

done < "$1"
  • Thanks! I am going to check it. In the meantime, do you know by chance any way to do the same thing in shell ? This is mainly because the rest of my script is in bash. – Armin Jan 28 '17 at 23:35
  • @Armin Sure, can be done. I can rewrite my python script in bash+sed or even awk, but I'm kind of running out of time here - have to leave my work in 20 minutes. I'll edit it and post once i get home. OK ? – Sergiy Kolodyazhnyy Jan 28 '17 at 23:37
  • @Armin I also threw in an awk version, which is fairly short an simple. Hope that helps – Sergiy Kolodyazhnyy Jan 29 '17 at 1:15
6

Taking the bash answer of @Serg and converting it to use bash builtins, rather than 2 or 3 processes per line. Processes are cheap but not free!

#!/bin/bash
# Use shell builtins, read, true, false, printf
flag=false
while IFS= read -r line
do
    case "$line" in
    (*"("*) flag=true ;;
    esac

    if $flag
    then
        line=${line//line/newline} 
    fi

    printf "%s\n" "$line"

    case "$line" in
    (*")"*) flag=false ;;
    esac

done < "$1"
  • Heh, OK, pretty good. I'd want to say that on modern systems with plenty of RAM and multi-core processes multiple processes wouldn't be an issue, but it might be useful on legacy systems. – Sergiy Kolodyazhnyy Jan 29 '17 at 1:10
  • *multi-core processors , small typo there – Sergiy Kolodyazhnyy Jan 29 '17 at 1:20
  • 1
    @Serg, Time it! Multi-core processors don't help as this is all sequential. On a system where each core reports about 5000 bogomips, it takes me about 6 milliseconds to start a sed, read a line, and exit, and 7 milliseconds to start a grep. This assumes that sed and grep are in the system cache. So this means assuming that all other overheads can be ignored you are limited to about 70 lines a second. Because I am ignoring all other overheads I can process at an infinite number of lines a second :-) using shell builtins. – icarus Jan 29 '17 at 1:47
  • @icarus, how would you change line=${line//line/newline} to replace anything in this line with empty space ? Many thanks in advance. – Armin Jan 29 '17 at 21:16
  • @icarus Eh . . . I somehow doubt shells , even with built-ins, perform noticeably better. If we want speed and performance, then it's job for C or other compiled language, heck maybe even for Python. – Sergiy Kolodyazhnyy Jan 30 '17 at 0:03
5

If perl solution is okay and file is small enough to be processed as a whole:

$ perl -0777 -pe 's/\([^)]+\)/$&=~s|line|newline|gr/ge' ip.txt    
myFunction (newline0

newline1

newline2

newline3

newline4) 
  • -0777 slurp entire input file
  • \([^)]+\) pattern to match - ( followed by non ) characters and ending with )
  • $&=~s|line|newline|gr the matched pattern is referenced here using $& and the desired replacement (line to newline) is done. Note the r flag to return the result as the replacement string
  • e flag allows to use expression instead of string
  • use perl -i -0777 -pe for inplace editing
  • Thanks @Sundeep. Before trying this, please note that I don't know what is the exact value for variable line. How can I say whatever between to parentheses to be replaced by let's say an empty space? – Armin Jan 29 '17 at 20:22
  • @Armin not sure of your question, something like this? perl -0777 -pe 's/\([^)]+\)/( )/g' ip.txt – Sundeep Jan 30 '17 at 1:51
4

For the question and data as originally presented, a sed 1-liner works

  sed '/(/,/)/s/line/newline/g'

which says for each region that starts with a line containing an ( and ends with a line containing a ')', substitute globally line for newline. Remove the g if you only want to change the first line on an input line.

For the modified question,

 sed -e '/(/{' -e ':loop;s/(.*)/()/;t;N;b loop' -e '}'

works. It loops over the input,printing it until it finds a (. At this point it tries to change everything inside a ( ) pair including the delimiters to just (). If this succeeds it breaks out of the loop, prints the result and continues. If it didn't manage to do so, usually because it hasn't yet seen the ), it appends the next input line and continues the loop. If you don't want it on one line then writing it as

sed -e '/(/{
:loop
s/(.*)/()/
t
N
b loop
}'

makes it easier to follow.

  • Pretty good. Gave me idea for a similar awk solution. – Sergiy Kolodyazhnyy Jan 29 '17 at 1:16
  • 1
    As data are presented, even a sed 's/line/newline/g' would do the job.... – George Vasiliou Jan 29 '17 at 1:29
  • OK, using the data as presented and the problem description, you need the range limiting. The point of the "data as presented" comment is that it allows us to ignore nesting of parenthesis, and that there are no line before an ( or after a ) that need to be left unchanged. – icarus Jan 29 '17 at 1:49
  • 1
    @GeorgeVasiliou as presented - yes. But you gotta think one step ahead. OP likely has file with more than just one function. You have to anticipate that there are line words that OP might not want to edit or they are outside of parenthesis – Sergiy Kolodyazhnyy Jan 29 '17 at 2:01
  • 1
    @Serg Yes, sure. I just believe that OP should enrich the question. Even thinking that this is a source code file with lot of functions, the correct question & answer i think should recognize myfunction ( line0 , etc) instead of just (line0,etc ) – George Vasiliou Jan 29 '17 at 8:51
3
Using awk:
awk '
  function mysub(str) {
    if (str) gsub(/line/, "newline", str); return str
  }
  BEGIN {
    OFS=FS="("
  }
  NF>1 {
    if (FS=="(") {
      print $1,mysub($2); OFS=FS=")"
    } else {
      print mysub($1),$2; OFS=FS="("
    }
    next
  }
  {
    print FS=="(" ? $0 : mysub($0)
  }' /path/to/input

The custom function mysub is where you do the replacements you want to do between the parentheses. The assumption is that parentheses aren't nested.

How it works:

There are two states, inside and outside the parentheses.

  • Outside (the starting state), the input and output separator is set to the opening parenthesis (OFS=FS="(").
  • Once it comes across a line with more than one field separated by the input separator (NF>1) and…
    • …you are currently in the outside mode (FS=="("), everything before and after the field separator is outputted (with the output separator in-between) but with latter going through the replacement function (mysub($2)) and then the the mode is flipped by changing the input and output separators (OFS=FS=")"),
    • …otherwise (else), you are in inside mode, and everything before and after the field separator is outputted but this time with former going through the replacement function (mysub($1)) and the mode flips here as well.
  • On all other lines, the whole line is outputted unchanged if outside (FS=="(") or otherwise goes as a whole through the replacement function (mysub($0)).
Compressed into a single line:
awk 'function m(s){gsub(/line/,"newline",s);return s}BEGIN{OFS=FS="("}NF>1{if(FS=="("){print $1,m($2);OFS=FS=")"}else{print m($1),$2;OFS=FS="("}next}{print FS=="("?$0:m($0)}' /path/to/input
My trickier test data (which some the one-liner solutions here will fail with):
line96
line97 myFunction (line0

line1

line2

line3

line4) line98
line99
Output for it:
line96
line97 myFunction (newline0

newline1

newline2

newline3

newline4) line98
line99
Variation where everything between the parentheses is deleted instead (except for the first and last line breaks):
awk '
  BEGIN {
    OFS=FS="("
  }
  NF>1 {
    if (FS=="(") {
      print $1,""; OFS=FS=")"
    } else {
      print "",$2; OFS=FS="("
    }
    next
  }
  FS=="("' /path/to/input
Output for this one:
line96
line97 myFunction (
) line98
line99
  • @Armin If this answer solves your problem, would be kind except thanking phk also to accept this answer as a solution. – George Vasiliou Jan 29 '17 at 0:49
  • Thanks @phk. Would you lease let me know how in this script could I replace "any thing" (not a specific string), with let's say an empty space. Still I am struggling with replacing anyThingMultiLine with empty space between parentheses. I appreciate your help. – Armin Jan 29 '17 at 21:05
  • @Armin See my latest edit of my answer. – phk Jan 29 '17 at 21:19
2

Combining all the clarifications, comments and revised data, my offer would be like bellow.

First of all lets consider a source file d.txt containing your advised myFunction (...) plus one more function to be more realistic.
To be on the hard side lets suppose that those two functions in this d.txt file have almost identical contents, like this:

$ cat d.txt
myOtherFunction (x as boolean
y as integer
d as string
e as whatever)

myFunction (xx as boolean
yy as integer
dd as string
ee as whatever)

Lets suppose now that in another file d2.txt we have a different function.

$ cat d2.txt
BrandNewFunction (xxx as integer
yyy as boolean
ddd as integer
eee as whatever)

Forgetting the sample names of line, newline , etc , and considering your comments it seems that what you really want is to replace in your initial source code file d.txt the existing myFunction (..) with the BrandNewFunction (...) present in file d2.txt.

This can be done easily using pure bash:

$ a="$(sed -n '/myFunction (/,/)/p' d.txt)" #isolates myFunction from the source file d.txt
$ b="$(cat d2.txt)" #get contents of file d2.txt (BrandNewFunction)
$ c="$(cat d.txt)" #get the whole source file d.txt
$ echo "${c/$a/$b}" #in source file d.txt ($c) replace $a with $b (d2.txt)
#Output:
myOtherFunction (x as boolean
y as integer
d as string
e as whatever)

BrandNewFunction (xxx as integer
yyy as boolean
ddd as integer
eee as whatever)

Or even as one-liner:

$ a="$(sed -n '/myFunction (/,/)/p' d.txt)";b="$(cat d2.txt)";c="$(cat d.txt)";echo "${c/$a/$b}"

Above command just prints on your screen the replacement results (echo). To save the results just send echo to >d.txt for overwritting the existing file or even a new file if you like.

Sed seems not to be quite good to replace multiple lines separated by newlines, since it is focused on line operations.

AWK should be good for the job, but i'm not good in AWK.

Bash is the easiest solution, which can successfully replace multilines.

PS1: If file d2.txt contains more functions and you want to isolate the BrandNewFunction (..), similarly to source file d.txt you just need to modify the $b variable definition like this:

$ b="$(sed -n '/BrandNewFunction (/,/)/p' d2.txt)"

PS2: If you just want to replace myFunction (...) of source file d.txt with a blank function of the same name, you can just hardcode variable b like this (you know already which function from source file d.txt you want to delete, right?)

$ b="myFunction ( )" 
  • @Armin This solution should cover you entirely. – George Vasiliou Jan 29 '17 at 23:54
  • @Armin It would be kind to leave a comment just to advise what made you unaccept this answer. It would help me to improve future answers on similar posts. – George Vasiliou Jan 31 '17 at 18:30
  • @Georges, indeed I used parts of your code to complete mine. Specially sed -n idea was very good. I didn't do a copy-past of the entire thing that you proposed, by I certainly used part of it. Please accept my apologies if I miss something as I am really new in this community. Thanks anyways for all your very useful advices. – Armin Jan 31 '17 at 22:24
  • @Armin Ok. I just got happy yesterday to see that you accepted my answer as a solution (maybe by mistake) and got unhappy today to see that my answer was unaccepted. Anyway, is your right to accept the best answer for you. – George Vasiliou Jan 31 '17 at 22:32
  • @Armin I think that you also changed the accepted answer again. Yesterday you accept my answer, this afternoon you unaccept my answer and you accept the perl solution of Sundeep and now i can see that you also unaccept Sundeep solution and you accepted dirkt solution. You can upvote all solutions if you like them but you can accept only one solution. Do not hop around :-) – George Vasiliou Jan 31 '17 at 22:37
2

If you also want to process nested parenthesis correctly, use a tool for a context-free language instead of a regular language, for example sgrep:

sgrep -o '%r ' '(start .. end) extracting ("("__")")' < input_file

This way, for example, the following

myFunction (line0

line1

(line2)

line3

line4) 

anotherFun (x y)

becomes

myFunction ( ) 

anotherFun ( )
  • Thanks @dirkt. Unfortunately still I don't have sgrep on my machine properly installed. – Armin Jan 30 '17 at 22:37
  • Thanks @everyone. Eventually I had to use a combination of the solutions and tricks that you had proposed. Here is the algorithm that I implemented successfully in bash: (a) read line by line the file to detect myFunction and its contents, (b) commented myFunction using sed, (c) inserted new myFunction in the right place using a combination of sed and print. – Armin Jan 31 '17 at 21:22
0

A new version of GNU sed supports the -z option.

Normally, sed reads a line by reading a string of characters up to the end-of-line character (new line or carriage return).
The GNU version of sed added a feature in version 4.2.2 to use the "NULL" character instead. This can be useful if you have files that use the NULL as a record separator. Some GNU utilities can genertae output that uses a NULL instead a new line, such as "find . -print0" or "grep -lZ".

You can use this option when you want sed to work over different lines.

sed -z 's/([^)]*)/( )/g' inputfile

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.