1

Please help me solving following problem. Remove all pairs of characters \ n from Test_Macro in all files. Please see below example:

Fil1.txt

Test_Macro(abc, def, "\n string1 string2 \n test string",
       "test string2 \n");
// Some code or text

Test_Macro(asdsadas, "test String1");
// Some code...

dir1/File2.txt

Test_Macro(abc, def, "\n string1 string2 \n test string",
       "test string2 \n",
        123456);
// Some code or text

Test_Macro(asdsadas, "test String1");
// Some code...

Expected Result:

File1.txt

Test_Macro(abc, def, " string1 string2 test string",
   "test string2 ");
// Some code or text

Test_Macro(asdsadas, "test String1");
// Some code...

dir1/File2.txt

Test_Macro(abc, def, " string1 string2  test string",
   "test string2 ",
    123456);
// Some code or text

Test_Macro(asdsadas, "test String1");
// Some code...

Any help or suggestions are much appreciated. I am planning to write some script. Because I have many different types of files and many such macros. Thanks in advance!

The arguments to Test_Macro can have nested calls to other macros, and may have any characters inside strings.

closed as unclear what you're asking by G-Man, HalosGhost, Wildcard, techraf, Anthon Oct 28 '16 at 7:35

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • I did something very similar once for saving Lisp programs out of the Read-Eval-Print-Loop output of CLisp, wherein I have to look for nested parentheses and pull out the matching end paren correctly; read here. – Wildcard Oct 27 '16 at 1:26
  • @Sundeep: My final goal is to remove all \n from Test_Macro() in all files. – mehtame026 Oct 27 '16 at 4:19
  • @Sundeep: I have updated my final output for both these files. – mehtame026 Oct 27 '16 at 6:45
  • if you are okay with perl solution, try grep -rlZ 'Test_Macro' | xargs -0 perl -0777 -i -pe 's/(Test_Macro\(.*?\))/$1 =~ s|\\n||gr/ges' ... use some test files though before applying it on actual files – Sundeep Oct 27 '16 at 6:55
  • Thanks Sundeep. I am fine with perl solution. I tried but it gives some error. Backslash found where operator expected at -e line 1, near "s/(Test_Macro\(.*?\))/$1 =~ s/\" syntax error at -e line 1, near "s/(Test_Macro\(.*?\))/$1 =~ s/\" Execution of -e aborted due to compilation errors. – mehtame026 Oct 27 '16 at 18:19
0

File1:

$ sed '/Test_Macro/{N;$!N;s/.*\(Test_Macro[^)]*);\).*/\1/;p;};d' abc.txt
Test_Macro(abc, def, "\n string1 string2 \n test string",
       "test string2 \n");
Test_Macro(asdsadas, "test String1");

File2:

$ sed '/Test_Macro/{N;$!N;s/.*\(Test_Macro[^)]*);\).*/\1/;p;};d' abc2.txt
Test_Macro(abc, def, "\n string1 string2 \n test string",
       "test string2 \n",
        123456);
Test_Macro(asdsadas, "test String1");

ps, the easiest way to remove all line breaks is;

echo -e "line \n break" | tr "\n" " "

without line breaks;

$ sed ':a;N;$!ba;s/[^;]\n[ ]*/ /g;' abc2.txt  | grep Test_Macro
Test_Macro(abc, def, "\n string1 string2 \n test string" "test string2 \n" 123456);
Test_Macro(asdsadas, "test String1");

without "\n" but with line breaks... lol

$ sed '/Test_Macro/{N;$!N;s/[ ]*\\n//g;s/.*\(Test_Macro[^)]*);\).*/\1/;p;};d' abc2.txt
Test_Macro(abc, def, " string1 string2 test string",
       "test string2",
        123456);
Test_Macro(asdsadas, "test String1");

just removing "\n" string (and trailing space);

$ sed ':a;N;$!ba;s/\\n[ ]*//g;' abc2.txt
Test_Macro(abc, def, "string1 string2 test string",
       "test string2 ",
        123456);
// Some code or text

Test_Macro(asdsadas, "test String1");
// Some code...

once again (hopefully the last time)... removing string "\n" when in the function Test_Macro, but not outside, and not removing line breaks;

$ sed '{N;/Test_Ma/{s/[ ]*\\n//g;};s/\(Test_Macro[^)]*);\)/\1/};' abc2.txt
Test_Macro(abc, def, " string1 string2 test string",
       "test string2",
        123456);
// Some code or text \n

Test_Macro(asdsadas, "test String1");
// Some code...

update;

$ sed '{:a;N;/Test_Ma/{s/[ ]*\\n//g;};ta};' abc2.txt 
Test_Macro(abc, def, " string1 string2 test string",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
       "test string2",
        123456);
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n
// Some code or text \n

Test_Macro(asdsadas, "test String1");
// Some code...
  • This is not working. It gives an error like "Command not found" – mehtame026 Oct 27 '16 at 2:00
  • did you copy paste the $ by any chance? – mikejonesey Oct 27 '16 at 3:12
  • Thanks mike. Can you tell me how can I combine this two commands? I actually want to remove all "\n" from these Test_Macros. – mehtame026 Oct 27 '16 at 19:43
  • Thanks again mike. I think you misunderstood me. Please see my expected output in the question. I just want to remove all "\n" characters from these macros. Don't want to remove line breaks. – mehtame026 Oct 27 '16 at 21:38
  • 1
    removing all pairs of backslash and n can be done with sed 's/\\n//g'. – icarus Oct 27 '16 at 22:10
2

There is a phrase that should be remembered, "regular expressions can't count".

This matters in this case because a lot of 'simple' unix tools are based on regular expressions. The counting here is counting the open and close parentheses ('round brackets') that might be used inside the arguments to Test_Macro.

If the calls to Test_Macro never have nested parentheses, then there is an easy trick. First change every ) character to a newline, and vice versa. Then delete every line which doesn't contain Test_Macro, and remove everything up to the Test_Macro. At this point a part of the processed File2.txt would look like this

Test_Macro(abc, def, " string1 string2 test string",)   "test string2 ",)    123456

So now we need to convert the ) back. At this point you have a couple of options. I favor using sed to get rid of the extra spaces at the same time. We also need to add back in the ) and maybe the ;

Putting this together, we have

find . -type f | while read -r fn
do
   < "$fn" tr ')\n' '\n)' | sed -n 's/.*Test_Macro(/Test_Macro(/p' | \
     sed 's/) */ /g;s/$/);/'
done

If there is the possibility that the arguments to Test_Macro includes nested parentheses then you need to bring out significantly bigger guns, as you need to parse the input rather than just pattern match it. (In theory if you can restrict the nesting level then you can pattern match but in practice this gets very very complicated very quickly and you should discount this approach). There are parser frameworks for languages like python, or you can build tools on top of tools like lex.

  • Thanks icarus. Unfortunately I have lot of macros with nested parentheses. Btw thanks for your prompt help and reply. – mehtame026 Oct 27 '16 at 19:18
  • OK. In that case, because regular expressions can't count, we need more powerful tools and the solution will not fit inside this comment. What do you have available? C compiler?, lex or flex?, python?, lua?, something else? – icarus Oct 27 '16 at 21:47
  • I have C/C++ compiler, python, perl. I also have lex & flex. But not familiar with it. – mehtame026 Oct 27 '16 at 21:52
  • Having read all the comments and the edited original post, my solution here is for a different problem. I was attempting to remove everything from the file except the macro calls and then get the macro calls onto a single line. It appears that what @ronme026 wants is to remove all pairs of backslash and n characters that occur within the arguments of Test_Macro. We knew this requires us to parse the data. – icarus Oct 28 '16 at 0:20
  • Yes. I only want to touch that macro and remove all occurrences of "\n". I don't want to change anything else. I am sorry if I haven't explained my problem clearly. – mehtame026 Oct 28 '16 at 0:25
1

Edit: This answer was prepared before the question was revised. The original form of the question included:

When I tried to find some pattern using "grep", it only prints first line. But I want till end of the bracket.

Regular expressions can't count, but Sed can loop.

Here is a Sed snippet that will grab from any line containing Test_Macro to the line with the appropriate close paren, even if there are nested parentheses:

#n
/Test_Macro/{
  p;
  :rep
  s/([^()]*)//;
  trep
  /^[^(]*$/d;
  h;
  n;
  p;
  x;
  G;
  brep
}

Converted to a one-liner, this looks like so:

sed -n -e '/Test_Macro/{p;:rep' -e 's/([^()]*)//;trep' -e '/^[^(]*$/d;h;n;p;x;G;brep' -e '}'

Input and output look like so:

$ cat temp 
Test_Macro(abc, def, "\n string1 string2 \n test string",
       "test string2 \n");
// Some code or text

Test_Macro(asdsadas, "test String1");
// Some code...
$ sed -n -e '/Test_Macro/{p;:rep' -e 's/([^()]*)//;trep' -e '/^[^(]*$/d;h;n;p;x;G;brep' -e '}' temp 
Test_Macro(abc, def, "\n string1 string2 \n test string",
       "test string2 \n");
Test_Macro(asdsadas, "test String1");
$ 
  • Yes, once you add conditional jumps you have a Universal Computing machine (assuming no line length limits), so we can parse. My gut is saying that extending the code in sed is going to become too complicated, although I would not object to sed being a target language for a translation from something simpler to read. – icarus Oct 28 '16 at 4:24
  • 1
    @icarus, my favorite explanation of Sed (from a comment somewhere on this site, in fact, modified slightly) is: Sed is the assembly language of text processing, Awk is the command line spreadsheet, and Perl is everything else you can imagine. :) – Wildcard Oct 28 '16 at 4:31
  • I like it, can I steal it? – icarus Oct 28 '16 at 4:49
  • @icarus, sure. I stole it myself. :) You can give attribution to "someone on the Unix & Linux Stack Exchange" like I did; drive more traffic to the site (maybe). ;) – Wildcard Oct 28 '16 at 4:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.