New answers tagged

0

Based on this answer: sed -n '/your-pattern-here/{x;p;x;p;d;}; x' infile


1

With GNU sed: sed ':a;N;s/\n/,/;ta' file | sed 's/,Cat/\nCAT/g' or tr '\n' ',' < file | sed 's/,Cat/\nCAT/g'


1

Some sed implementations have support for that. ssed has a PCRE mode: sed -R 's/AB.*?AC/XXX/g' AT&T sed has conjunction and negation when using augmented regexps: sed -A 's/AB(.*&(.*AC.*)!)AC/XXX/g' Portably, you can use this technique: replace the end string (here AC) with a single character that doesn't occur in either the beginning string (...


0

If you're comfortable with line-oriented tools like sed, grep, awk etc AND you only want to do simple extractions of key/value pairs, you'll probably find jsonpipe easier to work with than jq. By default, he keys are printed as a /-delimited path, separated from the values by a tab. e.g. $ json='{"TopicArn": "arn:aws:sns:us-west-2:596873354795:demo"}' $ ...


1

No, sed regexes don't have non-greedy matching. You can match all text up to the first occurrence of AC by using “anything not containing AC” followed by AC, which does the same as Perl's .*?AC. The thing is, “anything not containing AC” cannot be expressed easily as a regular expression: there is always a regular expression that recognizes the negation of ...


2

awk awk ' /Cat/ { if (NR>1) print "" printf "%s", $0 next } {printf ",%s", $0} END {print ""} ' file Another version that heavily relies on awk variables: (added before I read your comment about "Cat" needing to be a case-insensitive regex) awk 'BEGIN {RS="Cat"; FS="\n"; OFS=","} NR>1 {$1=RS; NF--; print}' ...


1

You could do something like this with sed: sed '1{h;d;};/^Cat$/!{H;$!d;};x;s/\n/,/g;${x;/^Cat$/H;x;}' infile explained: sed '1{ # if this is the 1st line h # copy over the hold space d # and delete it } /^Cat$/!{ # if the line doesn't match Cat H # ...


1

This solution does not require to read the whole file into memory. In other words: It will work on a 1 TB file being processed on a 1 GB machine as long as the full lines are less than 1 GB. perl -ne 'BEGIN { $sep = shift; } if(/^$sep$/o) { @p and print join(",", @p)."\n"; @p = (); } chomp; push @p, $_; END { print join(",", $...


0

You could do something like sed '/AB.*AC/{s/AB/\ &/;h;s/.*\n/\ /;s/AC/\ &/;s/AB.*\nAC/XXX/;H;x;s/\n.*\n//;}' this should be quite portable though with ancient seds you might want to write it as: sed '/AB.*AC/{ # if line matches this pattern s/AB/\ # add a newline before the first AB &/ h ...


1

In your case you can just negate closing char this way: echo 'ssABteAstACABnnACss' | sed 's/AB[^C]*AC/XXX/'


2

Sed regexes match the longest match. Sed has no equivalent of non-greedy. Obviously what we want to do is match AB, followed by any amount of anything other than AC, followed by AC Unfortunately, sed can’t do #2 — at least not for a multi-character regular expression.  Of course, for a single-character regular expression such as @ (or even [123]), we ...


4

Try double quotes for the outer ones: sed -i "/, false);/adefine( 'WP_MEMORY_LIMIT', '64M' );"


2

After reading the comment of @don_crissti I think I understand how n command affects flow control of sed. n excludes the read (or next) line from the analysis ..."/c/{... in the next cycle. So if use n 2 times in my example, abc will be matched then cde will be read, then fg will be read and deleted. $ echo -ne "abc\ncde\nfg\n" | sed "/c/{ n n /f/d }" ...


4

In your program.sed you should have: s/the/zee/g; s/The/Zee/g Or: s/the/zee/g s/The/Zee/g


6

The problem, your line fg is never matched by the pattern /f/. The first line abc matched /c/, then the commands inside block are executed. The n command print the current pattern space, which is abc and replace the pattern space with next line of input, which is cde. cde doesn't match /f/, then it was not deleted, and printed to standard out. Now, next ...


5

Your sample file is in the structured format called JSON. sed is not a convenient tool to deal with this type of files. Instead install a parser for JSON format, for example jq. To install on Debian-family Linux: sudo apt-get install jq To install on Red Hat-family Linux: sudo yum install jq For other distributions and platforms check this page. ...


0

Similar to Stephane's perl solution, awk can avoid the issue of protecting a 'dangerous' interpolated value with an alternate approach: export pattern="anypattern" # or whatever shell quoting is needed awk 'NR==1,$0~ENVIRON["pattern"] {sub(ENVIRON["pattern"],"replacement")} 1' input # or gsub if you want multiple matches on the first line with any match ...


12

Your sed command 's/^ $/^$/' won't do what you want. It just replace all lines contains one space with a line contain ^$. Depend on what characters mark end of sentence, you can do: sed -e 's/\([.?!]\) \{2,\}/\1 /g' <file This will replace 2 or more spaces after ., ? or ! with one space only.


11

sed 's/\. */. /g' < file replace dot followed by two or more spaces with dot followed by a single space.


7

This is what you might be looking for, tr -s " " <filename Sample, $ echo "This is the output. Hello Hello" | tr -s "[:blank:]" This is the output. Hello Hello Using sed, $ echo "This is the output. Hello Hello" | sed 's/\. \+/. /g' $ echo "This is the output. Hello Hello" | sed 's/\. \{1,\}/. /g' This is the output. Hello Hello


0

Portably: sed ' s/.*/ & /; # add a leading and trailing space :1 s/\([[:blank:]]541[^[:blank:]]\{2\}\)\([^[:blank:]]\{5\}[[:blank:]]\)/\19\2/g # replace in a loop until there is no more match t1 # remove the blanks we added earlier: s/^ //;s/ $//' You can avoid the temporary adding of the leading and trailing spaces by looking for ...


1

You could use perl perl -lane 'map{length==10&&/^541/&&s/.{4}/$&9/}@F;print join(" ",@F)' file Use map to perform a checks and sub on each field. Then prints the array of fields join by a single space(so will mess up formatting if not consistent single spaces between fields) or just using regex perl -lane 'map{s/^541.\K.{6}$/9$&/}...


2

As see from example OP mean a word not a string so sed 's/\b541./&9/g' file If the 541 can start from elsewhere in word (not from the beginning) sed 's/\b\S*541/\n&/g #mark a beginning of word(s) with pattern s/\n\(....\)/\19/g #remove mark and do adding ' file You can limit symbols quantity the word(s) as follows sed 's/\b\(541....


6

While: sed "0,\~$var~s~$var~replacement~" Can be used to change the regex delimiter, embedding variable expansions inside sed (or any other interpreter) code is a very unwise thing to do in the general case. First, here, the delimiter is not the only character that needs to be escaped. All the regular expression operators need to as well. But more ...


1

Character 16 in the sed script does not exist. This is the " character that sed is complaining about, and that means that your editor or input method replaced it with some non-ASCII rendition of ". My guess would be either “ or ” or ¨. Use file on your file in order to get some guess on the encoding. It should be "ASCII". Anything else hints to the file ...


2

grep was built for this. to search case_-i_nsensitive, and return 1 line _B_efore, use similar to grep -i -B1 tell\ to\ your\ mom your_msg_dump_with_each_msg_on_seperate_line.txt


3

If you mean that the replacement should not be done if there are more than 8 adjacent (TAB)s, you could do: sed ' s/_/_u/g; # escape _ s/|/_p/g; # escape | s/(TAB)/|/g; # use a single character in place of (TAB) s/.*/<&>/; # add leading and trailing non-| character s/\([^|]\)|\{2,8\}\([^|]\)/\1|\2/; # replace up to 8 | provided ...


4

I'm not sure I understand how the "maximum 8" clause is supposed to apply, but the naive approach would be something like this: sed 's/\((TAB)\)\{2,8\}/(TAB)/g'


4

& is special in the replacement text: it means “the whole part of the input that was matched by the pattern”, so what you're doing here replaces user=&uidX with user=user=&uidXsysuserid.. To insert an actual ampersand in the replacement text, use \&. Another thing that looks wrong is that . in the search pattern stands for any character (...


0

With POSIX awk: awk '{sub(/\r/,"")}1' CRLF.txt > LF.txt awk '{sub(/$/,"\r")}1' LF.txt > CRLF.txt


2

Rather than look in the operating system, look at scripts used for building the programs. Any autoconf-generated configure script contains sed-scripts, and itself generates a script which contains a sed-script. That substitutes values for names in the template files such as Makefile.in to produce a Makefile. For example (conveniently found with web search)...


4

With ed instead of sed ed -s << EOF file.txt 0,/MATCH/-1 r text.txt ,p q EOF or as a one-liner printf '0,/MATCH/-1 r text.txt\n,p\nq' | ed -s file.txt ----- add this text file before first match MATCH ----- MATCH ----- MATCH ----- (replace ,p by w for in-place editing).


1

Try this $ sed '0,/MATCH/ s/MATCH/add this text file\nbefore first match\nMATCH/' file.txt or just use other sed expression for your output $ sed '0,/MATCH/i prependme once' file.txt | sed 1d for adding file content $ sed -e '0,/MATCH/s/MATCH/$(cat text.txt)\nMATCH/' file.txt Other possible solution using ----- as your matching string $ sed '0,/----...


7

To select only the SSE flags, try: awk '/SSE/' ORS=' ' RS=' ' The key thing here is setting the record separators on input and output to a space. That way, each option is accepted or rejected separately. For example: $ SUNCC_CXXFLAGS="-D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__SSE4_1__ -D__SSE4_2__ -D__AES__ -D__PCLMUL__ ..." $ newFLAGS="$(echo "$...


0

If you have awk, you can use the parentheses as field separators: awk -F '[()]' '{ for (i=1; i<=NF; i+=2) { if ($i) { gsub(/; */,"\n",$i) printf "%s", $i if ($(i+1)) printf "(%s)", $(i+1) } } print "" }' <<END ProductName: Threat Emulation; product_family: Threat; Destination: (...


3

Under -E, you need to use unquoted parentheses to create a capture group. sed -E 's:(foo):(\1):'


3

For there to be two or more numbers then there must be at least one separator. If this is a space then the grep would simply be grep ' ' If you file may have extraneous spaces at the end of the line then search for a space followed by a digit or . (in case a number may be .25). grep ' [0-9.]' If you may have extraneous spaces at the beginning or end ...


1

You can do this with grep or sed. With sed: sed -e '/^[[:space:]]*[[:digit:].]\+[[:space:]]*$/d' input >output With grep, use the -E option for regular expressions, and -v option to exclude matches: grep -v -E '^[[:space:]]*[[:digit:].]+[[:space:]]*$' input >output The reason for the pattern is to ignore lines which do not have a numeric value, ...


8

One approach using awk. Uses NF (number of fields) to only print the lines where number of fields is greater than 1. awk <oldfile >newfile 'NF>1' Example awk <oldfile >newfile 'NF>1' cat newfile 0.0 4 6 5 1 2 9 4 5 1 ..... 0.5 3 1 1.0 3 7 8 2 1.5 3 3 3 4 6 4 5 2.5 6 7 6 9


1

GNU grep is able to do this quite simply. From man grep: Two regular expressions may be joined by the infix operator |; the resulting regular expression matches any string matching either subexpression. grep "ERROR\|FAIL\|WARN" /path/to/example.log egrep eliminates the need for escaping the | symbols. egrep "ERROR|FAIL|WARN" /path/to/example.log


1

Now, that your Data format has been established, the answer becomes a lot simpler: grep was built for this. Use as grep '<PATTERN>' <dataFile> Where <PATTERN> is SearchWORD1 or SearchW1\|SearchW2 The answer below was written, when me and @murphy still had wrong assumptions about the dataformat: Here is a one-line awk program that only ...


1

I suppose your log file looks like this? example.log: [09:44:22] [main] ERROR [url/location] - A ONE LINE ERROR [09:44:22] [main] ERROR [url/location] - A MULTI LINE ERROR with whitepace indention [09:44:22] [main] ERROR [url/location] - A MULTI LINE ERROR with tab indention [09:44:22] [main] SOMETHING DIFFERENT [09:44:22] [main] SOMETHING ...


4

A little bit confusing regex for sed but workable sed ' :a #mark return point s/\(\(^\|)\)[^(]\+\);\s*\([^)]\+\((\|$\)\)/\1\n\3/ #remove ; between ) and ( ta #repeat if substitute success s/[[:blank:];]\+$// #remove ;...


5

The awk solution: awk '{ getline a <"file2" split(a,A) for(i=1;i<=NF;i++) printf("%s_%s ", $i, A[i]) print "" }' file1 paste + sed: paste file1 file2 | sed ' :a s/\(\(^\|\s\)[^_[:blank:]]\+\b\)\s*\(.*\t\)\(\S\+\)\s*/\1_\4 \3/ ta s/\s*$// ' bash loop: exec 3<file1 4<file2 while read -u 3 a ; ...


6

Try: awk '{f=1} $4 ~ /^192.168/{f=0} $4 ~ /192.168.(125.100|126.100|155.240)/{f=1} f' file Example Consider this test file: $ cat file Jul 13 21:47:41 192.168.100.254 "user from 192.168.100.101" Jul 13 21:47:41 192.168.125.100 "user from 192.168.100.101" Jul 13 21:47:41 192.168.126.100 "user from 192.168.100.101" Jul 13 21:47:41 192.168.155.240 "user ...


1

Apart from \r remove the loop can be subtitute by sed -ni 'P;F' file1 file2 file3 … ^P;F^N;s/\n/\t/p^


2

The ^M are \r, "carriage returns" and were, apparently, already there in your original file. You can remove them with: sed -i 's/\r//' file1 file2 file3


3

With sed: sed ' /^Query_/{ #starts loop when meet patten :a $!{ N /\nQuery_/!ba #untill meet next pattern } /\(\n.*\)\{6,\}/{ #checks how many lines in block $b #for end of file s/\nQuery_/\n&/ #marks lines to print } } ...


2

With awk: awk '{if($1~/^Query_/){c=0;delete a;a[0]=$0}else{c++} if(c<5){a[c]=$0} if(c==5){for(i in a){print a[i]}} if(c>5){print}}' file In the first line the first field $1 is checked whether it begins with Query_. If so, the counter variable c is set to 0. The array a is removed, and the first element of the array is set to the ...


1

you haven't defined the rcps variable you defined scenarios but used scenario Here's a version of your script that fixes those problems and makes a few other improvements: formatted with extra newlines and aligned indentation to improve readability use arrays rather than space-separated strings use sed once rather than multiple times plus tr double-...



Top 50 recent answers are included