Tag Info

1

The range of lines in the sed command by @muru caught my eye. You can do this relying less upon the format of cal: cal 12 2015 | sed '/[[:alpha:]]/d' | tr ' ' '\n' |sed -e '/^$/d' POSIX does not specify the format for cal, so the length of the header could vary by implementation. 3 You came close. You need to suppress sed's output before selectively printing lines:$ cal 12 2015 | sed -n '3,7p' 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Your sed might not have -n. In which case, use d to delete all lines except the ones you want. ...

0

you can do it in python too in this way. lines1 = [ line.rstrip() for line in open("file1") ] lines2 = [ line.rstrip() for line in open("file2") ] for i in xrange((len(lines1))): print lines1[i] + " ||| " + lines2[i] ... 1Mo 1,1 I love you. ||| 1Mo 1,1 Ich liebe dich. 1Mo 1,2 I like you. ||| 1Mo 1,2 Ich mag dich. Hi 1,3 I am hungry. ||| Hi 1,3 Ich habe ...

3

You can do: sed -n '/4$/p' file.txt /4$/ matches 4 at the end of the line i.e. at the end of the last column.

4

Using Gnu Awk: awk '/\<TML[0-9]*E\>/' log Match any word (field) on a line (record) that begins with TML, has any number of digits and then ends in E.

2

The following should work in any version of awk that supports user-defined functions, as well as built-in sprintf() and rshift() functions. This includes GNU awk. I borrowed and adapted the decimal to dotted-quad IP Address algorithm from here: http://stackoverflow.com/questions/29025177/how-can-i-convert-a-hex-ip-address-to-dotted-decimal-notation As ...

0

If you want to avoid actually modifying the value of your variable, you can just stream it: printf "%s\n" "$PATH/$ID" | sed -e's|[]$/.\^[]|\\&|g' \ -e's|.*|/path = &/g|' | sed -e':a;N;s/\n/&/2;Ta' -f- -eP\;D /collection.txt It assumes neither variables can contain a newline. It also makes an assumption about: /path = ... 2 So you want something like this: tree | sed 's/├\|─\|│\|└/ /g' It replaces all those "line" characters with spaces. See:$ tree . ├── dir1 │   ├── file1 │   └── file2 └── dir2 ├── file1 └── file2 2 directories, 4 files $tree | sed 's/├\|─\|│\|└/ /g' . dir1 file1 file2 dir2 file1 file2 2 directories, 4 ... 0 This general trick works even if your curl config file contains miscellaneous options such as user-agent, referer, etc. First step, assume your config file named curl_config, then use awk '/^[Uu][Rr][Ll]/{print;print "output = dummy/"++k;next}1' curl_config > curl_config2 to create a new curl config file which incremently append different output file ... 0 Assuming by word, you mean any sequence of non-spacing characters, with GNU sed:$ printf '%s\n' "foo bar baz" more{1..5} | sed -Ee '1!b;:1;/(\S+\s+){4}\S+/!{N;b1};s//replacement/' replacement more3 more4 more5 replaces those first 5 words with one "replacement" word. The portable/standard equivalent is the same but more wordy. ...

2

This should do it in two lines: sed -n 's/\s*URL\s*=\s*$$.*$$/\1/p' /tmp/curl.conf|xargs -I {} curl -O "{}" sed -n 's/\s*URL\s*=\s*$$.*$$/\1/p' /tmp/curl.conf|xargs -I {} basename "{}"|xargs -I {} sed '/mortgage/q' "{}" The first sed command on each line extracts the URLs from your urls file (/tmp/curl.conf in the example). In the first line we use curl's ...

1

You have to use several types of quotes or escape them: sed -i "12iexec('/var/www/scripte/autostandby.sh > /dev/null 2>/dev/null &');" *.php Bonus: sed -i "s|exec('/var/www/streams/taketv.sh');|exec('/var/www/streams/taketv.sh > /dev/null 2>/dev/null \&');|"

0

sed '12i\ exec('\''/var/www/scripte/autostandby.sh > /dev/null 2>/dev/null &'\''); ' file.php You have to quote the whole thing. Your problem is you're trying to insert quotes from within a quoted string. It is doable, but you have to quote the quotes. command 'begin quoted arg'\'' close quotes backslash quote the apostrophe open ...

2

cdrtoip is indeed pretty slow, it looks like a useful utility script but was probably not intended to be called hundreds of times in a loop. I'm assuming it's a common tool that is used by other scripts or users, and you want to keep using it but make it faster. Just making one call to bc instead of 4 makes the script run in about 1/3 the time. Using shell ...

3

I know you said you wanted to stick to native applications, but GNU Parallel would allow you to execute separate processes in parallel, which would allow you to run this operation more quickly: sudo apt-get update sudo apt-get install parallel awk -F',' '{print $8}' file.csv | parallel -j+0 cdrtoip {} There are numerous ways of invoking parallel, but the ... 2 As John1024 points out, the big suspect for slowness is the calling of the cdrtoip 500,000 times. EDIT: based in the cdrtoip script provided, the entire implementation is in Python. It is much faster because there is no call to an external script. I would recommend you look at Python for this. The performance of Python is quite good for this type of task, ... 3 An awk (GNU) version awk '{printf ("%s ||| ",$0); getline < "fileB"; print $0 }' fileA With the getline command in awk, you can set$0 (all variables for columns) from next input record, if getline < "filename" you set the next $0 from the specified file. getline < "file" Set$0 from next record of file; set NF. Why your attempt ...

14

With POSIX paste: :|paste -d ' ||| ' fileA - - - - fileB paste will concatenate corresponding lines of all input files. Here we have six files, fileA, four dummy files from standard in -, and fileB. The list of delimiters include a space, three pipe and a space in that order will be used by paste circularly. For the first line of six files, fileA will ...

6

Well, this doesn't use sed, awk, or grep, but you can do it pretty easily in bash. The command is: (while IFS= read -r a <&3 && IFS= read -r b <&4; do echo "$a |||$b"; done) 3<fileA 4<fileB The problem with paste is that the delimiter is a single character. You could also insert a single character and the use sed to ...

2

{ printf '[13*%d-n[bs]pc]s%c\n' 9 a 7 b 5 c 3 d 1 e tr -s ' \n' lx <file.txt; } | dc | sed -f- -eb -e:s -e's/.*/\\textbf{&}/' exam.txt That uses the dc reverse-polish-notation calculator to generate a sed script that looks like: 8bs 17bs ...which is then concatenated with the sed scripts entered on the command-line and so results in ...

1

Using sed Let's create a test file: $seq 10 >file To move, say, line 5 to the top of a file:$ sed '1,4{H;1h;d}; 5{p;x}' file 5 1 2 3 4 6 7 8 9 10 How it works 1,4{H;1h;d} This saves lines 1 through 4 in the hold space. In more detail, H appends the current line to the hold space after appending a newline to the hold space. For line 1, we ...

1

The following pipeline should work well provided there are no backslashes in virtuser_NNN names: tbl=/etc/mail/virtusertable pw=/etc/dovecot/dovecot.passwd cut -d: -f1 "$pw" | grep -Fnf- "$tbl"| sed -e's|:.*/$$.*$$|s/:nouser/:\1/|' | sed -f- "$tbl" It will: first cut away all but the first sequence of any not-colon characters for each ... 0 sed -e:t -e'$!{1N;N;s/$$.*$$$$\n[^ ]*\1$$\{2\}$/\1\2/;tt' -e'P;D;}' <in >out ...that works. it recursively substitutes away the second in a series of three of input lines which it deems to be identical from the second space-delimited field on. it will continue to draw another input line to replace each it substitutes away until it can no longer do ... 2 I would do this in Perl. Save the email and associated username from the /etc/dovecot/dovecot.passwd in a hash and then replace in /etc/mail/virtusertable:$ perl -i -ape 'BEGIN{ open($fh,"/etc/dovecot/dovecot.passwd"); while(<$fh>){ @G=split(/:/); $k{$G[0]}=$G[2]; } } s/error:nouser ... 0 With awk one liner: awk '{n=$2$3$4$5$6$7}l1!=n{if(p)print l0; print; p=0}l1==n{p=1}{l0=$0; l1=n}END{print}' file The whole point is to manipulate few variables: n stores all fields except first in current line, l1 the same for previous line and l0 the whole previous line. The p is just a flag to mark if previous line was already printed.

0

A awk version: #!/usr/bin/gawk -f BEGIN { rep=0 ; prev="" ; OFS=FS=" "} !rep && prev== $2 { rep=1 ; prev=$2 ; next } rep && prev== $2 { time=$1 ; next } rep && prev!= $2 { rep=0 ; print time, prev; } { print ; ... 0 Perl to the rescue: perl -ne '($t, $r) = /([0-9]+\s+)(.*)/; print "$pt$p\n$_" if $r ne$p; $p =$r; $pt =$t; }{ print $t,$r' input-file \ | sort -nu | tail -n+2 -n reads the input line by line. $t is the timestamp plus whitespace,$r is the "rest". $p is the previous rest,$pt is the previous timestamp. ...

4

Your file starts with a UTF-8 byte order mark. It is unicode symbol U+FEFF which is encoded as three bytes in UTF-8. Those three bytes show up as 357 273 277 when you print them in base 8. To the sed command those bytes at the start of the line means that 1 is in fact not the first character on that line. Many other tools will treat it the same way. You ...

5

If you simply want to treat a string as a literal in sed there's already an answer for that: escaped_testx="$(sed -e 's/[\/&]/\\&/g' <<< "$TEST"; echo x)" escaped_test="${escaped_testx%x}" The extra x is to be able to handle trailing newlines, which would otherwise be removed by the command substitution. 0 In regular expressions you can escape characters by prepending a backslash. So / becomes \/. This will make your regex look hidious (s/\/home\/stuff\/dir1/\/home\/stuff/) but it works! 1 sed 's/\\\\/& /g;s/\\n/\ /g; s/\\\\ /\\\\/g' <in >out ...should handle the \newline replacements without misinterpreting backslash escapes - so you can still have a literal \\n in input. I'm a little fuzzy on how you can ever get a \n out of your history commands, though. I tried w/ zsh to find out how a \newline might be read out from the ... 5 sed -e'/./!d;$!G;/^Q/c\' -e'\\item' <in >out That will delete every blank line in input, Get a blank line out of hold space and append it to all non-blank lines which are ! not the $last, and change any pattern space ^beginning with the character Q to the one-line fixed-string \item on output. When run on your example input, the output is: \item ... 5 Another way with tr+sed: tr -s \\n <infile | sed '$!G;s/Question Nr.*/\\item/' tr squeezes all newlines and then sed appends hold space content (empty newline) to each line except the last one, replacing Question Nr.* with \item. With this method you won't be able to edit the file in-place. I chose tr here as it's faster then sed's regex (even if it's ...

4

Now with awk: awk '$1 ~ /[ABCDEM]/ {print$0"\n"} $1 ~ /Question/ {print "\\item"}' inputfile If the line starts with A, B, C, D, E or M (for Main), it prints that line and an extra \n. If the line starts with "Question", it simply prints \item. 5 If it doesn't need to be sed, Perl's "paragraph mode" is perfect for this. From man perlrun: -0[octal/hexadecimal] specifies the input record separator ($/) as an octal or hexadecimal number. [...] The special value 00 will cause Perl to slurp files in paragraph mode. [...] So, using -00 tells perl to define "lines" ...

6

With sed: sed 's/^Question Nr\..*/\\item/; s/^$$[A-Z] .*$$/\n\1/' file The first s/// replaces Question Nr. with \item similar to the sed command in your question. The second one replaces line that start with a capital letter from A to Z, but only one followed by a space. This whole line is replaced with itself \1 repending a newline \n. The output: ...

0

if there is no matter with performance, use this: cat PATH_OF_SOURCE_FILE | \ grep -n ^ | \ grep -E "^($(seq 1 90 | shuf | head -n 80 | paste -s -d '|')):" | \ sed 's/[0-9]*:$$.*$$$/\1/' > PATH_TO_TARGET_FILE first grep index lines; second grep select 80 random lines, and sed remove line number added by first grep. Note: pipe the last output to shuf ...

2

You're reimplementing history substitution. $mv /server/today/logfile1 /nfs/logs/ && gzip /nfs/logs/logfile1$ !!:gs/logfile1/logfile2 If you haven't actually executed the first command yet, and just want to execute a set of similar commands, consider a loop: for f in logfile logfile2; do mv /server/today/"$f" /nfs/logs && gzip ... 2 If you don't have GNU shuf, portably, you could do: awk -v n=90 -v p=10 ' BEGIN {srand()} rand() * n-- < p {p--; next} {print}' < file It's also going to be more efficient than the shuf+sed approach with high values of p since it's in o(n), while shuf+sed is in o(n*p). With n=1000000, the breaking point on my system is around p=35 with GNU sed ... -1 I think the challenge here is delete one of the 90 lines, then one of the remaining 89 lines, etc. -- we can't delete the 90'th line when only 89 remain. eval$(for i in {90..81}; do CMD="$CMD | sed$(( (RANDOM % $i)+1 ))d"; done; echo cat infile$CMD) > outfile The for loop accumulated a series of strings forming a pipeline in the form | sed NNd where ...

3

You probably wanted to use RANDOM % 90 rather then &. That's where the zeroes come from (deleting line 1 is OK, on the next run, the lines will be numbered 1 .. 89). There is a problem, though: The formula could generate the same number several times. To prevent that, use a different approach: shuffle the numbers and pick the first ten: shuf -i1-90 ...

2

You could pipe your command into a shell so it gets executed: echo "mv ..." | bash Or you could pass it as an argument to a shell: bash -c "$(echo "mv ...")" Or you could use the bash built-in eval: eval "$(echo "mv ...")" Note, however, that all of those code-generating commands look a bit brittle to me (there are ways they will fail as soon as ...

3

With GNU sed: sed 's/\f/\n/g' file Other seds: sed "$(printf 's/\f/\\\n/g')" file Or with tr: tr '\f' '\n' <file With perl: perl -pi -e 's/\f/\n/g' file The -i is to overwrite the file in-place. For in-place editing with tr: tr '\f' '\n' < file 1<> file 4 With sed alone: sed -i '/^* !^FROM_MAILER$/d' file To remove the whole line containing the exact string * !^FROM_MAILER, with nothing before and after that string. The d command deletes the line. Edit: If you want to do the replace in all files recusively, use the following: find /path -type f -exec sed -i '/^* !^FROM_MAILER$/d' {} + 0 you can use perl to do it: perl -p -i.bak -e 's/old/new/g' test.txt It's going to create a .bak file. Or without a .bak file : perl -pi -e 's/old/new/g' test.txt Or install sed-4.1.1 RPM from here. 0 You can do it with sed as well. echo$STRING | sed -e "s/$$PATTERN$$/[\1]/Ig"

0

If you don't mind Perl, it's: echo $STRING | perl -pe "s/($PATTERN)/[\1]/ig"

0

GNU sed does have an s///e command, but it sends the whole pattern space to the shell for evaluation: $echo "echo hello world" | sed 's/world/foo bar | rev/e' rab oof olleh So "world" is replaced by "foo bar | rev". Pattern space now is "echo hello foo bar | rev". This is sent to the shell, and the output is placed in pattern space which is then ... 1 You can select whanted parts but can remove unwanted instead: sed ' s|_[^_]* /[^/]*/|;| s|/[^/]*/$$...$$|;\L\1| s|?[^"]*" |;list;| s|/.*;|;retrieve;| s/ /;/ s/ .*$//'

Top 50 recent answers are included