Tag Info

Hot answers tagged

12

With a newer version (4.x) of GNU awk: awk 'BEGIN {FPAT="[0-9]+"}{s+=$1}END{print s}' With other awks try: awk -F '[a-z=]*' '{s+=$2}END{print s}'


12

If your grep support -o option, you can try: $ grep -o '[[:digit:]]*' file | paste -sd+ - | bc 784 POSIXly: $ printf %d\\n "$(( $(tr -cs 0-9 '[\n*]' <file | paste -sd+ -) ))" 784


10

If you have root permissions on that machine you can temporarily increase the "maximum number of open file descriptors" limit: ulimit -Hn 10240 # The hard limit ulimit -Sn 10240 # The soft limit And then paste res.* >final.res After that you can set it back to the original values. A second solution, if you cannot change the limit: for f in ...


8

If chaos' answer isn't applicable (because you don't have the required permissions), you can batch up the paste calls as follows: ls -1 res.* | split -l 1000 -d - lists for list in lists*; do paste $(cat $list) > merge${list##lists}; done paste merge* > final.res This lists the files 1000 at a time in files named lists00, lists01 etc., then pastes ...


7

With awk you can do: awk '{print >out}; /XYZ/{out="file2"}' out=file1 largefile Explanation: The first awk argument (out=file1) defines a variable with the filename that will be used for output while the subsequent argument (largefile) is processed. The awk program will print all lines to the file specified by the variable out ({print >out}). If the ...


7

Using awk, split the file using tabs and output the first field in full and the first 75 characters (at most) of the second: awk -F "\t" 'BEGIN { OFS=FS }; { print $1, substr($2, 1, 75); }' As pointed out by fedorqui, you can handle files with more than two fields by replacing the fields you need to truncate: awk -F "\t" 'BEGIN { OFS=FS }; { ...


7

Unless you have joined lines, the \n doesn't appear in sed's pattern space: the end-of-line anchor is $. So with GNU sed: sed -i 's/$/|/' temp.txt


6

With GNU sed: sed -i -e 's/^/ /' <file> will replace the start of each line with 5 spaces. The -i modifies the file in place, -e gives some code for sed to execute. s tells sed to do a subsitution, ^ matches the start of the line, then the part between the second two / characters is what will replace the part matched in the beginning, i.e., ...


6

With a modern ksh here's a shell variant (i.e. without sed) of one of the sed based answers above: { read in <##XYZ ; print "$in" ; cat >file2 ;} <largefile >file1 And another variant in ksh alone (i.e. also omitting the cat): { read in <##XYZ ; print "$in" ; { read <##"" ;} >file2 ;} <largefile >file1 (The pure ksh ...


6

Try this: pcregrep -M '\bThis\s+is\b' <<EOT This is an example file. EOT


6

The GNU grep can do it grep -z 'is\san\sexample\sfile.' file To fulfill some points which arise in comments there are some modifications to script: grep -oz '^[^\n]*\bis\s*an\s*example\s*file\.[^\n]*' file Regarding huge files I have no imagination of memory limitation but in the case of problem you are free to use sed sed '/\bis\b/{ :1 ...


5

You can use sed sed 's_^_ _' tmpin > tmpout Or awk awk '{print " " $0}' tmpin > tmpout Or paste (Thanks cuonglm) :| paste -d' ' - - - - - file Watch out. These can be addictive. You can solve many simple problems, but the time will come where you need to upgrade to a full scripting language. Edit: Sed script simplified based on Eric ...


5

You can use tr: LC_ALL=C tr -d '[:blank:]\n' < file_in > file_out Since when you have to work with 10k files, a better solution would be: find . -type f -exec perl -i.bak -pe 's/ |\t|\n//g' {} +


5

{ sed '/XYZ/q' >file1; cat >file2; } <infile With GNU sed you should use the -unbuffered switch. Most other seds should just work though. To leave XYZ out... { sed -n '/XYZ/q;p'; cat >file2; } <infile >file1


5

This is a job for csplit: csplit -sf file -n 1 large_file /XYZ/ would silently split the file, creating pieces with prefix file and numbered using a single digit, e.g. file0 etc. Note that using /regex/ would split up to, but not including the line that matches regex. To split up to and including the line matching regex add a +1 offset: csplit -sf file ...


5

You wrote in your last block, linux$ paste temp2 temp > temp2 You cannot do this. (Well you can, but it won't work.) What happens here is that the shell truncates temp2 ready to send output from the paste command. The paste temp2 temp command then runs - but by this stage temp2 is already zero length. What you can do instead is this, which uses a ...


5

You can use back-references: echo A627E39B | sed 's/\(.\)\(.\)/\2\1/g' This finds all occurrences of two characters and swaps them. As glenn jackman pointed out, enabling extended regular expressions (-r on GNU sed, -E on BSD) avoids having to escape the parentheses: echo A627E39B | sed -r 's/(.)(.)/\2\1/g'


5

I'd use Perl's paragraph mode: pactl list sink-inputs | perl -00ne 'print if s/(.*?VLC.*?\n).*/$1/ms' The -00 sets the input record separator to \n\n so a "line" is a paragraph. Then, the substitution will match everything until the first VLC and then anything until the 1st newline and save them as $1. Everything after that is removed (since we're ...


5

With ed: ed -s <<'IN' r !pactl list sink-inputs /VLC/+,$d ?Sink Input?,.p q IN It reads the command output into the text buffer, deletes everything after the first line matching VLC and then prints from the previous line matching Sink Input up to current line. With sed: pactl list sink-inputs | sed -n 'H;/Sink Input/h;/VLC/{x;p;q}' It appends ...


5

Another GNU awk one: awk -vRS='[0-9]+' '{n+=RT};END{print n}' A perl one: perl -lne'$n+=$_ for/\d+/g}{print$n' A POSIX one: tr -cs 0-9 '[\n*]' | grep . | paste -sd+ - | bc


4

Sort on keys 1 to 32 (-k1,32), with vertical bar as delimiter -t'|'. -u means: output only the first line of an equal run (of combined keys). See man sort for details of sort's options. sort -t'|' -k1,32 -u infile The above code, of course, results in a sorted output. However, if you want to maintain the same sequence of first-found lines as per the ...


4

You can use many standard tools, example with paste: :| paste -d' ' - - - - - file or shorter with awk: awk '{$1=" "$1}1' file or more portable with perl: perl -pe '$_=" "x5 .$_' file


4

Some more choices. I have saved your example text in file for simplicity. grep and PCREs: $ grep -oP '(GRAPE|FRUIT)=\K.*?(?=,)' file purple yes violet affirmative To get them on the same line, just parse. For example $ grep -oP '(GRAPE|FRUIT)=\K.*?(?=,)' | paste -d" " - - – purple yes violet affirmative sed $ sed ...


4

Try: sed -n 'h;n;p;n;G;p' < file.in > file.out For example: $ seq 9 | sed -n 'h;n;p;n;G;p' 2 3 1 5 6 4 8 9 7


4

Portably/POSIXly with sed: tab=$(printf '\t') sed "s/\($tab[^$tab]\{0,75\}\)[^$tab]*/\1/" Or to truncate every column: sed "s/\([^$tab]\{75\}\)[^$tab]*/\1/g"


4

With the subsequent awk program: awk ' BEGIN { FS = "([[:space:]]+|[|])" } NR==FNR { id[$1] ; next } /^>>/ && $2 in id { data = $2 ; f = 1 } f && /^[0-9]/ { data = data " : " $10 " " $11 } f && !NF { f = 0 ; print data } ' FileB FileA you will get this output: AT5G46880.1 : 317 ...


4

awk -F= '{sum+=$2};END{print sum}'


4

Try using awk: awk -F'"' '{ print $2 }' conf.txt


3

awk '{$1=$1;print}' or shorter: awk '{$1=$1};1' Would trim leading and trailing space or tab characters and also squeeze sequences of tabs and spaces into a single space. That works because when you assign something to one of the fields, awk rebuilds the whole record (as printed by print) by joining all fields ($1, ..., $NF) with OFS (space by ...


3

Answering your question literally, here's one way to list the last PID displayed by lsof: lsof … | awk 'END {print $2}' Awk is a text processing language which reads input and processes it line by line. In the code, END {…} executes the code in the braces after the whole input is processed, and effectively operates on the last line. $2 is the second ...



Only top voted, non community-wiki answers of a minimum length are eligible