Tag Info

Hot answers tagged

10

Use this: sed 's/|END|/\n/g' test.txt What you attempted doesn't work because sed uses basic regular expressions, and your sed implementation has a \| operator meaning “or” (a common extension to BRE), so what you wrote replaces (empty string or END or empty string) by a newline.


8

Your awk syntax is a little wrong. #!/bin/bash awk -F: -v keyword="$1" '$1 == keyword {$1=$1; print}' myfile.csv The trick here is reassigning the value of one of the fields forces awk to recalculate $0 using the output file separator. Here, the default OFS is a space, so assigning the value of $1 to itself changes the colons to spaces. A non-awk way to ...


7

Using sed, based on Famous Sed One-Liners Explained, Part I:: 39. Append a line to the next if it ends with a backslash "\" (except here we ignore the part about the backslash, and replace the \n newlines with the required | separator): sed -e :a -e '$!N; s/\n/ | /; ta' mydoc > mydoc2 should produce in mydoc2 134.27.128.0 | 111.245.48.0 | ...


7

The following worked fine for me: $ sed 's/|END|/\ /g' foobar T|somthing|something T|something2|something2 Notice that I just put a backslash followed by the enter key.


7

I would agree with you - it probably is a generic problem. Some common utilities have some facilities for handling it, though. nl nl, for example, separates input into logical pages as -delimited by a two character section delimiter. Three occurrences on a line all alone indicate the start of a heading, two the body and one the footer. It replaces any ...


6

You can use awk: awk 'FNR!=1{print l}{l=$0};END{ORS="";print l}' ORS=' | ' file > new_file ORS=' | ' set the output record separator to ' | ' instead of newline. or edit in-place with perl: perl -pe 's/\n/ | / unless eof' file


6

I doubt it will make a difference but, just in case, here's how to do the same thing in Perl: perl -ne 'print if ++$k{$_}==1' out.txt If the problem is keeping the unique lines in memory, that will have the same issue as the awk you tried. So, another approach could be: cat -n out.txt | sort -k2 -k1n | uniq -f1 | sort -nk1,1 | cut -f2- How it works: ...


5

Assuming that the first column is strictly ordered: $ awk -F, '$1==last {printf ",%s",$2;next} NR>1{print""} {last=$1;printf "%s",$0} END{print""}' file 1,cat,dog 2,apple 3,human Alternatively, allowing the input lines in any order (and output lines in no guaranteed order): $ awk -F, '{a[$1]=a[$1]","$2} END{for (i in a)print i a[i]}' file 1,cat,dog ...


5

You can use awk: $ awk -F'\\|END\\|' '{$1=$1}1' OFS='\n' file T|somthing|something T|something2|something2 -F'\\|END\\|' set field separator to |END| OFS='\n' set ouput field separator to newline $1=$1 cause awk reconstruct $0 with OFS as field separator 1 is a true value, causeawk print the whole input line


5

An awk based solution could be: awk '/^#define PF_BUILD_VERSION / {$3++} 1' infile >outfile && mv outfile infile


5

A Perl approach: perl -pe 's/^#define PF_BUILD_VERSION \K(\d+)/$1+2/e' file > newfile Or, to edit the file in place: perl -i -pe 's/^#define PF_BUILD_VERSION \K(\d+)/$1+2/e' file


5

Just using bash: string="H08W2345678" echo "${string:3}" W2345678 echo "${string:0:-4}" H08W234 See the Wooledge wiki for more on string manipulation.


4

I prefer the variant of not changing the existing data , but adding the sort criteria as new column, and removing that auxiliary sorting field at the end of the pipe: awk -F, 'BEGIN {u["kg"]=1000; u["g"]=1}; {print $1*u[$2], $0}' file | sort -n | cut -d" " -f2-


4

If your file is too large to hold in memory, you could do: $ awk -F, -v OFS="," '$2=="kg"{$1=1000*$1}1;' file | sort -n | awk -F, -v OFS="," '$2=="kg"{$1=$1/1000}1;' 1000,g,dog 1,kg,cat 20,g,apple


4

In Vim, you could limit your substitution to the lines that contain NX: :g/NX/s/N1/NX/ Preceding the substitution with /NX/ makes Vim perform it only on the next line that contains NX (using ranges), and using :g makes it run on all lines that match NX.


4

#!/usr/bin/perl use DB_File; tie %h, 'DB_File'; while(<>){ not $h{$_} and print and $h{$_}=1 } EDIT 1: Does it really work? (comparing) Sol1 : Terdon et all Schwartzian-transform-like one-liner cat -n _1 | sort -uk2 | sort -nk1 | cut -f2- Sol2 : perl + DB_File (this answer) perl dbfile-uniq _1 Sol3 : PO (John W. Gill solution has a ...


4

One possibility is to do this with the vim text editor. It can pipe arbitrary sections through shell commands. One way to do this is by line numbers, using :4,6!nl. This ex command will run nl on the lines 4-6 inclusive, achieving what you want on your example input. Another, more interactive way is to select the appropriate lines using line-selection mode ...


4

In-place sed requires making a backup file during the process. The -i option on Apple's sed requires an extension argument (for the backup file it creates) and consumes the next argument. That means you're telling it you want it to make a backup file with the extension "#s</head>#...". The error means it thinks you're referring to the append command. ...


4

This counts the number of ones and zeros in filename: $ sort <filename | uniq -c 5 0 5 1


3

Your Awk pattern is failing because the word "profile" does not start the record, [profile does... awk '/^\[profile/ {gsub(/]/,""); print $2}' test.txt dev prod Another approach would be to load an array, using split: awk -F'[][]' '/profile/ {p=split($2,profiles," "); print profiles[2]}' test.txt dev prod


3

2 thoughts: with sed, for any line that ends with a carriage return, join the next line sed '/\r$/ {N; s/\r\n//} ' file with awk, define the record separator for input and output: awk -v RS='\r\n' -v ORS='' 1 file


3

If there is only one foo3 in line sed -n '/foo3=/{s/.*foo3=//;s/\S*=.*//;p}' file.txt Suppress printing any line (-n options) exept which pushed by p. For lines which consists foo3=: Exchange everything before foo3= with it included (.*foo3=) to nothing (//). Remove everything which starts with some(*) non-space (\S) symbols with =. Prints resedue ...


3

I was curious to see how some of these (+ some alternatives) work speed-wise with a rather large file (163MiB, one IP per line, ~ 13 million lines): wc -l < iplist 13144256 Results (with sync; echo 3 > /proc/sys/vm/drop_caches after each command; I repeated the tests - in reverse order - after a couple of hours but the differences were negligible; ...


3

Another possibly awk command and using its RS option would be: awk '$1=$1' RS="\|END\|" file Will print those records (based on awk's Record Separator) which are not empty( has at least one field) to prevent printing empty lines. Tested on this input: T|somthing|something|END|T|something2|something2|END| Test|END| |END| Gives this output: ...


3

find . -name PKA.dump -type f -exec awk ' FNR == 20 {print; nextfile}' {} + nextfile, where available (GNU awk and some others like FreeBSD's and recent versions of mawk and soon to be added to the standard) will skip to the next file. Where not, it will be ignored (it's just like dereferencing a nextfile variable); it will still work but read the files ...


3

sed ' s/.*/YES(&)/;:t s/([^()]*)//g;tt s/.....*/NO/'


3

grep -fB A will do what you're looking for; the -foption specifies a file from which patterns are loaded, one pattern per line. Any line in A which matches a pattern in B will be output.


3

grep with extended regex: ping ... | grep --line-buffered -E 'time=([0-9]{4}|[5-9][0-9]{2})' | ... This will match any line with time= followed immediately by either: Any 4 digits (1000+ ms). One digit 5-9 followed by any 2 digits (500-999 ms). --line-buffered works around issues caused by grep's default output buffering when used in a pipe chain. Not ...


3

$ echo "H08W2345678" | sed 's/^.\{3\}//' W2345678 sed 's/^.\{3\}//' will find the first three characters by ^.\{3\} and replace with blank. Here ^. will match any character at the start of the string (^ indicates the start of the string) and \{3\} will match the the previous pattern exactly 3 times. So, ^.\{3\} will match the first three characters. $ ...


3

My solution using join: join -a1 -a2 -1 1 -2 1 -o 0,1.2,2.2 -e "NULL" file1 file2 I don't know much about awk for joining large files and always use join. key1 11 NULL key2 12 22 key3 13 23 key4 NULL 24 key5 NULL 25



Only top voted, non community-wiki answers of a minimum length are eligible