Tag Info

New answers tagged

0

Hoary old chestnut of an awkidiom awk '{ORS=NR%2?",":"\n";print}' file Station Name,Station Code A N DEV NAGAR,ACND ABHAIPUR,AHA ABOHAR,ABS ABU ROAD,ABR


2

If you're willing to take all the digits of the pid you could use pgrep to do this for you: pgrep -d " " -f ^/usr will print a space separated list of all the PIDs of processes whose command starts with /usr If you really want to use the ps command, awk would be a better tool than sed to do what you want. You could do it, with just the first 5 ...


0

Check this one out. Join every 2 lines in a file. http://www.theunixschool.com/2012/03/join-every-2-lines-in-file.html


0

The original file is actually Newline delimited JSON (NDJSON). Instead of using GNU tools, I recommend installing json from NPM. # say the file is test.log $ json -f test.log -g [ { "a": 1, "b": 1, "c": 2 }, { "a": 3, "b": 3, "c": 1 }, { "a": 1, "b": 2, "c": 3 } ]


0

Pure bash: while read line do echo "$line," done < <(head -n-1 testfile) echo `tail -n1 testfile` This script appends ',' to every line except the last, and then prints the last line unchanged.


2

You can use sed: sed '$n;s/$/,/' file The $n; means if it's the last line just continue. Else it adds a , at the end of the line. If you add the -i flag (not POSIX) the file will be edited inplace. If it must be awk: awk 'NR>1{print p} {p=$0","}END{print}' file1 Stores the content of the line in a variable p. The variable is printed, except in ...


0

How about in perl: #!/usr/bin/perl use strict; use warnings; use JSON; my $json_list; while ( my $json_str = <> ) { push ( @$json_list, from_json ( $json_str ) ); } print to_json ( $json_list, { pretty => 1, canonical => 1 } ); Takes data on STDIN or myscript.pl somefilename and takes the input you've designated, and builds a JSON ...


1

You can use awk: awk 'NR > 1{print line","}{line=$0;}END{print $0}' jsonfile


2

Do you have to use awk for this? The paste utility was designed exactly for this sort of thing. Assuming array is a shell array: array=(100 200) printf "%s\n" "${array[@]}" | paste -d, input.csv - > output.csv The printf is simply to put each array member on a new line. paste then pastes together input lines from input.csv and - (i.e. the piped ...


0

The problem is that shell variables (including arrays) are not available inside awk. You need to pass them explicitly via -v option. Moreover you cannot pass the whole array, but you can put array into single variable and split it inside awk: awk -va="$(echo "${array[@]}")" 'BEGIN{OFS=FS=","; split(a,b," ")}{print $0,b[NR]}' input.csv This will work as ...


3

Content of file: username:email@email.com:password:salt user:name:email@email.com:password:salt username:email@email.com:password:sa:lt user:name:email@email.com:password:sa:lt With GNU sed: sed -E "s/(.*):([^:]*@[^:]*):([^:]*):(.*)/\1', '\2', '\3', '\4/" file Output: username', 'email@email.com', 'password', 'salt user:name', 'email@email.com', ...


1

That's not correct. When you cat the files and send them to the pipe, it's no longer counting the files separately, it's just getting all its input from the pipe as a single file.


3

Use this instead: awk -F, '{print FNR}' file1 file2 The FNR variable in awk gives the number of records for each input file. But, when you use cat .. | awk awk reads for the stdin file descriptor, therefore awk sees only 1 "file". Try this to understand better (FILENAME contains the current file being processed): $ awk -F, '{print FILENAME" "FNR}' file1 ...


3

You concatenate (cat) them together, so for awk it is just one file (or standard input in this case). If you want to have desired result run instead awk -F, '{print FNR}' File1 File2


3

For your sample input: $ cat /tmp/data | 2015-08-21 - 10:15 | jones | view | Main.Home | | 172.29.192.106 | | 2015-08-21 - 10:31 | wilson | view | Main.Home | | 172.19.6.107 | | 2015-08-21 - 11:40 | smith | resetpasswd | wilson | Mozilla | 172.19.15.105 | | 2015-08-21 - 11:41 | james | view | Main.ChangePassword | | 172.19.15.102 | | 2015-08-21 - 11:41 ...


0

Easiest would be to tie the regex to some other field, e.g. :[0-9][0-9] | wilson, though this risks false positives. More complicated would be instead to use awk and split the data into columns, then apply the regex in awk against only the second column.


3

The long|many terms better put into array: awk -F, 'BEGIN{a[1000]="A";a[1001]="B";a[1002]="C"}$4!=a[$2]' file


1

use grep grep -v '.1000.*.A\|.1001.*.B\|.1002.*.C' input filename and your awk will be corrected as below awk -F, '{if(($2==1000 && $4!="A") || ($2==1001 && $4!="B") || ($2==1002 && $4!="C"))print $0}' inputfilename


2

You can do that with egrep: egrep -v '.*,((1000,.*,A)|(1001,.*,B)|(1002,.*,C))' filename Will match all lines that don't have 2nd column with 1000 and 4th with A, 2nd with 1001 and 4th with B or 2nd with 1002 and 3rd with C.


5

Use this: awk -F, '($2==1000 && $4!="A") || ($2==1001 && $4!="B") || ($2==1002 && $4!="C")' file In the curvy brachets are the 3 conditions; if one of them applies the line will be printed. The conditions inside the brackets are connected with a AND, so both must apply.


2

This works with mawk: awk 'NR==1{$7="G";print;next} \ $3~/^[A,C,G,T]$/ || $4~/^[A,C,G,T]$/ {$7="P"} \ $3~/^[I,D,R]$/ || $4~/^[I,D,R]$/ {$7="Q"} \ $4~/[A-Z][A-Z]/ || $3~/[A-Z][A-Z]/ {$7="Q"} 1' file line: In the first line write the G in the header. line: If $3 of $4 are A, C, G or T then $7 is P. line: If $3 of $4 are I, D, or R then $7 is Q. line: ...


0

Here's a sed solution using a sliding window (so there are never more than four lines in the pattern space at a time): sed '1{N;N};$!N;/.*\n.*\n.*Fail.*\n.*/{s/^/#/;s/\n/&#/g};P;D' infile On first line it reads in the Next two lines (so now there are three lines in the pattern space). Then, for each input line (including the first one) it pulls in the ...


0

$ sed -r 'H;1h;$!d;x; s/\n([^\n]*)\n([^\n]*)\n([^\n]*Fail[^\n]*)\n/\n#\1\n#\2\n#\3\n#/g' file Name Number Reason = Pass Reasult Name Number Reason = Pass Reasult #Name #Number #Reason = Fail #Reasult Name Number Reason = Pass Reasult #Name #Number #Reason = Fail #Reasult Name Number Reason = Pass Reasult How it works H;1h;$!d;x These commands read the ...


0

Another gnu sed one-liner: $ sed -rz 's/\n([^\n])/ \1/g;s/\n /\n/g' matrices.txt 1 2 3 4 2 3 4 5 3 4 5 6 3 4 5 6 2 3 2 5 2 3 4 5 2 3 5 6 2 3 4 5 $ Because the -z effectively causes sed to read the whole file as one line, then this will be less suitable for large input files.


1

I would suggest to use perl: perl -p0e 's/(.*\n)(.*\n)(.*Fail\n)/#\1#\2#\3#/g' file Here is how it works: -p: print program in the loop over all input lines -0: assume null as record separator -e: execute program from the command line s/x/y/g: substitute y for x anywhere in the file (): group together regular expressions .*: any character except newline ...


1

sed only: sed '/^$/!{ # if line isn't empty H # append to hold space $!d # if it's not the last line, delete it b end # branch to label end (this happens only if on the last line) } //b end # if line is empty, branch to label end : end # label end x # ...


2

If you like to get the current time_t, this can be done by calling srand() and then call t = srand(). $ awk 'BEGIN{srand(); print srand()}' 1440536144 This works as posix requires the random function to be initialized with the current time if called without arg and to return the previous seed.


4

Many tools can be handy: -n of grep is exactly what you are looking for. grep -n 'bla' file alternatively awk: awk '/bla/{print NR":"$0}' file alternatively perl: perl -ne 'print $.,":",$_ if /bla/' file alternatively sed: sed '/bla/!d;=' file |sed 'N;s/\n/:/'


2

You want to change these "string"s which are in lines where there are no # character or this character is after "string", so that you can have comments at the end of the lines: ##################################### # Blah blah blah string blah blah ##################################### PKG_NAME="string" PKG_DESC="string-foo" PKG_A="string" # this is ...


2

In sed you can stop doing actions if a pattern is found with an exclamation mark: sed '/#/!s/string/replacement/' file I.e. for lines matching '#' (at any position) do not do the replacement - else do. In your case result is: ##################################### # Blah blah blah string blah blah ##################################### ...


2

In Vi/Vim you can run simply: :%j to join all the lines together, or: :%v/^$/-1j to join all matrices separated by new line (Join lines between a certain text pattern in Vim). If you need this done from the command line, try either: ex -s +%j +"wq modified_matrices.txt" matrices.txt to join all lines, or: ex -s +'%v/^$/-1j' +'wq! ...


1

With tr and sed: Replace all newlines with hashes (pick any other symbol that is not present in your matrices if you have hashes), then double hashes are to be newlines and single ones just spaces: (GNUsed) tr '\n' '#' <file | sed 's/##/\n/g;s/#/ /g' POSIX sed tr '\n' '#' <file | sed 's/##/\ /g;s/#/ /g' Regarding the size of the file: ...


4

Possible awk solution could be: awk 'BEGIN { RS = ""; } { $1 = $1; } 1' matrices.txt > modified_matrices.txt


1

You can do that with a little bash script: $ cat data 1 2 3 4 2 3 4 5 3 4 5 6 3 4 5 6 2 3 2 5 2 3 4 5 2 3 5 6 2 3 4 5 $ cat foo.sh #!/bin/bash while read line; do if [[ "${line}" = "" ]]; then echo "" else echo -n "${line} " fi done echo "" $ bash foo.sh < data 1 2 3 4 2 3 4 5 3 4 5 6 3 4 5 6 2 3 2 5 2 3 4 5 2 3 5 6 2 3 4 ...


1

using awk awk ' BEGIN { FS="[|\" ]+" ; OFS="|" } { print $2,$3,$4,$5,$6,$7,$8"|" } ' file Explanation: BEGIN { FS="[|\" ]+" ; OFS="|" } first set the following: FS="[|\" ]+": fields are separated by any combination (+) (zero or more of any) of the set ([]) pipe, double quotes (need to be escaped) and space |\". OFS="|" separate the output fields with ...


1

Try this: sed -e 's/["| ]\+/|/g' -e 's/^|//' < file The first expression will replace each block containing one or more |, ", or spaces with a single |. The second will remove the | at the start of each line.


3

You could translate all spaces and double quotes to | (and squeeze) then cut from the 2nd character to the end of line: tr -s '[[:blank:]"]' \| <infile | cut -c2-


1

Assuming your data is in a file named 'data': sed -e s'/^"//' -e 's/|" "|/|/g' -e 's/" "|/|/g' -e 's/" "/|/g' -e s'/"$/|/' data


1

sed -i 's/\"//g' filename You can escape the " character by putting a \. In case you want to remove all spaces as well, do the following: sed -i 's/[" ]//g' filename


0

For sed lovers — read file1 for each line in file2 it is not good idea. Much better to use variable: my_var=$(<file1) ; sed "s/$/ $my_var/" file2 or more secure read -r my_var <file1 ; sed "s/$/ $my_var/" file2


4

Another awk: $ awk 'BEGIN{getline l <"file1"};{print $0, l}' file2 1 2 3 12 4 5 6 12 7 8 9 12 BEGIN block was executed first before reading input file. The first line in file1 was retrieve using getline() function, stored in variable l With each line of file2, we print it content $0 along with l, separated by OFS, which is a space by default.


0

$ sed -e "s/ *\$/ $(cat file1)/" file2 1 2 3 12 4 5 6 12 7 8 9 12


2

Using awk $ awk -v n=$(cat file1) '{print $0,n}' file2 1 2 3 12 4 5 6 12 7 8 9 12 On csh/tcsh, try: awk -v n=`cat file1` '{print $0,n}' file2 How it works -v n=$(cat file1) This assigns the contents of file1 to the awk variable n. print $0,n This prints each line followed by n. Using sed $ sed '1{h;d}; G;s/\n/ /' file1 file2 1 2 3 12 4 5 6 12 7 ...


0

This should work (in this case file2 gets updated, if you don't want that, remove the -i flag) sed -i "s/.*/& $(cat file1)/" file2


0

If you want to replace all numbers that are just 0.xxxx with 0, you could also use sed: sed 's/\(^\| *0\)[\.][^ ]*/\1/g' Where we look for beginning-of-line (^) or one or more spaces (*) followed by a 0 and save that pattern to be printed later followed by a literal . and any number of characters that aren't a space (so, FYI, 0.ignore would get matched) ...


0

With very large files it is often best to sort the data so the rest is easier and doesnt need any memory. sort is designed to handle very large data sets. The following removes the unwanted columns, sorts the data, then awk just suppresses the duplicated columns without needing to remember more than one line. The final sort is optional, to get the indexes ...


0

You can use awk to build an array (in memory) as you read the file – then END-process that array to output as required. However many fields remain after removing the first 2 and last 3 fields from a record, become the key to the array. awk -F, '{ ix=$NF sub( "[^,]+,[^,]+,","") # delete first 2 fields ...


0

try this if works awk -F, '{a[$3$4$5$6$7$8]++;if(a[$3$4$5$6$7$8] > 1)k[$3$4$5$6$7$8]=k[$3$4$5$6$7$8]","$11;else k[$3$4$5$6$7$8]=$3","$4","$5","$6","$7","$8":"$11}'END'{for(i in k) print k[i]}' data I have doubt on line 4596,87,cat,val1,val2val3,val4,val5,08,234,index6 there is no comma(,) between val2val3?? it this correct??


6

sed 's:\\1\\:\ :g' file you will have to escape the backslashes in your match pattern. In the replacement, that's a backslash followed by a literal newline character. Some sed implementations, like GNU sed also support \n there as an non-standard alternative. Output "evSchema" "UAT" "evSN" "uadb" "evDirsep" "/" "evRootPath" "/work_area/APP_UAT/" ...


2

With POSIX awk: awk '!(FNR%3==1)' <file With POSIX sed: sed -e '1d;n;n;d' <file With GNU sed: sed -e '1~3d' <file



Top 50 recent answers are included