Tag Info

New answers tagged

0

You are pretty close. You don't need the comma or the $ before your variable. Try: awk '{if (NF == 4) {print $0; name=$1} else {print name$0}}'


2

If you join two files, it's a job for join: join -1 1 -2 1 -a 1 -o 1.1 -o 1.2 -o 2.2 -e "0" FileA.txt FileB.txt Where: -1 1 -2 1 defines the fields to join (in both files the 1st) -a 1 to force join to print the unpairable lines from FileA.txt -o 1.1 1.2 2.2 is the output format and -e "0" defines the value to store in an empty field The output: A2M ...


0

It is unclear what you exactly want to achieve but here is a simplified version that might suit your needs: awk ' BEGIN { ORS = "$" } /^N° du cas:/,/^@@@/ { sub("N° du cas: ","\n") if($1!="@@@") print } { next } END { printf("\n") printf("Number of records processed : %d",NR) }' With you sample input, its output is: ...


3

awk 'BEGIN { ... } # the next line should NOT be within curly braces $1 ~ /^Observation/, $1 ~ /^@@@/ { ... } { ... } END{ ... }' input.txt > out.csv


1

Use awk: awk '$2" "$3=="FOR COLUMN"{a=$1; $1=$4; $4=a}1' file first compare the values if the second (FOR) and third field (COLUMN) if it matches, swap the first ($1) and the fourth ($4) field


1

Some shells, including bash, allow you to type any literal character (e.g. Ctrl-I for Tab, Ctrl-M for Return) by prefixing it with Ctrl-V, so you could type Ctrl-V Ctrl-I instead of \t wherever it appears in your sed one-liner.


0

not an answer, but too long for a comment x=3 while [ $x -le 8 ] do find . -name "S_${x}x${x}*" -print | sort -t'_' -nk3 | xargs -J {} awk 'FNR == 1 && !/^($|#)/ { sum+=$1) } END{ print sum }' {} >> cS_CPU.txt x=$(( x+1 )) done need for -F' ' ? awk will separate field by space ...


2

You are asking the wrong question, or asking the question wrongly and in the wrong stack, this is a better question to ask in the programming/stack-overflow for people to give you answers based on the algorithms used inside awk and sort. PS: also do the needed with nawk, mawk and gawk to give us some more details to "zone into" ;) and do the runs like a 100 ...


0

As can be seen, - also works: getline < "-" Or getline variable < "-"


0

You may use python3. with open('/path/file') as f: print('START') for line in f: print(line, end = "") print('END')


6

If you're already using sed, you can use 1 to match the first line and $ to match the last line (see Scott's answer). If you're already using awk, you can use a BEGIN block to run code before the first line and an END block to run code after the last line (see Michael Durrant's answer). If all you need to do is add a header and a footer, just use echo and ...


8

This can be done in sed with sed -e $'1i\\\nSTART' -e $'$a\\\nEND' 1i means insert before line 1; $a means append after the last line.  The $'…' syntax is bash-specific.  In other shells, you should be able to do this with: sed -e '1i\Enter START' -e '$a\Enter END'Enter


12

This works, as indicated by jasonwryan: awk 'BEGIN{print "START"}; {print}; END{print "END"}'


7

With awk: awk ' /^end/ { sub(" ", "", indent) } # Or { indent = substr(indent, 3) } { print indent, $0 } /^describe/ { indent = indent" " } ' <file


0

Change it like this: cat input | while read p; do { echo -n "$p "; lynx -dump "www.xyz.com/p?=$p"; } | awk '$2~/Hello/{print $1" "$3}' done Before the first pipe, you can group the commands using command grouping in the form { list; }. The advantage is redirections may be applied to the entire command list. The redirected output looks then like this: ...


1

Assuming your file names don't contain newline characters and your grep supports the -o option: find [[:upper:]] -type f | grep -Eo '^./[0-9]{3}' | sort | uniq -c


0

This will do what you've asked: for d in */* do n=$(find "$d" -maxdepth 1 -type f -printf "ok\n" | wc -l) printf "%s\t%d\n" "$d" $n done The primary differences to your code are that I've used find instead of ls so that weirdly named files won't break the count (think of \n in a filename), and that I've used printf to format the output.


0

Perhaps something like: find $PWD/ -type f -printf "%h\n\0" | uniq -zc Yield something like: 7 /foo 17 /foo/bar 9 /foo/baz Add a | sort -z before | uniq to sort it.


0

The simplest way to get what you want is to use the -n option to echo when you print the directory name. That avoids printing a newline at the end of whatever you're echoing, so the next output stays on the same line. Other languages may be a better option if you want to collect the information you're looking for and then run multiple transformations on it ...


0

You can built an array of files for each directory separately, and then just count the number of elements. In bash that would be something like for dir in */*/; do a=( "$dir"/* ); printf "%s\t%s\n" "$dir:" "${#a[@]}"; done If A/001 etc. contains directories too which content you would like to include, then add ** glob: shopt -s globstar for dir in */*/; ...


0

Please don't use awk sed etc. They cannot handle XML properly. XML does a bunch of stuff like having whitespace, linefeeds, unary tags etc. that means regular expressions aren't very robust - they break messily, following a perfectly valid change to XML down the line. The way to handle XML is with a parser. xmlstarlet is one commonly used on Linux. Because ...


-1

The speed difference is because 'sort' is a command (link), whereas 'awk' is a programming language (link). 'sort' command is takes input and return output. Whereas 'awk' is a programming language, which first interprets the code (terminal command) then starts processing on it. Simple as that.


3

Do you have to use awk for this? The paste utility was designed exactly for this sort of thing. Assuming array is a shell array: array=(100 200) printf "%s\n" "${array[@]}" | paste -d, input.csv - > output.csv The printf is simply to put each array member on a new line. paste then pastes together input lines from input.csv and - (i.e. the piped ...


0

The problem is that shell variables (including arrays) are not available inside awk. You need to pass them explicitly via -v option. Moreover you cannot pass the whole array, but you can put array into single variable and split it inside awk: awk -va="$(echo "${array[@]}")" 'BEGIN{OFS=FS=","; split(a,b," ")}{print $0,b[NR]}' input.csv This will work as ...


1

I've found that sort seems to be the fastest uniq tool as shown here --> Fastest way to delete duplicates in large wordlist


1

That's not correct. When you cat the files and send them to the pipe, it's no longer counting the files separately, it's just getting all its input from the pipe as a single file.


3

Use this instead: awk -F, '{print FNR}' file1 file2 The FNR variable in awk gives the number of records for each input file. But, when you use cat .. | awk awk reads for the stdin file descriptor, therefore awk sees only 1 "file". Try this to understand better (FILENAME contains the current file being processed): $ awk -F, '{print FILENAME" "FNR}' file1 ...


3

You concatenate (cat) them together, so for awk it is just one file (or standard input in this case). If you want to have desired result run instead awk -F, '{print FNR}' File1 File2


0

I'd suggest to write some script/small program that employs XML parser. Then you can count records as they getting parsed and filter out only the stuff you need.


1

The following awk script (plus column for output tabulation) will hande any sequence of placement of the sub-tags, and any whitespace separation of the tags - ie. it will handle the OP's sample input format, as well as the following sample which has no whitespace and differently ordered sub-tags: <HARDWARE><OS>Windows ...


0

with awk - arbitrarily set each column to be 15 characters long, left-aligned and filled with spaces: awk ' BEGIN { FS = "<[A-Za-z/]+>" } { if ( NR % 6 == 0 ) { printf"\n" } else if ( $2 != "" ) { printf"%-15s", $2 } }' file Or as in the other answers in combination with column awk ' BEGIN { FS = "<[A-Za-z/]+>" } { if ( NR % 6 == 0 ) { ...


6

With a slight modification to your XML, wrap all your XML in a parent <DATA> tag1, or another one of your choosing, file called data.xml: <DATA> <HARDWARE> <NAME>WIN1</NAME> <OS>Windows 7</OS> <IP>1.2.3.4</IP> <DOMAIN>contoso.com</DOMAIN> </HARDWARE> <HARDWARE> ...


-2

du path_to_your_files/*.jpg | awk '{ total += $1 }; END { print total }'


2

With your example and GNU sed: sed -n 's/<[^>]*>//g;s/^ *//g;/./p' file | paste -d ";" - - - - | column -t -s ";" Output: WIN1 Windows 7 1.2.3.4 contoso.com WIN2 Windows 8 10.20.30.40 contoso.com I assume that your file does not contain a ;. If you need a CSV remove | column -t -s ";".


3

The long|many terms better put into array: awk -F, 'BEGIN{a[1000]="A";a[1001]="B";a[1002]="C"}$4!=a[$2]' file


1

use grep grep -v '.1000.*.A\|.1001.*.B\|.1002.*.C' input filename and your awk will be corrected as below awk -F, '{if(($2==1000 && $4!="A") || ($2==1001 && $4!="B") || ($2==1002 && $4!="C"))print $0}' inputfilename


2

You can do that with egrep: egrep -v '.*,((1000,.*,A)|(1001,.*,B)|(1002,.*,C))' filename Will match all lines that don't have 2nd column with 1000 and 4th with A, 2nd with 1001 and 4th with B or 2nd with 1002 and 3rd with C.


5

Use this: awk -F, '($2==1000 && $4!="A") || ($2==1001 && $4!="B") || ($2==1002 && $4!="C")' file In the curvy brachets are the 3 conditions; if one of them applies the line will be printed. The conditions inside the brackets are connected with a AND, so both must apply.


2

This works with mawk: awk 'NR==1{$7="G";print;next} \ $3~/^[A,C,G,T]$/ || $4~/^[A,C,G,T]$/ {$7="P"} \ $3~/^[I,D,R]$/ || $4~/^[I,D,R]$/ {$7="Q"} \ $4~/[A-Z][A-Z]/ || $3~/[A-Z][A-Z]/ {$7="Q"} 1' file line: In the first line write the G in the header. line: If $3 of $4 are A, C, G or T then $7 is P. line: If $3 of $4 are I, D, or R then $7 is Q. line: ...


0

Here's a sed solution using a sliding window (so there are never more than four lines in the pattern space at a time): sed '1{N;N};$!N;/.*\n.*\n.*Fail.*\n.*/{s/^/#/;s/\n/&#/g};P;D' infile On first line it reads in the Next two lines (so now there are three lines in the pattern space). Then, for each input line (including the first one) it pulls in the ...


0

I would have used cibi_seperator="awk" # general case, use plain awk if [ "$SYSTEM_OS" = "SunOS" ]; then cibi_seperator="nawk" elif [ "$SYSTEM_OS" = "fooOS" ]; then cibi_seperator="/bin/special/awk" elif ... fi This way, cibi_seperator is always set, test is done only for OS that need special awk (like solaris).


1

Found out the problem: I set the parameter: SYSTEM_OS=`uname -a` before the if statement. Whereas, I should have set it to: SYSTEM_OS=`uname -s` Sorry for the inconvenience.


0

case "$SYSTEM_OS" in (SunOS) cibi_separator=nawk;; (Linux) cibi_separator=awk;; (*) echo "[$SYSTEM_OS] unsupported"; exit 1;; esac ... $cibi_separator -F, 'NR==1,NR==2{print $0;}' cibi.csv > section_header.csv


4

On a GNU system, you can use this: sed -i '/^#[[:blank:]]Person/{n;s/#root:[[:blank:]]\+marc/root:\tsomeone@something.tld/;}' file It searches for a line beginning with # Person. Then switches to the next line and replaces #root:<blanks>marc with root:<tab> .... The -i flag edits the file inplace. -i, \+ and \t are GNU extensions. The ...


0

$ sed -r 'H;1h;$!d;x; s/\n([^\n]*)\n([^\n]*)\n([^\n]*Fail[^\n]*)\n/\n#\1\n#\2\n#\3\n#/g' file Name Number Reason = Pass Reasult Name Number Reason = Pass Reasult #Name #Number #Reason = Fail #Reasult Name Number Reason = Pass Reasult #Name #Number #Reason = Fail #Reasult Name Number Reason = Pass Reasult How it works H;1h;$!d;x These commands read the ...


1

I would suggest to use perl: perl -p0e 's/(.*\n)(.*\n)(.*Fail\n)/#\1#\2#\3#/g' file Here is how it works: -p: print program in the loop over all input lines -0: assume null as record separator -e: execute program from the command line s/x/y/g: substitute y for x anywhere in the file (): group together regular expressions .*: any character except newline ...


2

If you like to get the current time_t, this can be done by calling srand() and then call t = srand(). $ awk 'BEGIN{srand(); print srand()}' 1440536144 This works as posix requires the random function to be initialized with the current time if called without arg and to return the previous seed.


4

Many tools can be handy: -n of grep is exactly what you are looking for. grep -n 'bla' file alternatively awk: awk '/bla/{print NR":"$0}' file alternatively perl: perl -ne 'print $.,":",$_ if /bla/' file alternatively sed: sed '/bla/!d;=' file |sed 'N;s/\n/:/'


3

uniq is the correct tool for that: uniq -D -f2 file Where: -D prints all dublicates -f2 to avoid to compare the first 2 fields Edit: If the fields 7 and above are not to be compared, you need awk: awk 'n=x[$3,$4,$5,$6]{print n"\n"$0;} {x[$3,$4,$5,$6]=$0;}' file The array item x[] (columns 3-6) is checked. If it's already set run the part in {...} ...


0

For sed lovers — read file1 for each line in file2 it is not good idea. Much better to use variable: my_var=$(<file1) ; sed "s/$/ $my_var/" file2 or more secure read -r my_var <file1 ; sed "s/$/ $my_var/" file2



Top 50 recent answers are included