Tag Info

Hot answers tagged

14

awk 'NR==1 {n=$2} { file = sprintf("file.%.4d", ($2-n)/10000) if (file != last_file) { close(last_file) last_file = file } print > file }' Would write to file.0000, file.0001... (the number being int(($2-n)/10000) where n is $2 for the first line). Note that we close files once we've stopped ...


13

Use this: nl file | shuf -n2 | sort -n | cut -f2- nl to number the lines, shuf to shuffle and limit the output to 2 lines (-n), sort to rebuild the original order, and cut to remove the numeration of nl. It will print 2 lines of your file in the original order of the file. Use shuf -n X, where X can be any number.


12

This works, as indicated by jasonwryan: awk 'BEGIN{print "START"}; {print}; END{print "END"}'


11

The error is giving the filename to process in the script; you should remove employee.txt from the script and run it as follows awk -f awktest1.awk employee.txt or even, if the script is executable, ./awktest1.awk employee.txt The script becomes #!/bin/awk -f BEGIN{print "start"} {print $2, "\t", $5} END{print "end"} As it is, awk is waiting for ...


11

There are no built-in functions in standard awk to get a date, but the date can easily be assigned to a variable. awk -F, -v date="$(date +%Y-%m-%d)" '$3>date' or in an awk script BEGIN { str = "date +%Y-%m-%d"; str | getline date; close str; } $3>date gawk, does have built-in time functions, and strftime can be used. gawk -F, ...


9

Set OFS as well: awk -F, -v OFS=, '{$4=$4/1024}1' The OFS determines how output fields are delimited. If you don't set it, the default is a space.


9

To complement @Stephen's answer, if you wanted to hardcode the name of the input file in the awk script, you'd write: #!/bin/awk -f BEGIN{ ARGC=2 ARGV[1] = "employee.txt" OFS = FS = "\t" print "start" } {print $2, $5} END{print "end"} For several files: #!/bin/awk -f BEGIN{ ARGC=1 ARGV[ARGC++] = "employee1.txt" ARGV[ARGC++] = ...


9

awk -F, '!( 1001 <= $10 && $10 <= 1010 )' file


8

Here is a portable way : java -version 2>&1 | PATH=`getconf PATH` awk -F '"' '/version/ {print $2}' Unlike the usual suggestions that try to guess the correct location depending on the Unix implementation, it uses the getconf PATH command that returns the path to POSIX compliant commands.


8

Hack one-liner version. Perhaps more suitable for Code Golf than this forum though. This generates split1, split2, split3 and so on, as filenames. awk '{if($2>b+9999){a++;b=$2}print >"split" a}' file.txt To have output files named split001, split002, split003, involves this extra sprintf: awk '{if($2>b+9999){a++;b=$2}print ...


8

This can be done in sed with sed -e $'1i\\\nSTART' -e $'$a\\\nEND' 1i means insert before line 1; $a means append after the last line.  The $'…' syntax is bash-specific.  In other shells, you should be able to do this with: sed -e '1i\Enter START' -e '$a\Enter END'Enter


7

With awk: awk ' /^end/ { sub(" ", "", indent) } # Or { indent = substr(indent, 3) } { print indent, $0 } /^describe/ { indent = indent" " } ' <file


6

You can use awk string function substr(): awk -F, -vOFS=, '$9 = substr($1,3,1)' <file


6

The right tool for this job is nl: nl -w2 -s'> ' file You may want to tune width option according to the total number of lines in the file (if you want numbers to be aligned nicely).


6

With a slight modification to your XML, wrap all your XML in a parent <DATA> tag1, or another one of your choosing, file called data.xml: <DATA> <HARDWARE> <NAME>WIN1</NAME> <OS>Windows 7</OS> <IP>1.2.3.4</IP> <DOMAIN>contoso.com</DOMAIN> </HARDWARE> <HARDWARE> ...


6

If you're already using sed, you can use 1 to match the first line and $ to match the last line (see Scott's answer). If you're already using awk, you can use a BEGIN block to run code before the first line and an END block to run code after the last line (see Michael Durrant's answer). If all you need to do is add a header and a footer, just use echo and ...


5

The selection of a random line from a file without sorting (or even knowing how many lines there are!) is covered in "The Art of Computer Programming", Volume 2, Section 3.4.2 by Donald E. Knuth. This is trivial to implement, e.g.: (echo foo; echo bar; echo zot) \ | perl -nle 'rand $. < 1 && ( $line = $_ ); END { print $line }' Or try shuf, ...


5

If you are not too fussy about white space, you do it with two awk rules like this: awk '$1 { printf "\n" $0 } !$1 { printf OFS $7 }' FS=, OFS='|' infile This leaves an empty line at the head of the output, and misses one at the end. To fix this add if and END clauses: awk '$1 { if(NR>1) printf "\n"; printf $0 } !$1 { printf OFS $7 } END { printf ...


5

Use this: awk -F, '($2==1000 && $4!="A") || ($2==1001 && $4!="B") || ($2==1002 && $4!="C")' file In the curvy brachets are the 3 conditions; if one of them applies the line will be printed. The conditions inside the brackets are connected with a AND, so both must apply.


4

Here is one method: $ v='\n*******************************************************************************************************************************************************************\n' $ awk -v ORS="$v" '{print;}' file ...


4

You can define functions in awk like: awk -F'[-,]' ' function abs(v) {return v < 0 ? -v : v} {print abs(360*($4-$1)+30*($5-$2)+($6-$3))}'


4

This has been answered before elsewhere on Stack Overflow. http://stackoverflow.com/questions/15361632/delete-a-column-with-awk-or-sed http://stackoverflow.com/questions/7551219/deleting-columns-from-a-file-with-awk-or-from-command-line-on-linux etc.. I believe awk is the best for that. awk '{print $1,$2,$3,$4,$5,$7}' file It is possible to use cut as ...


4

Or if you don't want to pass the filename employee.txt as a command line argument, you can do something like this: Code: #!/bin/awk -f BEGIN{ print "start" while((getline <"employee.txt") > 0){ {print $2, "\t", $5} } print "end" }


4

If your data is expressed in fixed width columns, you could do: For the first case: sed 's/^.\{4\}$/& -9/' (add " -9" to lines of 4 characters). For the second case: sed -e '/.\{11\}/b' -e 's/$/ /;s/\(.\{10\}\).*/\1-9/' (add up to 10 spaces and -9 to lines of less than 11 characters). Generally, to parse lines with fixed width fields, ...


4

You could do it with a while loop and $RANDOM like: while read line; do if ((RANDOM%2)); then echo $line; fi; done < _path_ which will print about half the lines, which you could control with the condition in the if


4

#!/bin/bash first=$( head -n1 file | awk -F" +" '{print $2}' ) last=$( tail -n1 file | awk -F" +" '{print $2}' ) for (( i=$first ; i<=$last ; i=i+10000 )) ; do awk -v start=$i -v end=$(($i+10000)) 'BEGIN { FS == " +" } { if ( $2 >= start && $2 < end ) print $0 }' file \ >> interval_"$i"_to_"$(( $i+10000 ))" done Test with ...


4

You can use FNR to get current line number: awk '{print "\"Process " FNR "\" - " $2}' <in >out


4

sort -k1,1 -u file Sort file by first column and take first unique entry only.


4

The simplest method is to use awk's output redirection. Awk output redirection is very easy to use in simple cases: the file is opened the first time a redirection is used, and subsequent redirections to the same file name use the existing file descriptor. If you wanted to add a suffix to the file name, it would be as easy as find -type f -iname "*.txt" ...


4

Another awk: $ awk 'BEGIN{getline l <"file1"};{print $0, l}' file2 1 2 3 12 4 5 6 12 7 8 9 12 BEGIN block was executed first before reading input file. The first line in file1 was retrieve using getline() function, stored in variable l With each line of file2, we print it content $0 along with l, separated by OFS, which is a space by default.



Only top voted, non community-wiki answers of a minimum length are eligible