Tag Info

New answers tagged

0

It can be easy done by awk awk ' NR==1{ print next } f!=$3{ if(NR!=2) print(line) printf("%s\t",$1) f=$3 } { line=$2 "\t" $3 } END{ print line } '


0

If the data are always in order (as you presented) then you can simply parse rows finding the first and last row in a "group" where the code is the same. For each group, save the "from" value for the first row and the "to" value for the final row (before the code changes). Print those groups using the values you've saved.


0

From man awk: A numeric expression is converted to string by replacing expr with sprintf(CONVFMT, expr), unless expr can be represented on the host machine as an exact integer then it is converted to sprintf("%d", expr). Sprintf() is an AWK built-in that duplicates the functionality of sprintf(3), and CONVFMT is a built-in variable used for internal ...


1

You are saying that only the literal "x y z" lines after PATTERN shall be replaced. awk ' /PATTERN/ { f=1 } f && /x y z/ { print $0, 1 ; next } { print } ' If it's lines with three arbitrary fields then use awk ' /PATTERN/ { f=1 } f && NF==3 { print $0, 1 ; next } { print } '


1

The looping can help: awk '/PATTERN/{print;getline;while(!/lineN/){$(NF+1)=" 1";print;getline}}1' Or to extend @Janis idea: awk '/lineN/{f=0}f{$(NF+1)=" 1"}/PATTERN/{f=1}1'


2

you almost make it awk '(NR>1) && ($8 > 2 ) ' foo > bar where NR is number of record (that is number of line) $8 is eight field && is logical and foo is the original file, unchanged bar resulting file implicit default action is to print the current input line note that header is striped from foo to bar, to keep it awk ...


3

With zsh: setopt extendedglob # best in ~/.zshrc extract_numbers() REPLY=${(SM)REPLY##<->-<->} for file (*<->-<->*.dat(no+extract_numbers)) { do-what-you-will-with $file } We define a sorting function (extract_numbers) that returns the Substring of the file name that Matches <->-<-> (<-> matching any positive ...


0

This command will give the filename of only one file sorted by the numeric fields, assuming the pattern is Sample_nnnnnnnnnn-ppp_xyz.dat: ls Sample_$(ls -l *.dat | awk -F ' |_' {'print $10'} |sort -n| head -1)*.dat


1

You can just do a simple sort. Assuming your file names contain no whitespace or other strange characters, this command will always give you the oldest file: find . -name '*dat' | sort -r | tail -n1 If you need to deal with strange characters, use find . -name '*dat' -print0 | sort -zr | tr '\0' '\n' | tail -n1 So, you could do something along the ...


1

assuming all files do not begin with Sample numeric field are sperated by _ you can't relay on file time I would use sort -t_ -k2n which yield for the four files given in sample Sample_0000137294-003_xyz.dat Sample_0000237826-001_xyz.dat Sample_0000237826-002_xyz.dat Sample_0000437564-001_xyz.dat


1

#!/bin/bash ls /tmp/*dat > filelist while read fn do #do something with /tmp/$fn done < filelist


3

To use awk builtin's: awk -F, -v field="Flag10" '{sub(field",.*",field);print split($0,any)}' doc.file The script will remove everything after the first occurrence of the value of field (in this case, Flag10) and then print the result of splitting the line on commas. Since we've deleted everything after Flag10, this results in the number of fields left ...


2

I typically create an associative array, mapping the header names to the column numbers awk -F, -v head="Flag10" ' FNR == 1 {for (i=1; i<=NF; i++) col[$i] = i} {print $col[head]} ' << END ,Flag2,,Flag4,Flag5,,,,,Flag10,Flag11,Flag12,Flag13 10,20,30,40,50,60,70,80,90,100,110,120,130 END Flag10 100


3

You could do like below, awk -F, '{for(i=1;i<=NF;i++)if($i=="Flag10")print i}' file Example: $ echo ',Flag2,,Flag4,Flag5,,,,,Flag10,Flag11,Flag12,Flag13' | awk -F, '{for(i=1;i<=NF;i++)if($i=="Flag10")print i}' 10


1

Assuming your data is as simple as you show, you could do awk -F, -v OFS=, '{k=$3; $3="\"new text\""; $4=k}1' file Or, perl -F, -lane 's/$F[1]/$F[1],"new text"/; print' file However, this will fail with data that can contain nested commas like: "foo", "bar,baz", "bar" The above is a valid csv file but these solutions will treat it as having 4 ...


2

To do not involve some heavy instruments (like sed or awk): paste -d',' <(cut -d',' -f-2 file) column.file <(cut -d',' -f3- file) This assumes that the column you want to add is saved in the file column.file.


0

With GNU sed you can do: sed -E 's/(\|[^ |]+) /\1| / s/(([^|]* )?([^|]*))\|(([^ ]*)(.*))/\1\6\n\2\4/ /\n/P;D' <infile ...which prints... TMPRSS2 AADAT Sample1 pp9284 AADAT Sample1 ERG TMPRSS2 Sample2 ERG pp9284 Sample2 TMPRSS2 ETV1 Sample3 pp9284 ETV1 Sample3 PDE4A MIA Sample4 PDE4A MIA-RAB4B Sample4 PDE4A RAB4B Sample4 ...


4

With bash: # important to use parentheses, not braces, to localize changes to IFS # the variable is purposefully unquoted split_pipe() ( IFS='|'; echo $1 ) while read -r first second third; do for word1 in $(split_pipe "$first"); do for word2 in $(split_pipe "$second"); do echo $word1 $word2 $third done done done < file


4

$ awk '{split($1,a,"|"); split($2,b,"|"); for (i in a) {for (j in b) print a[i],b[j];}}' file TMPRSS2 AADAT pp9284 AADAT ERG TMPRSS2 ERG pp9284 TMPRSS2 ETV1 pp9284 ETV1 PDE4A RAB4B-EGLN2 PDE4A EGLN2 PDE4A MIA PDE4A MIA-RAB4B PDE4A RAB4B To also print the third (SampleN) field, you can just add $3 to the print statement inside the loops, i.e. $ awk ...


1

Here are some approaches. You haven't explained what you want to do with the string you want to extract so I will just print it. awk with multiple delimiters echo Sample_0000860156-001-out_20150224150524.xml.gz | awk -F '[_-]' '{print $2"-"$3}' GNU grep echo Sample_0000860156-001-out_20150224150524.xml.gz | grep -oP '_\K.*(?=-)' sed echo ...


0

Using GNU awk with match awk 'match($0,/_([0-9]+-[0-9]+)/,a){print "file."a[1]".gz"}' <<< "$FILENAME" Ouptut file.0000860156-001.gz


1

This will create a new empty file for every *.gz on the directory that keeps the pattern of the example: touch $(ls *.gz | awk -F '_' {'print $2'} | awk -F "-out" {'print $1'}) or as suggested by Jidder : touch $(ls *.gz | awk -F '_|-out' {'print $2'})


2

You can say: $ awk '($4>7 && $4!=swap) {swap=$4; print $1, $4}' file 00:20:02 7.3 00:45:02 10.5 01:05:02 13.5 That is, store the last value of SWAP bigger than 7 in the swap variable and keep comparing the value. You can get a nicer output by setting the output field separator to tab: -v OFS="\t" or BEGIN {OFS="\t"}. Or you can also pipe to ...


0

Sounds like you want something like: awk 'BEGIN {RS="\n\n"} /What exactly is Free/' br.txt This should print the entire paragraphs where the text is found (RS="\n\n").


0

Through python3 #!/usr/bin/python3 import csv import sys file1, file2 = sys.argv[1], sys.argv[2] with open(file2) as second, open(file1) as first: second_list = second.readlines() first_list = first.readlines() for line1 in first_list: for line2 in second_list: if line1.split(':')[1] == line2.split(':')[1]: ...


2

With sed you can probably do: sed 's|[^:]*:\([^:]*\).*|/^[^:]*:\1:/s/$/:&/;t|' file2 | sed -f - file1 ...which would involve one sed process reading the second file and writing a sed script for editing the first into a second sed's stdin. As near as I can tell you shouldn't have any problem with directly injecting the contents verbatim into a regexp ...


8

This sounds like a task for join: join -t":" -o "1.1,1.2,1.3,1.4,1.5,2.1,2.2,2.3" \ -j 2 <(sort -k2,2 -t: test1) <(sort -k2,2 -t: test2) Output: Julian:Brude:Other:Other:Other:Jb:Brude:kemin Robert:Dillain:Other:Other:Other:R:Dillain:bodent Megan:Flikk:Other:Other:Other:Mb:Flikk:kentin Jesus:Kimmel:Other:Other:Other:Jbb:Kimmel:verlin ...


5

This is simple task for awk: awk -F':' -vOFS=':' 'NR==FNR{a[$2]=$0;next}{print $0,a[$2]}' file2 file1 First we set : as field separator both for input (with -F) and output (with OFS) then if first file is processed (file2) we assign whole line to table element indexed with second field. When next next file (file1) is processed we print its lines adding ...


0

One way to do this could involve unexpand. The description for it and the expand utility can be found here: The unexpand utility shall copy files or standard input to standard output, converting <blank> characters at the beginning of each line into the maximum number of <tab> characters followed by the minimum number of <space> characters ...


0

Liked @Costas suggestion, and another option is: gawk ' { f1=substr($0,2,2) f2=substr($0,4,36) gsub(/ *$/, "", f2) print f1 " " f2 } '


1

If the columns are separated by tabs you can specify the tab character as the field separator. This will prevent the default behavior of awk to treat spaces as separate columns. cat <data file> | awk -F"\t" '{print $1, $2}' root@ubuntu32:/tmp# cat testtext | awk -F"\t" '{print $1, $2}' 16 SQL*Plus 16 TOAD background query session


0

Convert it to a csv file and use a csv tool such as csvcut from the csvkit: $ cat test-cols.dat x y1 y2 y3 ei1 ei2 ei3 es1 es2 es3 1 4 5 4 7 7 2 4 7 7 2 7 3 3 3 8 3 3 3 8 3 2 1 4 4 9 6 4 4 9 Install csvkit: $ pip install csvkit Use tr with its squeeze option -s to convert it ...


0

Try: $ cat test01 | stdbuf -oL -eL awk -F',' '{print $2","$3 >> "data"$1".csv"; fflush("")}' In gawk 4.0.1, calling fflush() without any argument only flush stdout. You need calling fflush() with empty string "" to make awk flushes all open output files and pipes as well. With gawk 4.0.2 and later, if there is no argument, or if the argument ...


1

Just to clarify my original post: I agree with Orion and doubt that this is an awk bug. I also wouldn't bother with fflush. I think that this is just an awk problem caused by data. Specifically I think you may have too many open file descriptors - one for each date in your file. Or perhaps you've reached a buffer limit for the same reason - too many open ...


0

You can tell socat to associate a pty with the awk program; this forces awk to run in line buffered mode and you get the immediate responses you're looking for. Here's the command line to listen on port 9000: socat TCP4-LISTEN:9000 SYSTEM:'/tmp/awk.sh',pty,echo=0 And here's the contents of the script /tmp/awk.sh: #!/usr/bin/awk -f # BEGIN { print ...


2

Awk is printing to standard output, which is line buffered when it points to a terminal, and fully buffered if not; this means you should get output after it fills the buffer, ie it has printed (eg) 4KB. You can test this by sending a lot of output (eg using the yes program); to fix you should either: - unbuffer awk's stdout (see eg ...


0

You could try using sed : sed -rn ':a;/^([^~]*~){13}[^~]*$/!{N;s/\n//;ba};p' yourfile.tsv What it does The script has three parts separated by a ;: :a defines a label we can branch to /^([^~]*~){13}[^~]*$/!{N;s/\n//;ba} searches for a complete column /^([^~]*~){13}[^~]*$/ looks for a line with exactly 14 fields (0 or more occurrences of anything that ...


0

You don't need external languages, a few standard commands can do the trick. For example, to insert something at line 5 of origin.txt you could do: tmpfile=`mktemp` head -n4 origin.txt > $tmpfile echo "I'm line 5 now" >> $tmpfile tail -n +5 origin.txt >> $tmpfile mv $tmpfile origin.txt


0

sed "s/\($(printf "[0-9][0-9]%s" '' - - '\) \(' : : '')\)/\1T\2/g" Maybe try building out your pattern programmatically. That can sometimes make it a little easier to nail down the very specific types. The above expands to... sed 's/\([0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\) \([0-9][0-9]:[0-9][0-9]:[0-9][0-9]\)/\1T\2/g' (but without the ...


1

This is much simpler than the linked question. All you need is: awk -F, -v OFS=, 'NR==FNR{a[$1$2$3]=$4; next}{print $0,a[$1$2$3]}' file1 file2 Explanation -F, : set the input field separator to a comma. -v OFS=, : set the output field separator to a comma. This is useful to print comma-separated output by default. NR==FNR : NR is the current line ...


2

awk -F',' ' # start awk and use comma as a field separator FNR == NR { # if processed so far number of rows in current file if equal to overall processed number of rows do things in block {} if (FNR == 1) {next} # if it is first row then continue (skip to next row) a[$1] = $2; # create an array indexed with first field, with value of ...


-1

Just do this: du -shcl /directory/*


0

I think this gets what you want: for n in 3 0 do nl -w1 -s."$n" <&"$n" done 3<file1 <file2 | sort -t\| -unk2,2 -nk3,3 | sort -t\| -nk1,1 | grep '^[^|]*0|' ...which in my tests printed... 4.0|151244127834|7177638920385| 2|2015-02-04 07:14:22|2015-02-04 07:18:48 5.0|151244127834|7177638924073| 3|2015-02-04 07:18:40|2015-02-04 ...


1

awk -F'|' 'FNR==NR{a[$2]=$2;b[$3]=$3;next};{if($2 in a){print $0 > "new_file_1"};if(($2 in a )&& !($3 in b)){print $0 > "new_file_2"}} file1 file2 details ... {if($2 in a){print $0 > "new_file_1"} : if SID in file2 matches SID in file1 redirect to a file called new_file_1 .. if(($2 in a )&& !($3 in b)){print $0 > ...


1

awk -F'|' ' FNR==NR{ cid[$2]=1 sid[$3]=1 next } { if (!($2 in cid)) print else if (!($3 in sid)) print }' file1 file2 Brief explanation: Build two unique arrays (cid and sid values) from 1st file. Than compare it to 2nd and 3rd fields (the 1st field is counted from beginning of line to first ...


1

using sed as follows sed -n '/^ *<Person> *$/,/^ *<\/Person> *$/{/<Name>/{d};p}' file|md5sum target all lines in *<Person> *$/,/^ *<\/Person> *. /<Name>/ check to see if the range has the tag "Name" and then delete this line. Afterwards print the lines ranges and pipe to md5sum


0

Fill in the variab les outside the scope of awk (close ' and open again): cat log | awk '$1~/'${a}'|'${b}'/ {print }'


0

Through python3 using re module. #!/usr/bin/python3 import re import sys j = sys.argv[1] with open(j) as f: fil = f.read() m = re.findall(r'(?s)(?:^|\n)([^,]*,([^,]*),[^,]*,([^,]*)[^\n]*).*?\n([^,]*,\2,\3,[^\n]*)', fil) for i in m: print(i[0]+'\n'+i[-1]) Save the above code in a file named script.py and execute this file by running the below ...


3

Try the following awk-script: awk ' /<Person>/,/<\/Person>/{ if (! /<Name>/) a=a s $0 s="\n" } /<\/Person>/{ system("echo \""a"\" |md5sum - ") a=s="" }' input.xml Explanation: Collect all lines between tags Person into a variable When meet closing tag /Person call system command than empty a ...


0

Also with GNU wc (coreutils 8.4), it can handle multiple files wc -L *.txt lists the length of each individual file as well as provide the longest length of all files



Top 50 recent answers are included