Tag Info

New answers tagged

0

You can compile this c++ code for quite quick results. It completes in around 0.19 - 0.27 seconds on a 1000 line file. It currently reads 10000 lines into memory(to speed up printing to screen) which if you had 1000 characters per line would use less than 10mb memory which i wouldn't think would be a problem. You can remove that section completely though ...


2

You could also use the fmt command: ~$ cat f PAPER TEAM MANISH NISHA GARIMA JYOUTI ~$ fmt -1 f PAPER TEAM MANISH NISHA GARIMA JYOUTI


1

Using a for loop: for val in `cat file1` ; do echo $val >> file2; done;


2

You could simply do this through grep. By default grep, would print the match in a separate newline . grep -oP '\S+' infile > outfile OR grep -o '[^[:space:]]\+' infile > outfile


2

You can also do this using sed: $ sed -e 's/[ ]\+/\n/g' file1 > file2 NOTE: Doesn't handle the situation where the words contain spaces.


0

Python In python i would do like the below, #!/usr/bin/python import re import sys file = sys.argv[1] with open(file, 'r') as f: for line in f: print ' '.join(re.findall(r'(?<!\d)\d{4}(?!\d)', line)) Save the above script as script.py and then run it by applying the below command on the terminal. python script.py file Example: $ cat ...


3

Using tr, replace each repeated space character() with a single new-line(\n) character. tr -s ' ' '\n'< infile > outfile I think you want something like this? 1 2 3 4 1 a # a b c d --> 2 b $ # $ @ % 3 c @ 4 d % WRD=$(head -n 1 infile|wc -w); for((i=1;i<=$WRD;i++)); do awk '{print $'$i'}' infile| tr ...


3

Using awk, setting the output field separator (OFS) as the record (line) separator (RS): awk '{OFS=RS;$1=$1}1' file > file2


6

This will give you all lines whose 9th column matches MEF2: awk -F"\t" '$9~/MEF2/' file > output Assuming your file is always tab-delimited, this will work and you can rest safely. That's as close to 0 margin of error as you'll ever get. If, however, you have tried importing into something like R (presumably using read.table("file",sep="\t")) and ...


1

With a single s/// you can do this in a POSIX sed - but there are a lot of backslashes: sed 's/[^0-9]*\([0-9]\{5,\}\)*[^0-9]*\([0-9]\{4\}\)*.\{0,1\}/ \2/g ' <<\IN 92828 Hello 8473 World War 1 1914-1918 Hello 8391 World War 2 1939-1945 IN OUTPUT 8473 1914 1918 8391 1939 1945 It globally gobbles all of the sequences with 5 or more digits first - ...


0

with gawk echo '9228 Hello 8473 World War 1 1914-1918 Hello 8391 World War 2 1939-1945' | awk --re-interval -v RS='\\y[[:digit:]]{4}\\y' '{printf "%s ", RT}END{print ""}' 9228 8473 1914 1918 8391 1939 1945


4

You can use perl. perl -nle'print join " ", /(?<![0-9])[0-9]{4}(?![0-9])/g' This also works for multi-line input, so if you have: 9228 Hello 8473 World War 1 1914-1918 Hello 8391 World War 2 1939-1945 You'll get the following returned: 9228 8473 1914 1918 8391 1939 1945 (Add -0777 if you want the numbers on the same line.)


2

With perl: $ perl -Tnle 'BEGIN {$, = " "} print $_ =~ /\b\d{4}\b/g' file 9228 8473 1914 1918 8391 1939 1945


3

POSIXly: < file tr -cs 0-9 '[\n*]' | grep -xE '.{4}' | paste -sd ' ' -


7

I can answer with grep command: Input file: 9228 Hello 8473 World War 1 1914-1918 Hello 8391 World War 2 1939-1945 Command: grep -Eo '\<[0-9]{4}\>' file |tr '\n' ' ' Return any number with length=4. -E switches to extended regex -o print only the matching part Output: 9228 8473 1914 1918 8391 1939 1945 Update answer: Input file: 9228 ...


2

Similar to @terdon's answer, but with sed: { seq -s, 10; seq -s, 5; seq -s, 15; } | tee - - That's my infile - it looks like: 1,2,3,4,5,6,7,8,9,10 1,2,3,4,5,6,7,8,9,10 1,2,3,4,5,6,7,8,9,10 1,2,3,4,5 1,2,3,4,5 1,2,3,4,5 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 I can then do: sed ...


1

This will add the number of (comma separated) fields to the beginning of each line, print the line and then sort everything: awk -F"," '{print NF,$0}' *csv | sort -nk1,1 The -n is numerical sort and the -k1,1 ensures that it is only sorted on the first field. To remove the number of fields after sorting, use: awk -F"," 'print NF,$0' *csv | sort -nk1,1 | ...


2

Think this is what you want Add -F, for comma separated. E.g awk -F, '$(NF+1)=NF' file Add Number to end of rows awk '$(NF+1)=NF' file Input 1 1 2 3 1 2 1 2 3 4 5 6 a b Output 1 1 1 2 3 3 1 2 2 1 2 3 4 5 6 6 a b 2 Sort Rows awk '{a[NF]=a[NF]?a[NF]"\n"$0:$0;x=x<NF?NF:x}END{for(i=1;i<=x;i++)if(i in a)print a[i]}' Input 1 1 2 3 1 2 1 2 3 4 ...


1

Another alternative (piping two tr commands): ls -l partition | cut -c5-7 | tr -dc rwx | tr rwx cse


31

Yes, we see a number of things like: while read line; do echo $line | cut -c3 done Or worse: for line in `cat file`; do foo=`echo $line | awk '{print $2}'` echo whatever $foo done (don't laugh, I've seen many of those). Generally from shell scripting beginners. Those are naive literal translations of what you would do in imperative languages ...


2

Alternative way without ls: getfacl -c partition | sed -n '/group::/{s/.*:://;y/rwx/cse/;s/-//g;p;}'


3

I think you want this command: ls -l partition | cut -c5-7 | tr rwx cse |sed 's/-//' You can remove the one extra command(cut -d ' ' -f 1) and replace it with your last cut command(cut -c5-7) and also add sed 's/-//' at the end to remove all -s. Now you are done. you didn't need to adding extra |. And even better: you can also change the dash(- ...


1

Using Python’s standard library: >>> ipaddress.ip_address('192.168.1.1').reverse_pointer '1.1.168.192.in-addr.arpa'


7

As far as conceptual and legibility goes, shells typically are interested in files. Their "addressable unit" is the file, and the "address" is the file name. Shells have all kinds of methods of testing for file existence, file type, file name formatting (beginning with globbing). Shells have very few primitives for dealing with file contents. Shell ...


0

In bash, ksh should work as well, using only shell built-ins: #!/bin/bash # we require array support d=( $(< sample.txt) ) # quote arguments and # build up brace expansion string d=$(printf -- '%q,' "${d[@]}") d=$(printf -- '%s' "{${d%,}}' '{${d%,}}") eval printf -- '%s\\n' "$d" Note that while this holds the entire file in memory in a shell variable, ...


2

One option with Python is to memory-map the file and take advantage of the fact that the Python regular expression library can work directly with memory-mapped files. Although this has the appearance of running nested loops over the file, memory mapping ensures that the OS brings available physical RAM optimally into play import mmap import re with ...


0

would sed -r '/^..../s/:0+/:/g' be ok? (Improved answer from Gilles that keeps first 4 chars untouched)


4

Here's how to do it in awk so that it doesn't have to store the whole file in an array. This is basically the same algorithm as terdon's. If you like, you can even give it multiple filenames on the command line and it will process each file independently, concatenating the results together. #!/usr/bin/awk -f #Cartesian product of records { file = ...


2

while read -r first second ; do sed -n "/${first//\//\\\/}/,/${second//\//\\\/}/p" file2 ; done < file1 note, that ${first//\//\\\/} is used to replace possible / symbol to escape sequence \/ cause otherwise contruction may be broken. if you're completely sure you don't have / symbol in file1 then you can use just $first and $second.


2

With zsh: a=( Row1,10 Row2,20 Row3,30 Row4,40 ) printf '%s\n' $^a' '$^a $^a on an array turns on brace-like expansion (like in {elt1,elt2}) for the array.


5

I'm not sure this is better than doing it in memory, but with a sed that reads out its infile for every line in its infile and another on the other side of a pipe alternating Hold space with input lines... cat <<\IN >/tmp/tmp Row1,10 Row2,20 Row3,30 Row4,40 IN </tmp/tmp sed -e 'i\ ' -e 'r /tmp/tmp' | sed -n '/./!n;h;N;/\n$/D;G;s/\n/ /;P;D' ...


4

Well, you could always do it in your shell: while read i; do while read k; do echo "$i $k"; done < sample.txt done < sample.txt It is a good deal slower than your awk solution (on my machine, it took ~11 seconds for 1000 lines, versus ~0.3 seconds in awk) but at least it never holds more than a couple of lines in memory. The loop above ...


0

You can always match the last occurrence in a sequence of anything like: s/\(sequence\)*/\1/ And so you're on the right track, but rather than replacing the sequence with a space - replace it with its last occurrence - a single space. That way if a sequence of spaces is matched then the sequence is reduced to a single space, but if the null string is ...


3

If you're still interested in how to remove any zero except those within the first 4 digits, you can just do like: sed 'H;g;s/\(\n\([^:]*\)\)*0*/\2/g ' <<\IN 2001:0db8:03cd:0000:0000:ef0405:00060:0123 IN OUTPUT 2001:db8:3cd:::ef45:6:123 Ok, so this globally matches either zero or more sequences of a newline followed by zero or more not colon ...


3

A similar approach using awk: awk '{gsub(/:0*/,":")}1' file 2001:db8:3cd:::ef45:6:123


8

So you want to remove runs of 0s, but only when they are preceded by a colon. sed -e 's/:00*/:/g'


3

Another awk solution: awk '{print gsub(/"/, "")}'


0

This one handles filenames with whitespace or apostrophes, and works on systems which do not support xargs -d or sort -h: du -s * | sort -n | cut -f2 | tr '\n' '\0' | xargs -0 -I {} du -sh "{}" which results in: 368K diskmanagementd 392K racoon 468K coreaudiod 472K securityd 660K sshd 3.6M php-fpm


1

We can use grep with regex to make it more simple and powerful. To count specific character. $ grep -o '"' file.txt|wc -l To count special characters including whitespace characters. $ grep -Po '[\W_]' file.txt|wc -l Here we are selecting any character with [\S\s] and with -o option we make grep to print each match (which is, each character) in ...


5

sort -V input from man sort: -V, --version-sort natural sort of (version) numbers within text that will get you: CPL_V11.01.00_1 CPL_V11.01.00_2 CPL_V11.01.00_3 CPL_V11.01.00_4 CPL_V11.01.00_10 CPL_V11.01.00_35 CPL_V11.01.00_36 CPL_V11.01.00_37 CPL_V11.01.00_38 CPL_V11.01.00_39 CPL_V11.01.00_40 CPL_V11.01.00_41 CPL_V11.01.00_42 ...


1

\w match "word" symbols (letters, digits and underscore) but in your example there is / after com which is not :alnum: so your pattern match nothing == empty output. You can add / to pattern and look what is happend: grep -oP 'com/\K\w+' FYR -P option is experimental and can do which not expected in more systems, so you can do your task in other way: ...


1

You can use AWK to do this more effectively. $ awk -F: '$3 ~ /^1/ {print $1, $3}' /etc/group daemon 1 uucp 10 man 12 proxy 13 kmem 15 users 100 libuuid 101 crontab 102 fuse 103 avahi-autoipd 104 scanner 105 messagebus 106 colord 107


1

Notice any consistent pattern with groups and their corresponding group numbers that start with a 1? $ head -20 /etc/group root:x:0: bin:x:1: daemon:x:2: sys:x:3: adm:x:4:logcheck tty:x:5: disk:x:6: lp:x:7: mem:x:8: kmem:x:9: wheel:x:10:saml cdrom:x:11: mail:x:12: man:x:15: dialout:x:18: floppy:x:19: games:x:20: tape:x:30: video:x:39: ftp:x:50: One ...


3

Pure bash, one-liner: unset x y sum; while IFS=, read x y; do ((sum[$x]+=y)); done < input.csv; for i in ${!sum[@]}; do echo $i,${sum[$i]}; done Or in more readable form: unset x y sum while IFS=, read x y; do ((sum[$x]+=y)); done < input.csv for i in ${!sum[@]}; do echo $i,${sum[$i]} done The result: 100,400 201,400 300,600


1

With python this can be done more effectively. This program by default expects the file to be named as 'file.txt', which you can change if needed. #!/usr/bin/env python3 col1, col2 = [ list(y) for y in zip(*[ x.strip().split(',') for x in open('file.txt').readlines() if x != '\n' ]) ] for (offset,x) in enumerate(list(col1)): value = 0 while ...


1

I think this'll do: awk 'BEGIN{FS=OFS=","}{a[$1]+=$2}END{ for (i in a) print i,a[i]}'


3

Try: $ awk ' { gsub(/[^[:alpha:] ]/,""); for(i=1;i<=NF;i++) { a[$i] = a[$i] ? a[$i]", "FNR : FNR; } } END { for (i in a) { print i": "a[i]; } }' file | sort A: 3, 4 Are: 5, 6, 8 B: 6, 10 BFG: 5 C: 1, 7, 8 CB: 2 How: 5 You: 5


0

A perl solution: $ perl -anle ' for (2..$#F) { $h{join(" ",@F[0..1])}->{$_} ||= $F[$_]; $h{join(" ",@F[0..1])}->{$_} = $F[$_] if $F[$_] ne "."; } END { print "$_ @{$h{$_}}{sort keys %{$h{$_}}}" for sort keys %h } ' file chr1 45162 A . C T chr1 45257 A . . T chr1 45413 A . . T chr1 46844 A . C . chr1 72434 A G . . chr1 72515 A . C T chr1 ...


0

One more variant for unsorted lines: awk '{ k[$2]=$1; for(i=3;i<7;i++){ if(l[$2,i]=="." || l[$2,i]=="") l[$2,i]=$i; } } END{ for(n in k){ printf("%s %s ",k[n],n); for(m=3;m<7;m++) printf("%s ", l[n,m]); print ""; } }' file A short explanation: Going through file scripts make two assotiative arrays: k with field#2 as index and l ...


1

Here is one approach: $ awk '$2!=old && NR>1 {for (i=1;i<=NF;i++) printf a[old","i]" "; print"";} {old=$2;for (i=1;i<=NF;i++) {if (a[$2","i]=="." || a[$2","i]=="") a[$2","i]= $i}} END{for (i=1;i<=NF;i++) printf a[old","i]" "; print"";}' file chr1 45162 A . C T chr1 45257 A . . T chr1 45413 A . . T chr1 46844 A . C . chr1 72434 A G ...



Top 50 recent answers are included