Tag Info

New answers tagged

0

I would put numbers to arrays and then play a little bit with IFS variable to sum all array elements fast: h=($(grep -Po '[[:digit:]]+(?= hour)' file)) m=($(grep -Po '[[:digit:]]+(?= minute)' file)) s=($(grep -Po '[[:digit:]]+(?= second)' file)) ms=($(grep -Po '[[:digit:]]+(?= milli)' file)) IFS=+ t=$(( 3600*$((${h[*]})) + 60*$((${m[*]})) + ${s[*]} + ...


5

You can do this with sed — it's Turing-complete. But it isn't the best tool for the job. Sed doesn't have a convenient way of remembering that it's already made a replacement. What you can relatively easily do with sed is to blank all the lines starting /swapfile, and add a new one at the end: sed -i '$! s/^\/swapfile[\t ]//; ...


4

Since you are automating this with Puppet, it's better to let Puppet handle the fstab for you. Use the mount resource type. Something like: mount{'swapfile': name => 'none', fstype => 'swap', ensure => mounted, atboot => true, device => '/swapfile', options => 'sw', } should work. If this absolutely has ...


2

Version 1.7 includes support for JSON: http://mama.indstate.edu/users/ice/tree/changes.html Per the man page (under XML/JSON/HTML OPTIONS): -J Turn on JSON output. Outputs the directory tree as an JSON formatted array. e.g. $ tree -J ...


3

I find uconv (in icu-devtools package in Debian) useful to inspect UTF-8 data: $ print '\\xE9 \xe9 \u20ac \ud800\udc00 \U110000' | uconv --callback escape-c -t us \xE9 \xE9 \u20ac \xED\xA0\x80\xED\xB0\x80 \xF4\x90\x80\x80 (The \xs help spotting the invalid characters (except for the false positive voluntarily introduced with a literal \xE9 above)). ...


1

Python has had a built-in unicode function since version 2.0. #!/usr/bin/env python2 import sys for line in sys.stdin: try: unicode(line, 'utf-8') except UnicodeDecodeError: sys.stdout.write(line) In Python 3, unicode has been folded into str. It needs to be passed a bytes-like object, here the underlying buffer objects for the ...


3

If you want to use grep, you can do: grep -av '^.*$' file in UTF-8 locales to get the lines that have at least an invalid UTF-8 sequence (this works with GNU Grep at least).


2

It's not too hard with sed really. You can always delimit a section with a \newline or you can trade out a delimiter for a \newline temporarily. And you can do it without a loop: sed 's/$/START/;s/END/& /g; y/D\n/\nD/ s/\([^D]*START\)*[D\"]*/\1/g y/\n/D/;s/.....$// ' <<\IN \"XXX \ START sapiodj ...


-2

sed \$r01.txt new.txt sed \$r02.txt new.txt


1

With recent versions of GNU sed: sed -E '/^AUTH:/!b;s/([^\w:])(\w+)/\1\L\u\2/g' With older versions: sed -r '/^AUTH:/!b;s/([^[:alnum:]:])([[:alnum:]]+)/\1\L\u\2/g'


2

Assuming you don't have ` character in the file. If you do just change in the line bellow all ` to anything else. sed -e 's/END/`/g;:X' -e 's/\(START[^`]*\)["\]/\1/g;tX' -e 's/`/END/g'


1

With sed: sed 's/:/::/g;s/</:l/g;s/>/:g/g; # escape :, <, > s/START/&</g; s/END/>&/g; # replace START/END with <> :1 s/\(<[^>]*\)[\"]/\1/g;t1 s/[<>]//g;s/:g/>/g;s/:l/</g;s/::/:/g; # restore <>:' With perl: perl -pe's|START.*?END|$&=~y/\\"//rd|ge'


0

You have indicated in a comment that awk is also allowed. So I'm basing my answer on this. Assuming your STARTs and ENDs are balanced, if you split the line on either word, you find that you want to remove backslashes and double quotes from all even-numbered fields. To this end: awk -F 'START|END' '{ for(i=2;i<=NF;i+=2){ # For each ...


2

In awk: awk -F '' -vOFS='\t' '{$1=$1}1' file > new_file Borrowed the idiom {$1=$1}1 from an answer to one of your other questions. This sets the field separator to nothing (-F ''), which means that each record is read character-by-character. The output field separator is set to a TAB character (-vOFS='\t') and the idiom {$1=$1}1 is (as far as I can ...


0

I think you should give fold a go: tr \\n \\r <infile | fold -w1 | tr '\r\n' '\n\t' >outfile I preprocess fold's input w/ tr by replacing the instream \newline characters w/ \returns. fold is printing a \newline character for every column in input - each of your capital letters - but it resets its counter on \returns and so when tr does the final ...


2

sed -e '/^AUTH:\([^[:alpha:]]*\)/!b' -e 'h;s//\1/;x;s/// s/\([[:alpha:]]\)\([[:alpha:]]*[^[:alpha:]]*\)/\1/g;x;s//\ \2/g y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/;G;:l s/\n\(.*\n\)\(.\)/\2\1/;tl s/\(.*\)\n/AUTH:\1/ '<<\IN TITLE: Average title AUTH: SUPERMAN ...


5

If I have understood your intention correctly then this does it: sed -e 's/./&\t/g' -e $'s/\t$//' file The second replacement deletes the tab at the end of the line.


2

Try doing this using perl: perl -ne 'print join "\t", split //' file > new_file


3

> sed '/^AUTH/{s/^AUTH: //;s/\b\([[:alpha:]]\)\([[:alpha:]]*\)\b/\u\1\L\2/g;s/^/AUTH: /;}' file TITLE: Average title AUTH: Superman AFF: Something AUTH: The New One AFF: Berlin AUTH: Mars-Mensch AFF: Planet Mars


1

> awk '/^[^ ]/ {printf "\n%s",$1; next}; { printf ",%s",$3; }' file 360060e80056fc30000006fc30000513c,sdcm,sdcn 360060e80056fc30000006fc300005162,sdbu,sdbv 360060e80056fc30000006fc300005127,sdg,sdt


5

Depending on the version of sed on your system you may be able to do sed -i 's/Some/any/; s/item/stuff/' file You don't need the g after the final slash in the s command here, since you're only doing one replacement per line. Alternatively: sed -i -e 's/Some/any/' -e 's/item/stuff/' file The -i option tells sed to edit files in place; if there are ...


6

You can chain sed expressions together with ";" %sed -i 's/Some/any/g;s/item/stuff/g' file1 %cat file1 anything 123 stuff1 anything 456 stuff2 anything 768 stuff3 anything 353 stuff4


5

Multiple expression using multiple -e options: sed -i.bk -e 's/Some/any/g' -e 's/item/stuff/g' file or you can use just one: sed -i.bk -e 's/Some/any/g;s/item/stuff/g' file You should give an extension for backup file, since when some implementation of sed, like OSX sed does not work with empty extension (You must use sed -i '' to override the original ...


0

With awk: $ awk 'BEGIN{FS=OFS="|"}NR==FNR{a[$1]=$0;next}{$1=a[$1]}1' file_1 file_2 14595|Age 35|Salary xx|Position ax|2013|Info 1|Info 2|Info 3|Info 4|Info 5|Address xx|Info 6|Info 7|Info 8 14649|Age 30|Salary xx|Position az|2015|Info 1|Info 2|Info 3|Info 4|Info 5|Address xxxz|Info 6|Info 7|Info 8


2

Use join: $ join -t'|' file_1 file_2 14595|Age 35|Salary xx|Position ax|2013|Info 1|Info 2|Info 3|Info 4|Info 5|Address xx|Info 6|Info 7|Info 8 14649|Age 30|Salary xx|Position az|2015|Info 1|Info 2|Info 3|Info 4|Info 5|Address xxxz|Info 6|Info 7|Info 8 -t indicates the field separator. In order to join works, files must te sorted. You can use sort for ...


0

If you have GNU grep, you can use a perl-compatible regular expressions. This is handy due to the look-around assertions: grep -oP '(?<=track_name).*?(?=,)' filename


1

This is an example regex that matches your provided input and gives you the output you desire. Using this method, we're causing sed to do a search+replace (s) on each line, replacing the whole line with just the part in the middle, if the line contains track_name and a comma. We then only print matched lines (p). [me:~]$ cat work/tmp/example.txt ...


4

You're mixing character classes (a list of characters inside square brackets) with the smb.conf share names which are surrounded by square bracket literals. Also, the echo command is not well-formed: in the case where sed exits with a non-zero status, the shell will attempt to invoke the command Failed. A few suggestions: Remove the character class (outer ...


1

It can be done using, awk: awk -F: -v OFS=: '{split ($NF, groups, " "); $NF=""; for (i in groups) {printf "%s%s\n", $0, groups[i]}}' Assigning to a field (like in $NF="") causes $0 to have instances of FS replaced with the OFS, so we have to assign : to OFS. Don't expect any order in the output - the group field will be randomly output. Use a sort later ...


0

I wouldn't recommend it on a huge chunk of data, but I thought I'd try a bash solution. I'm not very bash-fluent, but this seems like the "obvious" way of handling things, "algorithmically speaking". This basically iterates over each file and searches for the good range. I've given some details in the comments. The script must be executed this way: $ ...


2

Assuming that salaries and salary bounds are always going to be integers and that salary ranges are non-overlapping, here's something that might work (in bash) { printf "indi Rank\n" ; join -o1.1,2.2 <(tail -n +2 file1 | sort -b -k2,2) -1 2 -2 1 \ <(awk 'FNR > 1{for (i=$1; i<$2; ++i) printf "%d %s\n", i, $3}' file2 | sort -k1,1) ;} indi Rank ...


2

$ paste prices fruits | sort -k2 | cut -f1 1.01 2.18 4.11 4.52 1.73 1.69 1.09 paste combines the two files, line by line. sort -k2 sorts them on the second column (the fruit name). cut -f1 returns just the first column (the prices). For the above, I assumed that the line numbers shown in the display of the fruits and prices files were an artifact of the ...


4

Hold buffer method: sed '$x;1!H;1p;$!d;x;s/\n// ' <<\IN aaaaaa bbbbbb cccccc dddddd eeeeee IN ...that will Hold every line which is !not the first, and the first it prints. On the $last line it exchanges hold and pattern spaces before it does the Hold - which gets the saved lines appended to the last line - then deletes from output all lines which ...


3

Depending on the size of the file you can do: sed '/^*/!H;//p;$!d;g;s/\n//' That stacks in Hold space lines which do not match /^*/. Those that do match are printed as they occur in input. Then all lines which are !not the $last are deleted from output. On the last line we get hold space by overwriting pattern space, then the first \newline character is ...


1

Here is a solution using ed (and expanding a bit on the vi solution posted by Kaz, which underneath uses ex - a beefed up ed). Replace w with ,p\n to print the output instead of writing to file. Move the matching lines to the end of the file: ed -s file <<< $'g/^\*/m$\nwq\n' The global subcommand marks every line that matches the pattern. ...


8

This problem might actually be easiest to do with ed, since it's a basically a scriptable text editor, rather than a stream processor. Using ed, you don't have to save all the lines of the file into an array, for instance, since it's already doing that for you. # Create test file ~> printf "%s\n" aaaaaa bbbbbb cccccc dddddd eeeeee >test.txt ~> cat ...


1

With perl : perl -e 'my @lines = <>; print for @lines[0, $#lines, 1..$#lines-1]' file With awk : $ awk ' {lines[NR]=$0} END{ print lines[1], lines[NR]; for (i=2; i<NR; i++) {print lines[i]} } ' OFS=$'\n' file OUTPUT aaaaaa eeeeee bbbbbb cccccc dddddd


2

A naive approch using awk: ~$ awk '{a[NR]=$0}END{print a[1];print a[NR];for(i=2;i<NR;i++){print a[i]}}' f aaaaaa eeeeee bbbbbb cccccc dddddd You store every line in the a array, then print the array in the order you want (1st line, last one (NR) and from 2 to penultimate. Using a combination of head/tail and sed: ~$ head -1 f;tail -1 f;sed '1d;$d' f ...


1

Through awk and keeping middle column: awk '{printf("%.f ", ($1/65)+0.5)}1' infile > outfile 24 1533 C_P.doc 11 691 C_M.doc 14 905 G_S.doc 15 945 J_Z.doc 24 1549 J_M.doc 27 1701 L_B.doc Through awk and without middle column: awk '{printf("%.f", ($1/65)+0.5); $1=""}1' infile > outfile 24 C_P.doc 11 C_M.doc 14 G_S.doc 15 J_Z.doc 24 J_M.doc 27 L_B.doc ...


0

If you don't mind using python in shell and assuming that a.txt is your file: [sreeraj@server ~]$ cat a.txt 1533 C_P.doc 691 C_M.doc 905 G_S.doc 945 J_Z.doc 1549 J_M.doc 1701 L_B.doc [sreeraj@server ~]$ for i in $(awk -v c=65 '{ print $1/c }' a.txt) ; do python -c 'print int(round('$i',0))' ; done >> b.txt [sreeraj@server ~]$ paste b.txt a.txt > ...


1

You can use perl: $ perl -MPOSIX=ceil -anle '$F[0] = ceil($F[0]/65);print "@F"' file 24 C_P.doc 11 C_M.doc 14 G_S.doc 15 J_Z.doc 24 J_M.doc 27 L_B.doc


1

Let's say you recipes folder in in /home/username/recipes Here is a simple script: #!/bin/bash count=0 dir=/home/username/recipes/ for recipe in $(ls $dir); do if [ $(cat $HOME/recipes/$recipe | grep $1) ]; then ((count++)) fi done echo "$count $1" Once you've saved the file, you have to make it executable. Do this with: chmod 740 script.sh ...


2

Use grep -c For example kaustubh@hacked:~/test$ cat a salt salt salt salt salt kaustubh@hacked:~/test$ cat b here sis salt their is salt and everywhere is salt kaustubh@hacked:~/test$ cat c salt hash hash salt sweet salt kaustubh@hacked:~/test$ cat d this is line salt Now I want count of word salt in files: grep -c salt * kaustubh@hacked:~/test$ ...


1

Others have provided general answers for your question which demonstrate good ways of parsing json however I, like you, were looking for a way to extract an aws instance id using a core tool like awk or sed without depending on other packages. To accomplish this you can pass the "--output=text" argument to your aws command which will give you an awk parsable ...


2

I would use the following: sed -r '/(SEM|AFF|CON)/ s/([:,] *)[Tt]he */\1/g' file Add -i option to change file in place.


1

Try this way: egrep -rl "^(SEM|CON|AFF)\: (t|T)he" * | xargs sed -r -i 's/(^(SEM|CON|AFF):\s)((t|T)he[ ]*)/\1/g'


2

Just use one sed expression (needs GNU sed): sed -r -i -e '/(SEM|AFF|CON)/s/([:,]\s*)the\s+/\1/ig' * The search pattern at the begin of the sed command restricts the replacement to the lines which begins with the selected categories. The i flag for the replace command (s//) makes the pattern case-insensitive, the g flag allows more than on replacement in ...


4

You could use perl (get the file content and substitute pattern with pattern+file content): perl -pe '$text=`cat insert.txt`; chomp($text); s/PAT/$&$text/' file.txt add -i to edit in place; g to append after each PAT (pattern) occurrence, e.g.: perl -i -pe '$text=`cat insert.txt`; chomp($text); s/PAT/$&$text/g' file.txt Another way, using ed: ...


0

Whenever you use wc -l FILENAME, it would always print file name along with a number. wc -l /etc/hosts 34 /etc/hosts In order NOT to print the file name, you can use cat and pipe it's output to wc -l as follows: cat /etc/hosts | wc -l 34 So, in your case, if you could change the total variable to total=$(cat t | wc -l), you should get only a number ...


1

I don't know if there is a flag to prevent wc from printing the filename, but with cut you can just cut out your number: wc -l filename | cut -d' ' -f1



Top 50 recent answers are included