Tag Info

New answers tagged

3

I'd do this in Perl: perl -lne 'if(/^2009/){$n=$_; next} print "$n $_"' file or, more concisely perl -lne '/^2009/ ? ($n=$_) : print "$n $_"' file The idea is to save the current line ($_ in Perl) as $n if it begins with 2009 and if not, print the current line along with the current value of $n.


0

An alternative in Perl; assuming the same interpretation of your question as sg-lecram's : perl -lne 'tr{ }{}d; # Remove whitespace in current line $lines{$_}++; # Record the current line in a hash END{ # After all lines have been processed for(keys %lines){ # Iterate over hash keys ...


2

Using sed: sed -n ' /^2009/ { h } /^2009/ !{ G; s/^\(.*\)\n\(.*\)$/\2 \1/p } ' in_file Explanation as requested: -n - makes sed not print anything unless we tell it to. /^2009/ { h } - when we reach a line beginning with 2009, put it in the hold buffer. /^2009/ !{...} - the pattern inside the {...} is gets applied to every line that doesn't start ...


3

This should probably work awk '/^2009/{Prepend = $0; next} {print Prepend, $0}' file.txt So output looks like: awk '/^2009/{Prepend = $0; next} {print Prepend, $0}' file.txt 2009 150 0 0 0 0000 75.316 0.0400390625 0.00007 0.00000 0.8980 2009 150 0 0 0 0000 76.216 0.0400390625 0.00007 1.00000 0.9046 2009 150 0 0 0 ...


4

You can use awk for this: awk '/^2009/{a=$0;next}{print a" "$0} ' file.txt This will only prepend a space to any lines before the first match of 2009, you can set a default string to prepend like this: awk 'BEGIN{a="My default prepend string";}/^2009/{a=$0;next}{print a" "$0} ' file.txt


0

As far as I got it the following rules lead to your desired output: A1,A2: same letter (i.e. "group") --> looking for A2,A1: not found --> print A2,A1 B1,B2: same letter (i.e. "group") --> looking for B2,B1: not found --> print B2,B1 C1,C2: same letter (i.e. "group") --> looking for C2,C1: found --> don't print C2,C1: same letter (i.e. "group") --> looking ...


1

The purely awk solution, no redundant cat or grep commands: awk '/MemTotal/ {TOT=$2} /MemFree/ {FREE=$2} END {printf("%d kB Used\n", TOT-FREE)}' /proc/meminfo I see awk_FTW beat me to it but I though formatting the output could be nice.


0

if the order doesn't matter, you can simply use this: awk '{for(i=2;i<=NF;i++)print $1,$i}' file


0

Awk this awk script will work on an arbitrary number of columns > 2 and order of appearance will be preserved as across then down with no assumptions made about what the columns are (i.e. doesn't matter if they are numeric or not, sorted or not, etc): { for (i = 2; i <= NF; i++) { a[j + i] = $1 " " $i } j += (i - 1); } END { ...


0

Here is an awk solution: $ awk '{a[i++]=$1" "$3;print $1,$2}END{for(i=0;i<length(a);i++){print a[i]}}' file 0 0 0.05 9.6877884e-06 0.1 4.2838688e-05 0.15 0.00016929444 0.2 0.00036426881 0.25 0.00055234582 0.3 0.00077448576 0.35 0.00082546537 0.4 0.0012371619 0.45 0.0013286382 0 0 0.05 0.0024898597 0.1 0.0049595502 0.15 0.0074092494 0.2 0.009839138 0.25 ...


0

The problem with CSV is that there is no standard. If you need to deal with CSV-formatted data often, you might want to look into a more robust method rather than just using "," as your field separator. In this case, Perl's Text::CSV CPAN modules are exceptionally well-suited to the job: $ perl -mText::CSV_XS -WlanE ' BEGIN {our $csv = ...


0

If you don't want to loop through each entry of the second array for each value in the first, then you'll need to use an associative array. In awk you could do this: $ a=(a b c d); $ b=(a h c d l k); $ awk 'BEGIN{RS = FS} NR == FNR {a[$1] = 1; next} $1 in a' \ <(echo "${a[*]}") <(echo "${b[*]}") a c d Pure bash would look something like this ...


0

It's not clear whether you are trying to group lines by unknown output patterns or unknown keywords in known patterns. In the first case, if you have logs such as: [2010-04-02 12:00:00] Error: BaseController Something went wrong 2010-04-02 12:01:00 Warning - Something happened UserController (2010-04-02 12:02:00) failed with exit status: 1 [2010-04-02 ...


0

Recent bash versions (> 4) also of associative arrays, i.e. you can also use a bash 'one liner' for it: PATH=$(IFS=:; declare -A a; NR=0; for i in $PATH; do NR=$((NR+1)); \ if [ \! ${a[$i]+_} ]; then if [ $NR -gt 1 ]; then echo -n ':'; fi; \ echo -n $i; a[$i]=1; fi; done) where: IFS changes the input field ...


0

Here is an AWK one liner. $ PATH=$(echo -n $PATH \ | awk -vRS=: -vORS= '!a[$0]++ {if (NR>1) printf(":"); printf("%s", $0) }' ) where: echo -n prints the content of $PATH without a trailing newline RS=: changes the input record delimiter character (default is newline) ORS= changes the output record delimiter to the empty string a the name of an ...


0

I'm not exactly sure what you want to do because you don't show the actual input, just the output you want and various bits of code that are used in various stages. However, I think the following will do what you want (make sure you set -F: on the command line. If not, I've tried to describe each part to give you an idea of how to modify it. !/^#/ { ...


0

If you don't mind using gawk >= 4.0, this (which is pretty much the same as terdon's) will produce the desired output, with optional name and key ordering: NF { Names[$3][$1] = 1; Names[$3][$2] = 1; } END { PROCINFO["sorted_in"] = "@ind_str_asc"; # if you want `Name` ordered for (Name in Names) { PROCINFO["sorted_in"] = ...


1

Personally, I would just do the whole thing in Awk from the original file rather than half in PHP and half in Awk or Perl. Given file1.txt above, the following will produce the desired output: { Vals[$1]++; Vals[$2]++; Third_col[$1, $2] = Third_col[$2, $1] = $3; } END{ for (i in Vals) { for (j in Vals) { if (i == j || (i ...


1

Oh, that's an easy one. Here's a simple version that keeps the order of the keys as they appear in the file: $ awk -F, ' /.+/{ if (!($1 in Val)) { Key[++i] = $1; } Val[$1] = Val[$1] "," $2; } END{ for (j = 1; j <= i; j++) { printf("%s %s\n%s", Key[j], Val[Key[j]], (j == i) ? "" : "\n"); } ...


0

As others have mentioned, it's not clear exactly what you want to do, but you can fix the syntax of your approach thus: awk -F, '$2 ~ /^[$*&%#]+$/ { count++ } END {print count}' sample.txt


1

Here's another Perl approach: $ perl -ane 'foreach(@F[0..1]){$k{$F[2]}{$_}++} END{ foreach $v (sort keys(%k)){ print "$_ " foreach(keys(%{$k{$v}})); print "$v\n" }; } ' file This produces: 47723284 47196436 name1 42672249 430695 52856963 name2 380983 55094959 ...


0

How about this awk -F, '{if ($2 ~ /[^:alnum:]/) l=length($2);print $0" "l}' sample.txt 101,aaa,d01 102,*&%,d02 3 103,$%&,d03 3 104,###,d04 3 Prints each line the count of "special chars", here chars that are not alnum.


1

There are many ways to do this. You don't specify this in your question, but your awk approach is attempting to count those second fields that consist entirely of special characters so that's what my solutions are doing as well. If that's not what you want and instead you want to count the fields that simply contain at least one special character remove the ...


1

Please try reading some documentation before posting here. What you're asking is trivial to find with a 5 minute google search. You might want to read through our Help page on how to ask questions to get a better idea of how this site works. Anyway, you can either pass variables as arguments using the -v option: awk -F'[]]|[[]' -v var1="2014-04-07 23:00" ...


1

Another awk solution: $ awk 'FNR==NR{a[$1,$2]=$3;next} {print $0,a[$1,$2]?a[$1,$2]:a[$2,$1]?a[$2,$1]:0} ' file1 file2 3377090 8145216 1.5 1405541 8145216 0 1405541 3377090 0 53595498 8145216 0 53595498 3377090 0 53595498 1405541 1.53637


4

If you don't mind the order of output: $ awk -F',' 'NF>1{a[$1] = a[$1]","$2}END{for(i in a){print i""a[i]}}' file jkl, words and numbers, words and numbers, words and numbers, words and numbers abc, 12345, 56345, 15475, 123345 ghi, something else, something else, something else, something else def, text and nos, text and nos, text and nos, text and nos ...


2

A perl solution: $ perl -ane '$h{$F[2]} .= " ".$F[0]." ".$F[1]; END { for $k (sort keys %h) { print $_," " for grep {!$seen{$_}++} split(" ",$h{$k}); print "$k\n"; } }' file 47196436 47723284 name1 42672249 52856963 430695 name2 55094959 380983 name3 17926380 55584836 3213456 34211 54321 name4


1

awk 'NR==FNR{a[$1>=$2?$1SUBSEP$2:$2SUBSEP$1]=$3;next}; {k=$1>=$2?$1SUBSEP$2:$2SUBSEP$1; print $0, k in a?a[k]:0}' file1.txt file2.txt 3377090 8145216 1.5 1405541 8145216 0 1405541 3377090 0 53595498 8145216 0 53595498 3377090 0 53595498 1405541 1.53637


2

Ungainly, but seems to do the job awk '$3 != prev {if (NR != 1) print prev; prev=$3; delete a}; !($1 in a){a[$1]++; printf "%s ", $1}; !($2 in a){a[$2]++; printf "%s ", $2}; END {print prev}' ccc.txt 47196436 47723284 name1 42672249 52856963 430695 name2 55094959 380983 name3 17926380 55584836 3213456 34211 54321 name4


2

In your is_mounted function you should test the output of mount to confirm that something was returned and then return a 0 or 1 accordingly. Also I would highly encourage you to use the debugging facilities built into bash. You'll quickly see what's going wrong rather than have us try and debug it. Just put a set -x at the beginning of the block of code ...


3

This is not a problem which should be solved with regex (although technically you could find a way to do it). Since you are going to be using this more than once, you might want to use some real datetime functions rather than just parsing strings. The following Awk script may work for you. # Expects time strings in the form of `[2014-04-25 15:38:23]` with ...


1

Using sed : #!/bin/bash E_BADARGS=23 if [ $# -ne "3" ] then echo "Usage: `basename $0` \"<start_date>\" \"<end_date>\" file" echo "NOTE:Make sure to put dates in between double quotes" exit $E_BADARGS fi isDatePresent(){ #check if given date exists in file. local date=$1 local file=$2 grep -q "$date" ...


1

One alternative to awk or a non-standard tool is to use GNU grep for its contextual greps. GNU's grep will let you specify the number of lines after a positive match to print with -A and the preceding lines to print with -B For example: [davisja5@ehsbbrlp01 ~]$ cat test.txt Ignore this line, please. This one too while you're at it... [2014-04-07 23:59:58] ...


4

Check out dategrep at https://github.com/mdom/dategrep Description: dategrep searches the named input files for lines matching a date range and prints them to stdout. If dategrep works on a seekable file, it can do a binary search to find the first and last line to print pretty efficiently. dategrep can also read from stdin if one the filename ...


8

You can use awk for this: $ awk -F'[]]|[[]' \ '$0 ~ /^\[/ && $2 >= "2014-04-07 23:00" { p=1 } $0 ~ /^\[/ && $2 >= "2014-04-08 02:00" { p=0 } p { print $0 }' log Where: -F specifies the characters [ and ] as field separators using a regular expression $0 references a complete line $2 ...


3

You can maybe do: awk '$10 ~ /^[0-9a-zA-Z]+$/ {count++} END{print count}' file This also works to me: to check from 0 to z, being all numbers and letters in the range. awk '$10 ~ /^[0-z]+$/ {count++} END{print count}' file The idea is to check if the given field ($10 in your case) is just based on a set of alphanumeric characters. This is accomplished ...


3

Something like this should work: awk -F, '$10 ~ /^[[:alnum:]]+$/ { count++ } END{ print count+0 }'


0

This should do the trick: (grep -oP "Event 100" file && \ sed -ne 's/<\/\?AZ>//g' -e '7,9p' file) | \ awk 'BEGIN {RS=""; FS="\n";} \ {printf "%s %s %s %s\n", $3, $2, $4, $1}' Explanation: file: the file that contains you output above grep -oP "Event 100" file: Searches for "Event 100" sed -ne 's/<\/\?AZ>//g' -e '7,9p' file: If the ...


1

I came up with this nasty oneliner (line breaks for readability): awk -F'[<>]' '($0 ~ /[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/) {time=$3} ($0 ~ /[0-9][0-9]\/[0-9][0-9]/) {date=$3} ($0 ~ /\[[0-9]+\]/) {tag=$3} ($0 ~ /Event 100 occurred/) { print date, time, tag, "Event 100 occurred"}' < testfile This ...


0

It looks like your list is sorted by size, or could easily be sorted by size, so if you wanted to optimise this a bit using First Fit Decreasing strategy, you could use the following Awk script (lovely, and understandable) with tac to read the input file from the bottom (only required because it's sorted by file size in ascending order): BEGIN { ...


0

Instead of using -v, you need to pass your arguments directly to the execution part by using the double quotes ": awk 'NR==18 {$0='$nstep'}{print}' < input > output an example with more expressions: awk -e 'NR==18 {$0='"$nstep"'}{print}' -e 'NR==22 {$0='"$p"'}{print}' -e 'NR==24 {$0='"$p"'}{print}' -e 'NR==82 {$0=".false."}{print}' ...


0

In case you were wondering, it is possible to do this with join, although the order of the lines is not preserved: ( join <(head -n1 file1) <(head -n1 file2) join -a 1 -e 0 -o '0 1.2 1.3 1.4 2.2 2.3 2.4' \ <(sed -n 2,5p file1 | sort) \ <(sed -n 2,5p file2 | sort) join <(tail -n1 file1) <(tail ...


5

awk ' NR==FNR {vals[$1] = $2 " " $3 " " $4; next} !($1 in vals) {vals[$1] = "0 0 0"} {$(NF+1) = vals[$1]; print} ' file2 file1 TABLES REF-IO HEAD-IO DIFF-IO REF-SELECT HEAD-SELECT DIFF-SELECT test 200 500 -300 5 7 -2 exam 2 3 -1 0 7 -7 final 2 1 1 12 6 6 mail 4 2 2 0 0 0 TOTAL 208 506 -298 20 23 -3


0

This script should work: touch resultFile while read f; do header1=$(echo $f | awk '{print $1;}'); values1=$(echo $f | awk -F "$header1 " '{print $NF;}'); while read g; do header2=$(echo $g | awk '{print $1;}'); values2=$(echo $g | awk -F "$header2 " '{print $NF;}'); if [ $header1 = $header2 ]; then echo ...


0

This one should do but it is slow ( 1m18s for 500 entries) #!/bin/bash #reformatting the initial file to remove tab SRC=`cat file.txt | expand` outputs_dir="outputs" if [ ! -d "$outputs_dir" ];then mkdir "$outputs_dir" else echo "$outputs_dir exist" #rm "$outputs_dir"/* fi #init file outputs array with 2 files first one is in case files is bigger ...


2

To replace the first line with the expected timestamp, just use sed "1c TS: $(date '+%F %T')" The 1 at the beginning means the first line, c means "replace the line with the following string". $(...) inserts the output of the given command, see man date for details.


4

The simplistic approach would be to just use sed and replace the string with the correct one. So, if the current time is 2014-04-02 21:34:13 you could simply run sed 's/2014-03-31 13:56:01/2014-04-02 21:34:13/' file Since, presumably, you want to do this dynamically, you can pass sed the result of the relevant date call instead: sed "s/2014-03-31 ...


0

Lovely understandle Python: #!/usr/bin/env python with open('unix_StackExchange_question.txt') as f: files = f.read() files = files.split('\n') group = list() group_size = 0 n = 0 for f in files: pair = f.split() if not pair: break size = int(pair[0]) name = pair[1] group_size += size group.append(f) # assume the ...


3

This should work with bash: #!/bin/bash accumulated_size=0 file_counter=0 out_file=file_0000 while read size name; do if [ "$accumulated_size" -gt 5368709120 ]; then (( file_counter += 1 )) out_file=$(printf 'file_%04d' "$file_counter") echo -n >"out_file" accumulated_size=0 fi echo "$size $name" >>"$out_file" (( ...


7

It sounds like you're asking for a solution to the bin-packing problem. So far as we know, there isn't a way to do it both optimally and quickly (but we don't know there isn't one, either; its an open question). But you can get close. One utility to do it is called datapacker. Haven't used that one; found it by searching. I'm sure there are more. ...



Top 50 recent answers are included