New answers tagged

0

PATH_TO_FOLDER="/some/path/to/folder/" FILE_EXTENSION=".ext" FILES=$(ls ${PATH_TO_FOLDER}/*${FILE_EXTENSION}) for FILE in $FILES; do FILENAME=$(basename $FILE ${FILE_EXTENSION}) TMP_FILE=${PATH_TO_FOLDER}/${FILENAME}.tmp awk -v Lines=2 '{if($0~"radius-server"){ print; for(i=0;i<=Lines;i++) { getline; if(i==Lines){ next; }else{ print; } } } else{ ...


0

$ FLAG=$(awk -F\| '{printf("%s, ", $1)}' file1.txt) $ echo $FLAG 1234, 1345, 8427, 2132, 3243, $ sed "s/FLAG/${FLAG%%, }/" select.sql SELECT * FROM CUSTOMERS WHERE ID IN (1234, 1345, 8427, 2132, 3243); This depends on the flag list being small enough to fit on the command line. If it's not, you can use getline in awk to process file1.txt, gathering ...


1

sed seems like the right tool: sed -i '/radius-server/!b;n;n;d' filename How it works: /radius-server/!b # as long as it's NOT 'radius-server' do nothing (branch to end) n # read next line (replace current line with next line) n # same as above - we now read 2 lines d # delete the current line UPDATE - to modify multiple files, simply use glob ...


1

Unless you have GNU awk 4.1.0 or later... You won't have such an option as sed's -i option so instead do: for file in * do awk -v lines=2 'BEGIN { ignore = -1 } /radius-server/ { ignore = NR + lines } NR != ignore { print }' "$file" done > result.txt This works as follows: BEGIN { ignore = -1 } # initialize ignore with -1 so NR will never ...


0

#! /bin/ksh inputfile='file1.txt' sqlfile='select.sql' S_ids=$(awk -F"|" '{gsub(/^|$/,"\\'\''",$1) ; print $1","}' "$inputfile" | xargs | sed -e 's/,$//') sed -i "s/FLAG/${S_ids}/g" "$sqlfile" Output: $ cat select.sql SELECT * FROM CUSTOMERS WHERE ID IN ('1234', '1345', '8427', '2132', '3243'); This uses awk's gsub() function ...


0

Try awk to achieve this : awk '/dn/{if(b)print b;b=$0;next}NF{b=b "|" $0}END{print b}' filename > file.out


0

You can use paste to do that, like : paste -d'|' -s filename


2

The reason this doesn't work can be inferred from the error message (that you omitted to provide): sed: -e expression #1, char 14: unterminated `s' command The sed command does not accept a multi-line value. You have to collapse your multiple lines into a single line. You could do this with a script such as this: #!/bin/ksh S_ids="'$(cut -d'|' -f1 ...


0

There are 2 ways I recommend going about this. 1) Put a function in your bashrc / bash_profile and create an alias to call that function (this will make global usage of this) 2) create a shell script file and can create an alias of that file as well. #!/bin/bash function matchString(){ REGEX="$1" FILE="$2" RESULTS=$(grep -n "$REGEX" $FILE | awk -F ":" ...


0

Try this awk. echo "ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955" | awk '{for(i=1;i<=NF;i++) if(match($i,"ABC")>0) line=line "\n"$i;else line=line $i; sub("^\n","",line); print line}' Creates a line with all fields, adding in front of the fields that begin with "ABC" a line break. Finally it eliminates the first line break and prints


0

sed -E -e 's/ (ABC)/\n\1/g' The sed command replaces any instance of ABC with a newline followed by ABC. It uses the () to capture part of the match (ABC without the leading space) and \1 to include it in the replacement. e.g. $ echo 'ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955'| sed -E -e 's/ (ABC)/\n\1/g' ABC123xxx:: 2345 ABC345yyy:: 5678 ...


0

With sed: $ sed -e 's/ ABC/\ ABC/g' <file ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955


0

perl -00 -n -e 'print if (m/select /i)' XXARXADLMT.txt This uses perl's paragraph-reading mode (-00 option) to print only the paragraphs matching the case-insensitive regular expression select (with a trailing space). "paragraphs" are blocks of text separated by 1-or-more blank lines. Output: select * from emp; select * from dept; (there is ...


0

A solution - not one that is as elegant as those that change the *RS variables, but perhaps reasonably clear: PATH=`awk 'BEGIN {np="";split(ENVIRON["PATH"],p,":"); for(x=0;x<length(p);x++) { pe=p[x]; if(e[pe] != "") continue; e[pe] = pe; if(np != "") np=np ":"; np=np pe}} END { print np }' /dev/null` The entire program works in the BEGIN and END ...


0

This is a small improvement to Gilles awk solution (thanks Gilles!), but does require nawk: nawk '{if (/\\$/) {$0=substr($0,1,length($0)-2); printf "%s",$0} else print}' This will create a continuous line if the line wraps, but does not include the "\" and space character. (I found this helpful when grepping for PATH statements since the "\" can lead to ...


2

Another perl, without slurping whole file in memory: perl -pe 's/oooo/++$i/ge; $i = 0 if /endsnippet/' <file


0

I use the following awk code, it seems work very well. awk 'BEGIN {x=0} { if ($0~/oooo/) {x++; gsub(/oooo/,x); print $0} else if ($0~/snippet/) {x=0; print $0} else {print $0}}' demo.snippet edit: this can be rewrite as awk 'BEGIN {x=0} /oooo/ {x++; gsub(/oooo/,x); print $0 ; next } /snippet/ {x=0; } {print } ' ...


2

A possible perl solution perl -00 -F/\nendsnippet\n/ -pe '$n=1; s/oooo/$n++/ge' demo.snippet


2

With python (version 2 or 3): from __future__ import print_function oooo = None with open('demo.snippet') as fp: for line in fp: if line.startswith('snippet '): oooo = 1 while 'oooo' in line: line = line.replace('oooo', str(oooo), 1) oooo += 1 print(line, end='') This also works if there ...


0

Assuming that your entries are always separated by exactly one blank line, you can do the following: awk 'BEGIN{RS="\n\n"; FS="\n"; OFS="\n"; ORS="\n\n";} {if ($1 ~ /\<select\>/) {print};}' file.txt Input file: select * from emp; select * from dept; selection end; select something more selected wrong select again * from class; Output: ...


0

If commands are ended by ; you could: awk -v RS=";" '/from/' file If you want just the table names after "from" you can try perl -0nE 'say join(";", /from\s+(\w+)/ig)' file


0

As Ziggy says, is not very clear what you need but maybe this AWK will do. awk ' /select \*$/{flag=1;print;next}/selection/{flag=0}flag' file.txt


0

There could be different ways to do that. What comes to my head right now is to use grep with it's -B & -A flags $ grep -B 1 -A 1 'from' file.txt If you want the empty line, set the -A to 2


2

Since it's either all spaces or all tabs you could pipe it to sed 'H;$!d;g;: m;/\n[^\n[:blank:]]/!s/\n[^\n]/\n/g;t m;s/.//' That's gnu sed (I don't think other seds support [\n]). It works by appending each line to the Hold buffer and then deleting it if it's not the last one ($!). On the last line it copies the hold space content over the pattern space ...


0

Here's another way with join/sort/uniq: join -1 1 -2 2 -a1 -e "0" -o 1.1 2.1 <(sort file1) \ <(cut -d' ' -f1 file2 | sort | uniq -c) Basically, it joins the names in file1 with the unique names and counts from file2 using 0 for the missing fields in file2. With awk I would run: awk 'NR==FNR{s[$1]++;next} {if ($1 in s) {print $1, s[$1]} else ...


2

test.awk: FNR == NR{ names[$1] next } ($1 in names){ ulog[$1]++ } END{ for(name in ulog){ print name ":" ulog[name] } } and run it as awk -f test.awk user.list user.log FNR==NR # does the file record number == the record number, if it does then we are still in the first file next # as we are still in the first file, skip the ...


2

Something simple like: mapfile -t names < file1 for name in "${names[@]}" do echo "${name}" $(grep -c "^$name " file2) done Will provide output like: Peht 2 Mawo 3 Stso 1 Makr 0 Bavo 2 The grep string says to anchor the username at the beginning (^) of the line, and enforce a trailing space after the line.


0

if you are only concerned with the first line of the output then limit what sed looks at by only addressing line 1: printf ' some text\n\n some more text\n' |sed '1s/^[ \t]*\([^ \t]\+.*\)$/\1/' this will ignore any white space at the beginning then match from anything that isn't white space plus the rest of the line and only on line 1.


0

I tried the following on a jessie pi and had no problem with stdbuf -oL in front of hexdump working as intended. while sleep .5;do echo -n x;done | stdbuf -oL hexdump -v -e '1/1 "%02x\n"' | ts %.s | cat The while is to simulate slow input, ts timestamps each line, and the cat is just to provide another pipe. The output is regular and the timestamps differ ...


0

If I understand correctly, there are two problems in your script: case sensitivity, matching parts of longer words. Moreover, instead of setting a variable you can use a comma operator. Concluding, the following should do what you wanted: awk -v IGNORECASE=1 '/select /,/;/' XXARXADLMT.txt


0

This will do even if your file is longer then the one you inserted : while read line; do if [[ "$line" == "select *" ]]; then echo "$line" read line echo "$line" read line echo "$line" fi done < XXARXADLMT.txt Or you can replace the repeating read line; echo $line; with : for i in {1..2}; do read line; echo $line; done


0

You can use the following code: awk -F"[>\n]" -v RS="" '{print $4"|"$3"|"$2}'


0

The same time comes from buffering. Try to find the command which buffers in the in the queue and try the command unbuffer.


4

A stream editor is a special type of filter. A filter is a program that takes text on standard input, does some magic, and spits it out on standard output. grep, and basically all of the coreutils are filters. A stream editor is a special type of filter: It applies one or more editing commands on the incoming text. In awk, the following three functions ...


0

Did you mean to do something like this? It's the only way i can think of to make sense of your script. awk -v OFS=$'\t' ' FNR == 1 { $5 = "sdev" ; print } FNR > 1 { a = $4 # field 4 is 'avg' n = NF-1 # exclude the 'avg' field from the ss calculations. for (i=1; i <= n; i++) { ss += ($i - a)^2 } $5 = ...


2

Where is "n"? You write: sd = sqrt(ss/n) but where in your code did you assign the variable "n"? The way awk sees it, "n" is zero. Also, where is column 5 in a=$5 (and, third issue, why is this assignment in the END section)? Your example contains only 4 columns.


0

Here's another solution: sed 's/^<[^>]*>//g' file.txt | tr '\012' '|' | awk -F'|' '{OFS=FS; print $3,$2,$1}' 9251000014|220095|Operation successful. If you want to strip the trailing . you could do that either with the sed or the awk (no need for both). Here's a drop-in solution with sed: sed -e 's/^<[^>]*>//g' -e 's/\.$//' file.txt


1

sed: sed -rn 's/<ResultDescxmlns.*>(.*)/\1/;N;N;s/\n/ /g;s/([^.]*). ([0-9]*) ([0-9]*)/\3|\2|\1/p' s/.*ResultDescxmlns.*>(.*)/\1/ : find the match for line .*ResultDescxmlns.*>(.*) get string after > in \1 N;N;: Add next 2 lines to to the current line s/\n/ /g: replace newline by space s/([^.]*). ([0-9]*) ([0-9]*)/\3|\2|\1/p : match ...


1

Using "perl" and Regular Expression perl -0777 -pe 's/<ResultDescxmlns=.*?>([^\n]*).([^\n]*).([^\n]*)/\3|\2|\1/gs' file.txt This will extract the 3 lines following the ResultDescxmlns tag and rearrange them into the desired order.


0

If the file.txt has only 3 lines, this works as well : l3=$(tail -1 file.txt) l2=$(tail -2 file.txt | head -1) l1=$(tail -3 file.txt | head -1 | cut -d">" -f2) echo $l3"|"$l2"|"$l1


0

This works on your data. Perhaps you can give more info on what you need. awk -F'>' '/ResultDescxmlns/{ a = $2; getline b; getline c; printf "%s|%s|%s\n",c,b,a }' <file.txt When it matches a word in a line, it splits the line on the char >, and puts field 2 in variable a. It calls getline to read the next line in variable b, and again c. It ...


3

Since the documentation is fairly explicit, I would simply file a bug report. Comparing with BWK (one-true-awk or original-awk), it behaves as the documentation implies. If gawk's developer had some other reference implementation in mind for this detail, it should be documented. Testing gawk's -c (compatibility mode), it treats RS as described in the ...


2

grep can do that: grep -v '^..00' inputfile


2

There are lots of ways, e.g., awk '/^..00/{ next; }{ print; }' < mydata sed '/^..00/d' < mydata


0

Looks like your work can be reduced to: perl -e ' use POSIX qw(strftime); my ($start, $num) = (30_000_000, 10_000); my @all = ( $start .. $start + $num -1 ); # 10,000 numbers my @odds = grep { $_ % 2 == 1 } @all; my @evens = grep { $_ % 2 == 0 } @all; my $filename = strftime "%Y%m%d", localtime; my $fh; open $fh, ...


3

Using "awk" This will print lines with N0 < LIMIT: # -v sets variables which can be used inside the awk script awk -v LIMIT=10 ' # We initialize two variables which hold the two previous lines # For readability purposes; not strictly necessary in this example BEGIN { line2 = line1 = "" } # Execute the following block if ...


1

awk can write to different files by itself. e.g. awk '{print > "new/"FILENAME}' *.txt You could use something like that inside your NormalizeFiles.awk script. If you don't want to type > "new/"FILENAME for every print statement, you could do something like this: awk 'FNR == 1 { out = "new/"FILENAME } ; { print > out }' *.txt That changes the ...


1

It appears you are trying to run input files through awk and save the results as a different file name. Unless you need very special file naming, this should serve: for i in *.txt; do awk -f NormalizeFiles.awk $i > $i.out done Although your question is tagged bash, your syntax, as someone pointed out, is not bash, e.g. the misuse of parentheses. ...


1

For the record, you could also do this with sed: sed -s '$!N /.*PATTERN.*\n/{/\n.*PATTERN/{x;/^1$/!s/.*/1/;b v};//!{x;/^1$/{s/./0/;b v};//!D}} //!{${/PATTERN/{x;/^1$/{b v}}};D;};: v;x;P;D' file1 file2 ... fileN That's gnu sed. With other seds you'd have to process one file at a time: sed '$!N # if not on the last line pull in the next ...


2

I'll use the same test file as thrig: $ cat file a pat 1 pat 2 b pat 3 Here is an awk solution: $ awk '/pat/ && last {print last; print} {last=""} /pat/{last=$0}' file pat 1 pat 2 How it works awk implicitly loops over every line in the file. This program uses one variable, last, which contains the last line if it matched regex pat. ...



Top 50 recent answers are included