awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
4answers
111 views

How to add a new text line at the first line of a file? [duplicate]

File: TABLE1 1234 9555 87676 2344 Expected output: Description of the following table: TABLE1 1234 9555 87676 2344
2
votes
2answers
137 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
-1
votes
2answers
32 views

errors using alias for an awk command

I'm attempting to add this alias to my .zshrc but run into errors. I've tried escaping the ==, ", and && values and still no luck. What am I missing? alias broke="ssh -t server@5.555.555.555 ...
2
votes
1answer
51 views

Can I determine type of an awk variable?

I have the gawk version of awk. In this part of gawk manual, it is stated that awk variables have "attributes", which are used to determine how to treat them in various operations. For example, a ...
0
votes
0answers
33 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...
-1
votes
1answer
36 views

Unix/Linux - File Formating

We have a source file which needs to be formatted into a fixed width file. Source File Format: Hdr1 Dtl1.1 Dtl1.2 Dtl1.3 Dtl1.4 Trailer0004 Hdr2 Dtl2.1 Dtl2.2 Dtl2.3 Dtl2.4 Dtl2.5 Trailer0005 . . . ...
-4
votes
2answers
28 views

How to process only certain columns and ignore header and footer in delimited text file using awk?

I have an input text file called text.txt: Header Column1|Column2|Column3 C11|C12|C13 C21|C22|C23 C31|C32|C33 Footer My output should be: Column1|Column3 C11|C13 C21|C23 C31|C33 My command that ...
3
votes
1answer
32 views

Replace a word after matched word with incrementing values

I have a file like this - ! interface csa2549abA xxxx yyyy zzzz ! interface aso23apwoeAe aaa bbb ccc ! interface FastEthernet0/1 ppp qqq rrr ! and so ...
2
votes
3answers
43 views

linux + ssh + perform two tasks in one ssh and print to CSV

the following command print the two values in one ssh shut ssh test01 "hostname && cat /etc/sysconfig/rhn/rhnsd " test01 INTERVAL=240 but how to change the syntax in order to print the ...
1
vote
1answer
17 views

How do I interpolate a text file of 3D coordinates (X,Y,Z) onto a regular grid?

I have a 3D surface in a text file which I need to plot on a regular X/Y grid. However, the values for X and Y are not regularly spaced and are not necessarily in ascending order. I need to regularly ...
4
votes
1answer
378 views

How do I sort words by syllable count?

How do I perform an ascending sort of a word list, based upon how many syllables each word contains? Example Input: something important dog cat bookshelf Example Output: dog cat something ...
1
vote
3answers
63 views

Delete the nth line from a matched string (which only occurs once in the file)

I have a question related to deletion of strings that appear at the n+2 position after the matched string which is at position n using awk for multiple files. I am able to print it using the command: ...
-1
votes
2answers
30 views

How to convert rows to columns based on cloumn value in awk in unix

I have data like below dn: uid=032096,OU=Customers,OU=People,O=B2C objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user objectClass: inetOrgPerson objectClass: ...
2
votes
3answers
83 views

Replace values in file

I have a files: $cat file1.txt 1234|W 1345|S 8427|D 2132|C 3243|V and my sql file is : $cat select.sql SELECT * FROM CUSTOMERS WHERE ID IN (FLAG); and I have a shell with the next ...
0
votes
3answers
31 views

split lines and print it to new line for certain character

I need to split lines and print it to next line when pattern match. Like I have: ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955 And I want it to print to new line when ABC pattern will come: ...
1
vote
4answers
65 views

Replace the specified string to its appearance time using sed or awk

I want to replace oooo to 1, 2, 3, ..., like a example below: From > cat demo.snippet snippet anova.cca "anova.cca fun" anova.cca( ,${oooo:object} ${oooo:,...} ,permutations = ...
-1
votes
4answers
48 views

How to filter the table names from .txt file [duplicate]

select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by using: awk '/select/{a=1} a; /;/{a=0}' ...
1
vote
3answers
46 views

Elegant way of counting how many times patterns from a file occur in another file

Imagine that we have two for example files. The first file is filled with unique names of employees created by combining the first two characters of the first name and the last 2 characters of the ...
-4
votes
3answers
52 views

I need a command to filter the statements from a .txt file

I have a file with the following contents: select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by ...
2
votes
2answers
72 views

Find (and strip) common initial whitespace from file/pipe

I am looking for a way to find and remove the common initial whitespace from a text stream or file. I do not want to remove all leading whitespace (that would be a simple sed 's/^[[:space:]]*//'). ...
0
votes
2answers
47 views

Binding Timestamp with Hexdump?

I am running a script which is given below, sudo hexdump -v -e '1/1 "%02x\n"' /dev/ttyUSB0 | Many Pipes | gawk '{ print strftime(":: %s"), $0 }' Here the data is being hex dumped with timestamp ...
3
votes
1answer
77 views

Use awk as a stream editor

awk is the swiss army knife of text manipulation. If, however, I need to change small bits in a text, I reach out to sed generally. While it might be the best tool for the job, it is worth it to ...
0
votes
0answers
41 views

awk, how to split on special characters

Background Using awk {GNU Awk 3.1.7} Must split log file with awk on special characters: (id= To illustrate my problem This works: echo "aa_bc" | awk '{{split($0,var,"_")} {print ...
-2
votes
2answers
65 views

standard deviation using awk [closed]

I am using below command to get standard deviation of file A names 1 2 3 avg 23.3107 20.0372 21.7236 21.6905 awk '{x[NR]=$0 ;} END{a=$4; for (i in x){ss += ...
0
votes
6answers
100 views

how to do formatting of a file in unix

I have a file file.txt. which have below three records. <ResultDescxmlns="http://www.mpaet.com/boe/interface/common">Operation successful. 220095 9251000014 Now my desired format should be as ...
4
votes
1answer
64 views

Why does Gawk in compatibility mode split records in the same way as when GNU extensions are enabled?

I'm not sure if I found a bug in Gawk or if I'm misreading / missing something in gawk's man page. According to gawk's man page, when in compatibility mode, if RS contains multiple characters, only ...
1
vote
2answers
19 views

Deleting lines by matching 3rd and 4th character only

I am running a script that fetches me the data as given below, 1400d2b8 1f4d1e2507 1200781e As you can see the 3rd and 4th character is 0 in line 1 and line 3. I am trying to write a single line ...
1
vote
1answer
32 views

How to output multiple lines using a single condition?

I have a file with the pattern: n0 n1 n2 ... ni ------------------------------- N0 N1 <empty line> if N0 is less than a specific number, I want: 2 lines above the N1's line N1's line Empty ...
1
vote
2answers
50 views

How to apply an awk script to multiple files in one folder using a loop

I am trying to apply one awk script to all the files in one folder, I am using: for %%i in (*.txt) do (awk -f NormalizeFiles.awk "%%i" > new\"%%i") but then I get the message: Syntactic error ...
3
votes
4answers
69 views

Find matches on adjacent lines

I want to find adjacent matching lines, e.g., if the pattern matches are $ grep -n pattern file1 file2 file3 file1:10: ... file2:100: ... file2:1000: ... file2:1001: ... file3:1: ... file3:123: ... ...
2
votes
1answer
34 views

grep a range of unix timestamps from a logfile

I have a logfile where every log event has a unix timestamp: 1460332800 event a 1460332999 event b 1460342800 event c 1460417039 event d Is there a clever unix-style way to grep all log events ...
5
votes
3answers
256 views

How to subtract 1 from numbers matched by regular expression?

I'm trying to write a script that reduces each number in line by "1", but I'm getting all "0"s instead: awk '{a=gensub(/([0-9]+)/,"\\1","g",$0); if(a~/[0-9]+/) {gsub(/[0-9]+/,a-1,$0);} ...
0
votes
1answer
31 views

Printing all lines around a string until matching another string [duplicate]

I have an example dataset where a section has fields beginning with "something". I would like to obtain all of the lines in each "something" section when matching a specific string of "1234". I was ...
6
votes
3answers
78 views

sed/awk subscript numbers in chemical formulas using markup tags

I have hundreds of text files that include chemical formulas together with narrative including numerical values. The formulas are always preceded by white spaces but can be followed by white spaces, ...
2
votes
1answer
36 views

grep/awk specific lines based on specific fields; using ksh variable with awk

I have this input file: file_in.txt (delimited by pipe) 3345:tyg|rty|27|0|0|ty6|{89|io|}62|0 3346:tyg|rtyuio|63|0|1|ty6|{89|gh|}45|0 3347:tyu|ray|24|0|0|ty6|{89|uh|}27|0 ...
1
vote
1answer
23 views

Appending a value to all columns

I have a "simple" question yet I have not found an answer to it nor have I gotten my code to work. I have a bunch of columns, and I want to append the average of the last column to ALL columns. For ...
0
votes
1answer
34 views

Using Perl for Even/Odd numbers

Currently, I'm using a process to create a very specific set of four files. The Process I'm using is shown in the attached Process Image, below: The first line is part of a solution, given to a ...
1
vote
3answers
45 views

Need help defining a variable in awk

I have an input file thus: 3.59717487E+05 3.40210880E+06 4075.32 7066.00 4075.32 7066 4075.322 2 a_final_psdm_LY1-1250_20160307 3.59725248E+05 3.40211860E+06 ...
1
vote
2answers
51 views

I want search for a string and to pull all the lines between two patterns [duplicate]

I want search for a string and to pull all the lines between two patterns(including the pattern). START#1 Name S 101 Frankfurt_1_1 19 A+ Street END START#2 Name B 104 Berlin 20 A+ ...
4
votes
1answer
77 views

How Do I Remove Duplicate Words With Suffixes?

How do I compare a list of unsorted words of variable length and remove duplicate words that have a suffix? Example word list: iron curl curled railroad curler ...
1
vote
1answer
37 views

How to add an word for a line where transpose has done

I have a file fruits: apple grapes banana orange By using cat < fruits | tr "\n" "," It will give output as apple,grapes,banana,orange, Here I want to print the file name also along with ...
2
votes
2answers
41 views

Using awk to combine two csv files on three fields too create a new csv file

I would like to compare fields in 2 csv file and create new csv file for output. I have 2 csv files; this first file is PCs in my network: ...
3
votes
2answers
88 views

Search, insert, move and insert

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's AGE line and move that line to the top ...
0
votes
0answers
16 views

I want to search for a pattern in a file. If the pattern matches, it should check for second pattern and replace it with another pattern file [duplicate]

I want to search for a pattern in a file. If the pattern matches, it should check for second pattern and replace it with another pattern INPUT FILE Name#AAAAA std 1 sec c age 5 Name#BBBB std 2 ...
1
vote
1answer
47 views

I want to search and replace a pattern

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's age and insert a line age NIL at the end ...
0
votes
2answers
49 views

calculate ratio using awk

awk 'NR==1{$4="ratio"}NR>1{$4 = ($3)/($2)} {print $1 "\t" $2 "\t" $3 "\t" $4}' A B awk: cmd. line:1: (FILENAME=A FNR=18) fatal: division by zero attempted Can anybody help with this error? ...
1
vote
3answers
61 views

Bash script to replace some words present in one file from another file

I have two files, one, file1 is 1000 lines with router configurations in it. A portion of it is like this - ! ipv6 prefix-list CCAV6 seq 5 permit 2217:AB23::/40 route-map DENY-ALL deny 11 ! ip ...
0
votes
0answers
20 views

Printing specific column in bash script and avoiding duplicates [duplicate]

I am writing a Bash script to print a specific column from a file. But that column contains many same strings and I want to just print every content just once. I tried this - awk '!seen{print $9}' ...
2
votes
2answers
120 views

bash source script.sh with exit 0

I use a lot of external files in my scripts. But I have a problem, that if I source a file with exit in the contents of the file, the script exits from loading and the strings that I want to use are ...
1
vote
4answers
231 views

print specific line from multiple files

I have a collection of 5000+ files and I just want to create a output.txt file which contain 27th line of all file along with filename. well I am noob at linux level so what i got from internet is to ...