awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
29 views

Splitting Files via AWK

I found the following awk script in a previous answer for splitting large files into smaller parts based on a pattern (replacing "header-line" with a different pattern). However, I have the need to ...
1
vote
4answers
75 views

rewritting text with automatic change with other file

I am trying to rewrite a file text, "main.txt" with changes from "changes.txt". The main.txt file is like this: word_1 genre_A word_2 genre_A word_3 genre_B word_4 genre_C word_5 genre_A And the ...
1
vote
1answer
45 views

Better way to apply formula to specific column of inputfile, and pass-through all other columns as-is

I regularly need to apply simple formulas to input data give in textual tabular form. In the beginning, I wrote specific little functions for each use-case. Then I re-factored the "common code" into ...
-3
votes
0answers
30 views

Unix Line calculation in some fields of a file [on hold]

My input file is comma delimited with structure like: f1 f2 f3 f4 f5=f1-f3+f4 1000 wk1 500 200 700 1500 wk2 400 100 1200 1300 wk3 300 50 1050 900 wk4 400 100 600 ...
-3
votes
0answers
32 views

How to do calculation on current line based on the output of previous line in unix? [on hold]

My input file is like: Sales,week,INV,purchase,EOH 1000,w1,500,200,700 1500,w2,400,100,1200 1300,w3,300,50,1050 900,w4,400,100,600 I need o/p file like, for week 1: f1,f2,f3,f4 1000,500,200,700 ...
1
vote
3answers
49 views

Getting substrings in between delimiters multiple times from one string without looping

I have looked all over for a solution to this, but I couldn't find one. I am using qstat -x to pass a large string of job information. There is only whitespace in this output in certain areas. The ...
1
vote
2answers
68 views

how to use \n( and )\n as delimiter in awk

I have a file which contains below data 7[label = "ScanStep: T_b0\n(T.a = 1)\na\nb\nc\nd\ne\ndob\ntimestamp1\nUnc: Integer(8)\nUnc: Integer(8)\nUnc: Varchar(80)\nUnc: Numeric(10,2)\nUnc: Varbinary(80)...
-7
votes
0answers
40 views

How to get specific information out of /etc/passwd and display it into a file? [on hold]

Alright so I posted a previous POST which I figured out half my problem which I had was to get information like the user, hom dirs and bins to show and display out of /etc/passwd, this was done by: ...
1
vote
1answer
61 views

Need to strip off the comments in double quotes

I have test case which contains string message in double quotes and I want to strip off data in double quotes. I used awk command to strip off comment section data but it doesn't work awk '/^"/ { ...
1
vote
3answers
42 views

Reformatting negative numbers with sed or awk

I have a text file that contains some negative numbers except, they are formatted as: 00000012- With the hyphen after the number. I am processing the file using awk to reformat it into SQL insert ...
0
votes
2answers
42 views

How to add a column of numbers to a array data file

I have a data file with two columns of numbers: input file 4.182 4.1843 4.184 4.2648 4.2281 4.0819 4.2204 4.1676 4.0482 4.1683 4.0156 4.2895 4.4504 5.2369 4.3776 4.4979 4.3797 4.1372 4....
0
votes
1answer
25 views

How to multiply two columns in a file by a constant number

I have a file like this. input data 4.2394 4.4569 4.2427 4.1011 4.2879 4.1237 4.2106 4.4844 4.2373 4.1071 4.1322 4.0502 4.3103 4.4255 4.4342 4.5262 I need to multiply each element by a constant ...
2
votes
2answers
85 views

BASH: Using awk to filter unique lines results in 0 length array

Note: Thank you to Jeff Schaller and steeldriver. But as neither posted as an answer, I'm not sure how to mark as solved. I now have a better understanding of pipes/subshells. I'm pretty sure I once ...
3
votes
3answers
39 views

awk to cut portion of a field and still print entire line

I have this line: 08:30:02.480507 IP 192.168.100.150.65119 > 192.168.100.151:53 59865 [1au] A? click.em.redbox.com. (48) I need to cut out the ".480507" in the timestamp so the line looks like ...
14
votes
1answer
710 views

Why does awk print the whole line when I want it to print a variable?

When I type: echo "foo bar" | gawk -v var="content" '{print $var}' I expect the output to be content. Why is it foo bar?
1
vote
3answers
81 views

Find and Replace with command line

my log.txt file includes ... mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 ... how can I replace mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 with mynetworks = 127.0....
0
votes
1answer
24 views

Setting Threshold on log Output

I have to check the log & monitor the I/O output, the log is generated per second I seems a hard job to monitor the log which get generated every second, I am looking for some awk command which ...
0
votes
2answers
45 views

Read a file using shell script and make 2 new files based on Name of item

I have below file with me :- ====== 20160606:034441 ====== Mango(Test) TestName MangoT Row 0 Season N Name Safeda Location Delhi ====== 20160606:034441 ====== Mango(Result) TestName ...
-2
votes
1answer
50 views

filtering pattern in a file using sed or grep

I'm new to Linux. I'd like to store all the unique usernames present in the extended.log file into a new file using the commands awk, grep, and/or sed. The following are the field-names in my file, ...
1
vote
2answers
49 views

Make the 1st column as header and list its corresponding values using shell script

I have a large file contents are as shown below :- Quantity 20589 Quantity 12297 Quantity 100346 Quantity 0 Quantity 141999 Quantity 23662 Quantity 551071 Quantity 72917 ...
2
votes
1answer
29 views

Merging specific columns of 2 files based on field match

I know I'm not the first one asking this but my code still does not work: File 1: gi|1283| tRNAscan exon 87020 88058 . - . transcript_id "Parent=tRNA-Tyr5.r01"; gi|3283| tRNAscan exon 97020 97058 . +...
0
votes
4answers
47 views

Find a text in a file and Insert a text after it

I have a text file. The text file contains the below content. ServerName ServerAdmin webmaster@localhost DocumentRoot I need to insert a text www.mydomain.com, after the ServerName. Is there any ...
2
votes
2answers
106 views

Join seven files with awk line-by-line

I have seven (or eight and so on) files with same number of lines. file1 1.001 1.002 1.003 1.004 file2 2.001 2.002 2.003 2.004 file3 3.001 3.002 3.003 3.004 etc. Desired output: 1.001;2....
2
votes
1answer
124 views

Read from and append to file at the same time while preserving end-of-line

Conceptually I have an easy task... I have [loosely] structured data in a file: Testing: debug, default CXXFLAGS <100's of additional output lines> Testing: release, default CXXFLAGS <100's ...
3
votes
2answers
348 views

grep: How to find Closing Bracket?

One part of /usr/share/X11/xkb/symbols/us starts with xkb_symbols "dvorak" { and ends with the closing curly bracket }; which line-number I want to find. partial alphanumeric_keys xkb_symbols "...
1
vote
1answer
44 views

Make an IF condition out of result we get from remote server

I need to check for files availability in remote server, and based on the files availability i need to make an if condition. I have tried the below commands in a shell script to list the files in the ...
0
votes
0answers
21 views

awk - how to pass dynamic value in pattern? [duplicate]

I am passing following date pattern to retrieve log details for particular date from the application log file. awk '/2016-06-15/{print $0}' $application.log >> $LOG_FILE I want date value as ...
2
votes
1answer
53 views

mysterious awk script joins lines

I have a text file that has blocks of consecutive non empty lines separated by a blank line. I want to join the lines in a block (similar to the "J" key in the vi editor). Here I found the following ...
2
votes
2answers
73 views

Grep for dates older than 14 days in a file [duplicate]

I'm trying to find a way to grep how many messages in a file are older than 14 days and have a value of number of results return Example going with today's date of 20160616. $grep 'Put Date' filename:...
-8
votes
2answers
49 views

how to print in a single line with pattern matched

student:xxxxx student:yyyyy student:zzzzz student:kkkkk student:. teacher:aaaaa teacher:bbbbb teacher:lllll teacher:. student:sssss student:mmmmm student:. output should be, student:xxxxx yyyyy ...
1
vote
2answers
28 views

Insert text inside stanza at the end of the data string

I have a file I need to echo text into at the end of the file from the command line, the problem is the data is in a stanza, all within curly braces. This is the end of the file: text-string1,text-...
1
vote
2answers
57 views

Creating a text file from other text files

I have a text file like this word_1 root_of_word_A word_2 root_of_word_B word_3 root_of_word_C word_4 root_of_word_A And other document like this root_of_word_A root_of_word_1 root_of_word_B ...
-1
votes
1answer
29 views

How to record numbers of array elements based on selection comparision criteria

I have many files which has many 2x2 arrays. In the present example I show only one file (file1) Input file(file1): 1: 6.1703 541.631 46.0391 2: 6.1930 537.446 45.9239 3: 6.1931 177.171 288.579 ...
3
votes
2answers
72 views

Save the matches of grep as an array

I have a file with the following contents: FILETYPE=A:B:C:D How can I grep out A, B, C and D and save them as an array in a ksh script on Linux? I tried FILETYPES=`grep "FILETYPE" ${CONF_FILE} | ...
1
vote
2answers
45 views

print multiple words seperated by space

I have a text file and the data inside has format like (name age country): michael jordans 25 US adam smith 30 UK chris wood ABC 22 Aus if I use command: cat text.txt | awk {'print $1'} --> it ...
2
votes
2answers
64 views

How to get the last occurrence of lines between two patterns from a file?

I have a log file which reports on the output of a process, I'd like to extract all lines from between the last occurrence of two patterns. The patterns will be along the lines of; Summary process ...
4
votes
2answers
40 views

Is it possible to print an iterated range of arguments via cli from a csv? (see example)

I have hundreds of logs spread across a few thousand VM's and I'm trying to create a few scripts to quickly scan logs and I have figured out the majority of the presentation and wanted to know if ...
0
votes
1answer
32 views

difference between [-0-9] and [-0-9 ] pattern in awk command

What is difference between the following commands: awk '/[-0-9]/ {print $0}' data_file and: awk '/[-0-9 ]/ {print $0}' data_file the second command differs by a space in [-0-9] pattern from the ...
-5
votes
3answers
98 views

How to add all the numbers of a specific column in UNIX without using awk command?

Let us take a sample file with the contents below, an expense report containing items and their prices: $ cat file Item1,200 Item2,500 Item345,900 Item20896,800 Item111,600 Now, a friend challenged ...
1
vote
0answers
24 views

Read a log file to extract various fields and count the occurance [closed]

I have a logger file like this: 2016-06-11 07:34:01.542 98459 [Thread-23-Job-parser-bolt] INFO JobLoader c.t.commerce.common.utils.APIUtils - Enrichment data updated successful for partnumber : ...
2
votes
2answers
126 views

Multiplying numbers in-place in a file

I have ASCII file contain numbers as follow: 1 2 3 5 6 9 7 8 9 . . . . . . . . . . . . . . . etc I want to upgrade this file by multiplying each value by "3" to get : 3 6 9 15 18 27 21 24 27 ...
3
votes
3answers
90 views

How to do a loop to process two input files

I am using awk to do match two files and then multiply the elements of file2 and file1. awk 'NR == FNR{a[$1]=$2; b[$1]=$3; next} /:/ || !NF{print; next} {print $1, $2*a[$1], $2*b[$1]}' ...
1
vote
2answers
80 views

After small code change 'awk' prints header and blank lines, (not header repeated)

data-file content: k = 0.1667 0.0962 0.0000 ( 4922 PWs) bands (ev): -7.1689 -7.1043 -6.1682 -6.0658 1.1309 1.5196 3.0894 3.1087 3.3846 3.7588 4.2143 4.2220 4.3882 ...
0
votes
3answers
68 views

Awk command to get the next line from the matched pattern and crop the string in between two patterns

I need to get the next line after matched pattern and need cut or crop the value in between two patterns from that line. Sample Source file <h2>Commodity Information</h2> <dl> ...
0
votes
3answers
30 views

Extract a part of one line and sort

I have output similar to the following: [test output] my name is bob this is a random string [test output] my name is jim this is a randong string2 [test output] my name is bob this is a randong ...
1
vote
2answers
54 views

How to check if some recurring text is missing in a text file?

I have a text file that looks like this: { "mimeType": "web", "body": "adsfdf", "data_source_name": "abc", "format": "web", "url": "http://google.com/", "urls": "...
1
vote
1answer
64 views

stat(FILENAME) in awk

Title say it all. How can I get file attribute with awk ? or Is there an equivalent to c/php stat() function ? obviously when awk'ing a pipe, file attribute would be undefined. I know I can use ...
-2
votes
2answers
37 views

Data Manupulation required your help

I have a data as mentioned below. host_name Server1.domain.com contacts DL - Desktop contact_groups ravi, raj, rahim host_name Server2.domain.com contact_groups DL-Server host_name Server3....
1
vote
1answer
66 views

Convert vertical array data to horizontal array data

I have a file with 25 matrices in a file, and each matrix is has 128 rows and 5 columns. Each matrix are putted one after the other, in a vertical way. Like this File input(file1): Array1 Array2 ...
1
vote
1answer
42 views

Awk hangs during math operation on Solaris

I'm working on a SunOS 5.11/Solaris 11.3 machine. I have a bash script that calculates and exports CPU frequency because I use it frequently in some test scripts. Here are the two lines of interest: ...