awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
63 views

Need to strip off the comments in double quotes

I have test case which contains string message in double quotes and I want to strip off data in double quotes. I used awk command to strip off comment section data but it doesn't work awk '/^"/ { ...
1
vote
3answers
51 views

Reformatting negative numbers with sed or awk

I have a text file that contains some negative numbers except, they are formatted as: 00000012- With the hyphen after the number. I am processing the file using awk to reformat it into SQL insert ...
2
votes
2answers
99 views

BASH: Using awk to filter unique lines results in 0 length array

Note: Thank you to Jeff Schaller and steeldriver. But as neither posted as an answer, I'm not sure how to mark as solved. I now have a better understanding of pipes/subshells. I'm pretty sure I once ...
0
votes
2answers
52 views

How to add a column of numbers to a array data file

I have a data file with two columns of numbers: input file 4.182 4.1843 4.184 4.2648 4.2281 4.0819 4.2204 4.1676 4.0482 4.1683 4.0156 4.2895 4.4504 5.2369 4.3776 4.4979 4.3797 4.1372 4....
0
votes
1answer
30 views

How to multiply two columns in a file by a constant number

I have a file like this. input data 4.2394 4.4569 4.2427 4.1011 4.2879 4.1237 4.2106 4.4844 4.2373 4.1071 4.1322 4.0502 4.3103 4.4255 4.4342 4.5262 I need to multiply each element by a constant ...
3
votes
3answers
42 views

awk to cut portion of a field and still print entire line

I have this line: 08:30:02.480507 IP 192.168.100.150.65119 > 192.168.100.151:53 59865 [1au] A? click.em.redbox.com. (48) I need to cut out the ".480507" in the timestamp so the line looks like ...
1
vote
3answers
89 views

Find and Replace with command line

my log.txt file includes ... mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 ... how can I replace mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 with mynetworks = 127.0....
2
votes
1answer
29 views

Merging specific columns of 2 files based on field match

I know I'm not the first one asking this but my code still does not work: File 1: gi|1283| tRNAscan exon 87020 88058 . - . transcript_id "Parent=tRNA-Tyr5.r01"; gi|3283| tRNAscan exon 97020 97058 . +...
-2
votes
1answer
55 views

filtering pattern in a file using sed or grep

I'm new to Linux. I'd like to store all the unique usernames present in the extended.log file into a new file using the commands awk, grep, and/or sed. The following are the field-names in my file, ...
0
votes
1answer
28 views

Setting Threshold on log Output

I have to check the log & monitor the I/O output, the log is generated per second I seems a hard job to monitor the log which get generated every second, I am looking for some awk command which ...
0
votes
2answers
46 views

Read a file using shell script and make 2 new files based on Name of item

I have below file with me :- ====== 20160606:034441 ====== Mango(Test) TestName MangoT Row 0 Season N Name Safeda Location Delhi ====== 20160606:034441 ====== Mango(Result) TestName ...
1
vote
2answers
54 views

Make the 1st column as header and list its corresponding values using shell script

I have a large file contents are as shown below :- Quantity 20589 Quantity 12297 Quantity 100346 Quantity 0 Quantity 141999 Quantity 23662 Quantity 551071 Quantity 72917 ...
0
votes
3answers
51 views

Find a text in a file and Insert a text after it

I have a text file. The text file contains the below content. ServerName ServerAdmin webmaster@localhost DocumentRoot I need to insert a text www.mydomain.com, after the ServerName. Is there any ...
2
votes
2answers
110 views

Join seven files with awk line-by-line

I have seven (or eight and so on) files with same number of lines. file1 1.001 1.002 1.003 1.004 file2 2.001 2.002 2.003 2.004 file3 3.001 3.002 3.003 3.004 etc. Desired output: 1.001;2....
3
votes
2answers
369 views

grep: How to find Closing Bracket?

One part of /usr/share/X11/xkb/symbols/us starts with xkb_symbols "dvorak" { and ends with the closing curly bracket }; which line-number I want to find. partial alphanumeric_keys xkb_symbols "...
0
votes
3answers
3k views

How to convert Log file to CSV using bash

Is it possible to turn this log entry into a properly formatted csv file using sed/awk? I've tried multiple ways to do this using sed/awk. Alas I couldn't do it. Before I do it in perl. is this ...
1
vote
1answer
58 views

Make an IF condition out of result we get from remote server

I need to check for files availability in remote server, and based on the files availability i need to make an if condition. I have tried the below commands in a shell script to list the files in the ...
5
votes
3answers
8k views

How to apply the same awk action to different files?

I am new in awk and I do not know if it is possible to write an awk script that does this: I have hundred of data files that I have to sort. For each one I use the following one-liner: awk 'ORS=NR%3?...
-9
votes
2answers
50 views

how to print in a single line with pattern matched

student:xxxxx student:yyyyy student:zzzzz student:kkkkk student:. teacher:aaaaa teacher:bbbbb teacher:lllll teacher:. student:sssss student:mmmmm student:. output should be, student:xxxxx yyyyy ...
0
votes
0answers
21 views

awk - how to pass dynamic value in pattern? [duplicate]

I am passing following date pattern to retrieve log details for particular date from the application log file. awk '/2016-06-15/{print $0}' $application.log >> $LOG_FILE I want date value as ...
2
votes
1answer
57 views

mysterious awk script joins lines

I have a text file that has blocks of consecutive non empty lines separated by a blank line. I want to join the lines in a block (similar to the "J" key in the vi editor). Here I found the following ...
2
votes
2answers
88 views

Grep for dates older than 14 days in a file [duplicate]

I'm trying to find a way to grep how many messages in a file are older than 14 days and have a value of number of results return Example going with today's date of 20160616. $grep 'Put Date' filename:...
0
votes
2answers
330 views

Find line within file matching pattern and make a substitution

I'm trying to simplify my life here but I'm stuck. I have a file containing multiple lines. Each line has a user and a number. I'd like to know a bash one-liner to search for a user and change the ...
25
votes
12answers
12k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
1
vote
1answer
75 views

stat(FILENAME) in awk

Title say it all. How can I get file attribute with awk ? or Is there an equivalent to c/php stat() function ? obviously when awk'ing a pipe, file attribute would be undefined. I know I can use ...
1
vote
2answers
30 views

Insert text inside stanza at the end of the data string

I have a file I need to echo text into at the end of the file from the command line, the problem is the data is in a stanza, all within curly braces. This is the end of the file: text-string1,text-...
3
votes
2answers
75 views

Save the matches of grep as an array

I have a file with the following contents: FILETYPE=A:B:C:D How can I grep out A, B, C and D and save them as an array in a ksh script on Linux? I tried FILETYPES=`grep "FILETYPE" ${CONF_FILE} | ...
1
vote
2answers
57 views

Creating a text file from other text files

I have a text file like this word_1 root_of_word_A word_2 root_of_word_B word_3 root_of_word_C word_4 root_of_word_A And other document like this root_of_word_A root_of_word_1 root_of_word_B ...
-1
votes
1answer
29 views

How to record numbers of array elements based on selection comparision criteria

I have many files which has many 2x2 arrays. In the present example I show only one file (file1) Input file(file1): 1: 6.1703 541.631 46.0391 2: 6.1930 537.446 45.9239 3: 6.1931 177.171 288.579 ...
1
vote
2answers
48 views

print multiple words seperated by space

I have a text file and the data inside has format like (name age country): michael jordans 25 US adam smith 30 UK chris wood ABC 22 Aus if I use command: cat text.txt | awk {'print $1'} --> it ...
8
votes
2answers
17k views

Extract file name from path in awk program

I have an awk script and I have passed a CSV file to it. awk -f script.awk /home/abc/imp/asgd.csv What am I doing is to get FILENAME within script.awk. FILENAME gives me the whole path. As I am in ...
2
votes
2answers
79 views

How to get the last occurrence of lines between two patterns from a file?

I have a log file which reports on the output of a process, I'd like to extract all lines from between the last occurrence of two patterns. The patterns will be along the lines of; Summary process ...
4
votes
2answers
42 views

Is it possible to print an iterated range of arguments via cli from a csv? (see example)

I have hundreds of logs spread across a few thousand VM's and I'm trying to create a few scripts to quickly scan logs and I have figured out the majority of the presentation and wanted to know if ...
-5
votes
3answers
110 views

How to add all the numbers of a specific column in UNIX without using awk command?

Let us take a sample file with the contents below, an expense report containing items and their prices: $ cat file Item1,200 Item2,500 Item345,900 Item20896,800 Item111,600 Now, a friend challenged ...
0
votes
1answer
42 views

difference between [-0-9] and [-0-9 ] pattern in awk command

What is difference between the following commands: awk '/[-0-9]/ {print $0}' data_file and: awk '/[-0-9 ]/ {print $0}' data_file the second command differs by a space in [-0-9] pattern from the ...
2
votes
2answers
134 views

Multiplying numbers in-place in a file

I have ASCII file contain numbers as follow: 1 2 3 5 6 9 7 8 9 . . . . . . . . . . . . . . . etc I want to upgrade this file by multiplying each value by "3" to get : 3 6 9 15 18 27 21 24 27 ...
1
vote
0answers
25 views

Read a log file to extract various fields and count the occurance [closed]

I have a logger file like this: 2016-06-11 07:34:01.542 98459 [Thread-23-Job-parser-bolt] INFO JobLoader c.t.commerce.common.utils.APIUtils - Enrichment data updated successful for partnumber : ...
6
votes
4answers
47k views

How to use regex with AWK for string replacement?

Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem Statement and Basic Definitions 23" "#2") ("Exercises 31" "#30") ("Notes and References 42" "#34")) ) ...
1
vote
3answers
82 views

Creating a new file inside awk but in different directory based on input field value

I am trying to work with awk datetime=`date +\%Y\%m\%d` logdatetime=`date +\%Y\%m\%d` foldermonth=`date +\%B_\%Y` folderday=`date +\%d` inputdir="~/sboper/Standalone/fsplit/GLMS" ...
3
votes
3answers
96 views

How to do a loop to process two input files

I am using awk to do match two files and then multiply the elements of file2 and file1. awk 'NR == FNR{a[$1]=$2; b[$1]=$3; next} /:/ || !NF{print; next} {print $1, $2*a[$1], $2*b[$1]}' ...
0
votes
2answers
63 views

Comparing two lines in BASH using awk

I have text file like this. 2015-11-24 12:59:37.112 128.206.6.136 source 2014-11-24 12:59:36.920 8.8.8.8 source 2014-11-24 14:59:38.112 23.234.22.106 destination 2014-11-24 13:59:37.113 23.234.22.106 ...
1
vote
2answers
84 views

After small code change 'awk' prints header and blank lines, (not header repeated)

data-file content: k = 0.1667 0.0962 0.0000 ( 4922 PWs) bands (ev): -7.1689 -7.1043 -6.1682 -6.0658 1.1309 1.5196 3.0894 3.1087 3.3846 3.7588 4.2143 4.2220 4.3882 ...
3
votes
4answers
738 views

how to change one column and add other columns in a row by awk

I have a file MDD.TXT with following records. file : MDD.TXT 0|ABC|055309|20120929053309|11965150 0|ABC|061434|20120929063434|11155537 0|ABC|070228|20120929073228|11965150 0|ABC|082213|...
0
votes
3answers
74 views

Awk command to get the next line from the matched pattern and crop the string in between two patterns

I need to get the next line after matched pattern and need cut or crop the value in between two patterns from that line. Sample Source file <h2>Commodity Information</h2> <dl> ...
-1
votes
2answers
165 views

How to sum many numbers inside 2D array using awk

I need to sum numbers located in a file like this one: column1 column2 column3 row1 a(1,1) a(1,2) a(1,3) row2 a(2,1) a(2,2) a(2,3) row3 a(3,1) a(3,2) a(3,3) row4 a(4,1) ...
1
vote
1answer
53 views

Awk hangs during math operation on Solaris

I'm working on a SunOS 5.11/Solaris 11.3 machine. I have a bash script that calculates and exports CPU frequency because I use it frequently in some test scripts. Here are the two lines of interest: ...
0
votes
3answers
32 views

Extract a part of one line and sort

I have output similar to the following: [test output] my name is bob this is a random string [test output] my name is jim this is a randong string2 [test output] my name is bob this is a randong ...
1
vote
2answers
62 views

How to check if some recurring text is missing in a text file?

I have a text file that looks like this: { "mimeType": "web", "body": "adsfdf", "data_source_name": "abc", "format": "web", "url": "http://google.com/", "urls": "...
-2
votes
2answers
39 views

Data Manupulation required your help

I have a data as mentioned below. host_name Server1.domain.com contacts DL - Desktop contact_groups ravi, raj, rahim host_name Server2.domain.com contact_groups DL-Server host_name Server3....
1
vote
1answer
81 views

Convert vertical array data to horizontal array data

I have a file with 25 matrices in a file, and each matrix is has 128 rows and 5 columns. Each matrix are putted one after the other, in a vertical way. Like this File input(file1): Array1 Array2 ...