Questions tagged [gawk]

gawk is the GNU implementation of awk. The awk utility interprets a special-purpose programming language that makes it possible to handle simple data-reformatting jobs with just a few lines of code.

Filter by
Sorted by
Tagged with
2
votes
3answers
1k views

awk: extract string from a field [closed]

in the input fields are separated by pipe sign: CCCC|Sess C1|s1 DA=yy07:@##;/u/t/we DDDDD|Sess C2|s4 DB=yy8:@##;/u/ba I want to get output where last field is changed (extracted only what is between ...
2
votes
4answers
136 views

How to break line after matching case but needs to go back a couple of characters

I have the following sample: 1. TNT 00:00 2. Swing From The Gutters 7:34 3. Ten-Day Interval 13:29 4. I Set My Face To The Hillside 18:16 5. The Equator 24:26 6. A Simple Way To Go Faster Than Light ...
2
votes
3answers
528 views

How to repeat a sequence of numbers in a column using awk command

My input file contains data like this. //address: 9050004 data: 1 id: 1 len: 0 size: 2 //address: 9050008 data: 1 id: 2 len: 0 size: 2 //address: 902e004 data: 1 id: 3 len: 0 size: 2 ... //address: ...
2
votes
3answers
558 views

Replace the data in a column in a file with the data in a row from another file

I'm new working with linux enviroment and i need to write a script to process some data. I have a file that looks like this: File A: 1 X 1 1.0 2 X 1 1.0 3 X 1 1.0 4 Y 2 ...
2
votes
3answers
2k views

How to get the last occurrence of lines between two patterns from a file?

I have a log file which reports on the output of a process, I'd like to extract all lines from between the last occurrence of two patterns. The patterns will be along the lines of; Summary process ...
2
votes
2answers
453 views

Why do newer versions of awk print large integers instead of decimals?

I often use the awk (gawk) calculator: calc(){ awk "BEGIN{ print $* }" ;} In Ubuntu 14.04, it output big numbers in standard form, e.g. $ calc 56777654409823*4674990588446667952594759939814064128 2....
2
votes
2answers
196 views

Another awk(ward) question - mapping substrings with multi-keys

I have two files that look like this: mapping.txt: 100178 L08/2015 100180 L08/2015 103822 J03/2012 103823 A06/2012 103823 J02/2012 103823 J03/2012 ... input.txt: 100180011|225| 100180011|226|...
2
votes
2answers
2k views

Add string to list using either AWK or SED?

Hi I want to be able to edit the list of name like this using either AWK or SED. Sample input list file: john paul rose lily Desired output: I am john of earth; I am paul of earth; I am rose of ...
2
votes
4answers
590 views

awk question to process lines with zeros across columns

Hi I have a file with the following format: filename is foo.txt abcd 1 0 0 2 abef 2 0 0 0 able 2 1 2 0 able 1 0 0 0 efgh 0 0 0 0 I wrote an awk script to delete the row with all columns with 0. But ...
2
votes
2answers
3k views

gawk: Summing floats formatted with comma separator

I'm trying to sum floats which use the comma as separator, and gawk does integer calculations. Here's the script: BEGIN { FS=";" OFS=";" CONVFMT = "%2.2f" } { print $1 print $2 print $1+$...
2
votes
3answers
93 views

AWK - How to count stored or index on array

I have data that need to be count how many occurrences. 1 2 1 2 3 1 And used the awk command below and work properly. awk -F'","' '{ a[$1]++ } END { for (n in a) print n "=" a[n] } ' text.txt And ...
2
votes
4answers
472 views

Convert a column in-place on CSV

I need some help on date conversion in-place on a CSV. Basically, I was capable to convert the column and save in a new file, but I was unable to save it on the original CSV. I have a CSV with date ...
2
votes
3answers
727 views

access elements in string array - stray @ symbol

I have created an array (3 elements), and each element contains a comma delimited string. The array was created by reading a file line by line - the file contains fields output from a database. I have ...
2
votes
1answer
169 views

Keep lines containing only 3 fields

I have a .txt file in this format: USER:EMAIL:PHONE but some lines only have USER:EMAIL I want to extract only the lines containing 3 fields, example USER:EMAIL:PHONE
2
votes
1answer
1k views

awk exit code if regular expression did not match

I want to get exit code 1 if the 4th column did not match the regular expression, but it seems that awk will return 0, even though the regular expression did not match. Any idea how to make awk ...
2
votes
4answers
437 views

Can Awk read escaped newlines? I.e. read past newlines

I have input looking like this: entry1line1 entry2line1\ entry2line2\ entry2line3 entry3line1 I would like to read such input into an array in AWK, to then process it together with the embedded ...
2
votes
1answer
116 views

Print random lines of file but with maximum count of a certain value

I have a input file from which I want to randomly choose a line. In a column in every line is a value, and I want only a number of lines up to a maximum count of this value to be choosen randomly, not ...
2
votes
1answer
96 views

parsing several fields of a csv with gawk on solaris

I have a generated csv that has the following format: unixtime;host_name;ip_adress;description;2;0;1 I have been trying to parse this csv to do the following: change field 1 from unix timestamp to ...
2
votes
1answer
1k views

Extract hostnames from non-hashed ssh known_hosts

I would like to extract hostnames/ips from .ssh/known_hosts file in all (known) format conditions: Hostname only hostname pubkey Hostname with ip: hostname,1.1.1.1 pubkey Hostname with ip that is not ...
2
votes
1answer
879 views

Substitute one field with other if its entry starts with specific letter

If I have a file something like: 223 898 teeth 223 899 mouth 223 900 r9 445 800 round My question is how can I use awk command to substitute column 2 with column 3, only for the row in which column ...
2
votes
4answers
2k views

Passing arguments to awk script

I have a awk script where i want to be able to pass N arguments into it and also read from stdin. I would like to be able to do something like tail -f logfile | my_cool_awk_scipt var1 var2 var3 ... ...
2
votes
2answers
917 views

Escaping gawk variables for shell commands

I'm trying to run a small gawk script which will run certain shell commands using system(). I want to get the file list with find and run the script like that: find <arguments> -printf "%f\n" | ...
2
votes
1answer
383 views

Is there any other method to uncomment in awk?

This command works perfect for me to uncomment a line. awk -i inplace '/# MyLINE/ { sub ("^ *#","") } { print }' file Just for curiosity, I am wondering if there are any other methods for "awk" to ...
2
votes
1answer
1k views

Adding two columns together in CSV and outputting to new CSV file

I have a problem I have been trying to figure out: We have a stock file CSV, which contains the stock in multiple locations. The csv looks like this: stock_no,primary,secondary,tertiary,cstock,...
2
votes
2answers
100 views

Merge multiple columns based on the first column values

I have multiple files (10+) that I want to merge/join into the one output file, for example: file 1 2000 0.0202094 2001 0.0225532 2002 0.02553 2003 0.0261099 2004 0.0280311 2005 0.028843 file 2 ...
2
votes
1answer
902 views

gawk with gawkextlib

In my system (Slackware64-14.1) gawk is installed in version 4.1.0. I'd like to install gawkextlib to have access to some of its features, namely xml. From the README file of gawkextlib-0.4.9: To ...
1
vote
5answers
427 views

Merge a column based on 2nd column

I have file looking like this: pw1jc5ssyt6hx618,254343 ysezaratlycpuggl,254333 pht92h4adr3mrbz3,254343 hguvgstqxu3gowfg,254344 gqjp2rsjmk1a2v9c,254333 twdzyi2ddbnrfknd,254333 gcmj7krrx5x6nf8r,254341 ...
1
vote
4answers
403 views

How shall I perform multiline matching and substitution using awk?

In a text file, ignoring any trailing whitespace at the end of each line, I assume that if a line is not ended with a digit, then there is a line break between the line and the next line, and I would ...
1
vote
1answer
2k views

How to use grep/awk/unix to match all lines from one file in another file, even if they are duplicates

I'm trying to retrieve the the matching lines from a file, however the file that has the "index" or list of matches to retrieve also has duplicate entries and I want to print out the duplicates as ...
1
vote
3answers
34 views

In a line containing multiple fields, create a new line for each field, and remove that field from the old line

I have a file that looks like this: elephant kangaroo, snake, zebra baboon parrot, eagle, owl ... ... I want to change this list so that each line has only one element in it elephant kangaroo snake ...
1
vote
1answer
2k views

“^ backslash not last character on line” in gawk

I would like to match a number between / and , in each line, and increase it by 3. For example The Ubiquitous Backslash/49,Black becomes The Ubiquitous Backslash/52,Black My gawk command is: ...
1
vote
1answer
461 views

How to EDIT only the last line (or any specific line number(s)) using awk command?

I have a big multi columns file with #n number of records (lines). How I can only apply awk command in the last record or any specific records number (i.g. NR==4) or a range of lines (NR==[2-5])? ...
1
vote
3answers
647 views

Parse file with Awk only when first row matches pattern

I need to interrogate the header of a CSV file, and if a column exists proceed with the data rows. Context is when the data contains columns depending on when and what emitted it. Hoping for a "pure" ...
1
vote
2answers
229 views

awk system calls to linux - using extended regex when calling the shell from awk

It is useful to be able to call system commands from awk. However, if you try to use shell extended regex, you'll find it doesn't work. That is because awk calls /bin/sh instead of /bin/bash as you'd ...
1
vote
4answers
789 views

How to get value of the last day in the current month in awk

According to link last day can be seen using this date command: $ date -d "-$(date +%d) days -1 month" Wed May 31 21:20:31 CEST 2017 My question is, how can I get only the day and assign it to a ...
1
vote
2answers
465 views

Adding Contents of multiple files using awk

Say I have some files named file1, file2, file3, ... that have the following format file1 blah blah blah [PATTERN0] a10 a20 a30 [PATTERN1] a11 a21 a31 [PATTERN3] a13 a13 a33 file 2 blah blah blah [...
1
vote
1answer
682 views

split file into multiple pieces

Problem: given a file samplein, it can be split up into multiple pieces as follows: $ cat samplein START Unix Linux START Solaris Aix SCO $ awk '/START/{x="F"++i;}{print > x}' samplein $ ls F* ...
1
vote
4answers
3k views

Add line to configuration file from a bash script?

I am trying to figure out to locate a line in a configuration file, then to drop down two lines then to insert a line of code. I was attempting to do this in awk/sed but got stuck on the carriage ...
1
vote
1answer
348 views

What does vRS=“\n20” mean on the gawk command line?

Can any one please explain the below command: gawk -vRS="\n20" '/Job-157625/' log.1 We are using this command to get the log for the job. I know gawk is used to find/scan a pattern but I want to ...
1
vote
2answers
43 views

Is there any way to use the tr command in place?

I want to remove all non-english alphanumerals from a file. tr -sc '[:alnum:][:punct:]' ' ' <file gawk and sed have the -i flag but I cannot find anything similar in the tr man pages. How can I ...
1
vote
4answers
121 views

How to convert Syslog Date-Time [UTC] to Epoch?

I am currently piecing together a tool to work with Syslogs generated in my network, one of the requirements is to convert the DateTime from the format in which it is in syslog (%b %d %Y %T) to epoch. ...
1
vote
1answer
31 views

AWK if loop - If any value is less than one

I produce a huge multi-column output file running some analytical software on angle populations. Sadly, the angles are defines as -180° to 180° and I need 0°-360°. What I want do to is take this ...
1
vote
2answers
106 views

Shell-Script Bash

I have this loop in the shell script, but it seems I don't fully understand what it does: especially gawk -v for ((k=$1; $k<3 + 3 ; k=$k + $3)) do echo $k | gawk -v s=4 '{print $1*s}' done ...
1
vote
1answer
179 views

Unexpected split behavior

I read about Awk split behavior here: [...] the fs argument to the split function (see String Functions) shall be interpreted as extended regular expressions. These can be either ERE tokens or ...
1
vote
1answer
107 views

Concerning AWK programming, what do the -f switch and ~ operator do? [closed]

I can't seem to find anything on these switches. I know the "-F" option can be used to change the delimiter but that's the extent of what I've found other than it looks like maybe a compiling option; ...
1
vote
1answer
65 views

searching multiple files for a line with bigger number in column 3 of matched lines

I have multiple files with contents similar to: main file1: test01:6733:4370:5342 test02:7776:2018:1001 test03:9865:5632:1429 test04:8477:4757:1890 test05:8019:8860:5298 test06:5602:3100:6995 test07:...
1
vote
1answer
447 views

awk print with a condition expression with a ternary operator

Sample input: 1:2:3:0:5 1:2:3::5 Awk processing: $ echo -e "1:2:3:0:5\n1:2:3::5"|awk -F: '{if($4!="") print NR,$4; else print NR,"-"}' 1 0 2 - Question: can above be run in a short if notation? I ...
1
vote
4answers
239 views

Reverse the order of first 3 digits in a line

I am trying to do a assignment for university, but I am currently stuck. The goal is to read some phone numbers and reverse the order of the first 3 digits and put them in brackets. I can get it to ...
1
vote
3answers
281 views

Awk to remove extra space after first letter?

Awk to remove extra space after first letter? Our file contains: Blue sky. Nice weather. White cloud. Bright sun. Cool air. Bla bla bla. How to have ...
1
vote
2answers
440 views

Creating and exporting/importing variable with awk?

I have a file where every line is a new customers information, such as Pop corn, pop@gmail.com, 50, 200 I am currently trying to see if it is possible to use a awk/gawk script to assign each column ...