As of May 31, 2023, we have updated our Code of Conduct.

Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
-1 votes
1 answer
28 views

If the row delimiter is %=$% (there is no \n char) and column delimiter is |#@| how do I find rows with different number of column delimiters in Unix?

I have a delimited file called abc.txt. It has a header row with column names separated by comma (,). The row delimiter used is %=$% (there is no new line character to denote a new line) for all the ...
Iman's user avatar
  • 7
-3 votes
1 answer
35 views

How can find special row and replace special strings

I wrote a todo list script with "add","clear" and "find" argument. I need to pass a "done" argument and script find related row and replace its first column ...
Danial Rostami's user avatar
0 votes
3 answers
41 views

Linux bash awk print word with special letter

I have a URL: www.google.com/word/word1/word_2/word3/word4 I would like to use awk to print only the words that contain the _ special character. Sometimes the folder hierarchy may change: www.google....
user129387's user avatar
-1 votes
1 answer
60 views

Zero pad file names using find and execdir flag

I'm using smplayer and it doesn't sort files by natural order, e.g after playing 1*.sth it will jump to 11*.sth instead of 2*.sth. but as I've checked if I change file names to zero pad them then this ...
mh-firouzjah's user avatar
2 votes
3 answers
176 views

Join strings on different lines depending on indentation with awk

I have this multiline string: foo foobar bar baz bat bar and I'd like to end up with: foo foobar.bar foobar.baz foobar.baz.bat bar My thinking would be that for each line, I'd have to look ...
Foo's user avatar
  • 178
0 votes
2 answers
69 views

Add list of words at the end of the 1st line in a loop

I have a list with the names of several files: file1 file2 I would like to add at the end of the 1st line of each file the name of the file e.g.: file 1 first row will be > ATCGCCfile1 file 2 ...
fusion.slope's user avatar
0 votes
2 answers
59 views

Extracting text preceding a particular string

I'm trying to grep a word per line. For each line, I'd like to print everything that appears before the grep'd word. Example: echo help.me.nc.example.com is an alias for fd-nc-001.domain.com. I'm ...
dupa's user avatar
  • 13
2 votes
3 answers
234 views

How to use awk's gensub or alternatives to replace overlapping matches

I am using: $ echo ".a.b.c." | awk ' { t=gensub(/.([a-z])./,"[\\1]","g",$1); print t }' [a]b[c] I would like to end up with [a][b][c]. How can I get gensub to work ...
Foo's user avatar
  • 178
0 votes
1 answer
54 views

How to escape characters inside variables in awk?

I'm trying to make a script that uses awk to get the recent file names from .config/okularrc and pipe them into dmenu, and back to awk to get the path for the name selected. However, I'm getting an ...
white-hat-er's user avatar
0 votes
4 answers
60 views

How to remove commas from a certain field in CSV?

There is a field/column in a CSV file Im working with called name, this is always the second field. The values in this column are like: "Smith, John" "Brady, Tom" "Manning, ...
ababa's user avatar
  • 1
0 votes
3 answers
49 views

Remove hostnames from URL with sed/awk

I got lists of urls with different domains and I want the hostname removed with sed, awk or something similar and only keep the path. There are no urls with port or username@password in it. input: ...
pwrsheller's user avatar
0 votes
3 answers
81 views

How can I test nawk on Ubuntu?

I am writing a script which needs to be portable across Apple MacOs and Ubuntu. On the former 'awk' is (I believe) provided by nawk, while on the latter, gawk. There are significant differences ...
symcbean's user avatar
  • 4,744
-1 votes
3 answers
109 views

How to replace the contents of out.txt with in.txt in a third file?

I have three files, main.txt, out.txt and in.txt. I want to replace every occurrence of the contents of out.txt in main.txt with the contents of in.txt. Both out.txt and in.txt may contain multiple ...
Roger Bosman's user avatar
1 vote
2 answers
121 views

Combine rows based on shared elements (bash/awk)

I have a following file: a3 v2c v5 a7 a9 v2c v1c a3 a7c Desired output (without duplicates in each row): a3 a7c a9 v1c v2c a7 v5 What I want is to combine the rows sharing at least one element. In ...
Tokivideo's user avatar
1 vote
3 answers
42 views

Awk: add modified column to file and keep original one

I have a file with two columns: path/path/target1.target2 column2 Using awk I want to print: column1 column2 target1 target2 I tried the following to do the modifications but then I lose the ...
Pau Clavell-Revelles's user avatar
-2 votes
2 answers
52 views

How to group matching data from first two column

I would like to group data from first two fields as per below Sample Data: CHECK|checkadm|block1 CHECK|checkadm|block2 CHECK|checkadm|block3 CHECK|checkadm|block4 TEST|testadm|block1 TEST|testadm|...
Rocky's user avatar
  • 3
2 votes
2 answers
232 views

Why does awk concatenate strings from different lines

Good day. I'm trying to run a ping6 on IPv6 addresses pulled from /etc/resolv.conf.dnsph. It seems though, as if awk is stringing the IPv6 addresses onto one line. $ grep ^nameserver /etc/resolv.conf....
Bjoern's user avatar
  • 29
0 votes
1 answer
48 views

is there any way to get the unique word count from a csv file

I have a text file content as below Notes1,Notes2,Id3,Id4 I'd like to play tennis with you some day with everyone,Mary enjoys cooking,id1234,5678 Some of my friends can speak English well and turkish ...
Arya's user avatar
  • 294
-2 votes
1 answer
22 views

How to delete the whole row when it starts with 'H' in a comma separarted csv file

How to delete the whole row when it starts with 'H' in a comma separated csv file for below example, H,NYR,257,10,11,000, TESTING,055,000,2023-05-10 D,NYR,257,10,11,000, TESTING,055,000,2023-05-10 H,...
guest's user avatar
  • 1
0 votes
3 answers
81 views

Need assistance with awk/sed to identify/mark duplicate IP addresses

Good day. I have a text file which contains pod/node names and associated IPv6 addresses of which two pods have the same IP address, first pod k8-worker0001c-cif-9d86d6dd4-vf9b9 and last pod k8-...
Bjoern's user avatar
  • 29
0 votes
3 answers
59 views

bash awk how to exclude entries which contain the word 'Mouse'

The following bash command collects the xinput id's of all devices with a name that includes ILITEK ILITEK-TP. name='ILITEK ILITEK-TP' ids=$(xinput --list |awk -Fid= "/$name/"'{print $2+0}') ...
Cheetaiean's user avatar
2 votes
2 answers
80 views

How can I select only successful values from leakcheck output?

I have these results that came from leakcheck. How do I properly exclude "false" results and still include the email using bash test1@email.com {"success":false,"error":&...
noobwithskillissue's user avatar
3 votes
6 answers
227 views

How to reverse lines inside repeated blocks in a text file (using sed/awk?)

I have a file like this: x = { y = { z = { block = { line1 line2 line3 } } } } x2 = { y2 = { block = ...
farraget's user avatar
-2 votes
2 answers
64 views

How to find the common occurring values in a column in all rows [closed]

I have a list of images drawn from pdfimages. Each page has many images. I have a csv file of this data Every page as 3 images in common. Each of 11.0K, 983B, 1613B. I want a bash script to identify ...
GDLC DEMO's user avatar
-1 votes
3 answers
83 views

Joining multiple lines based on a regex

I have the output of a pandoc conversion to HTML which looks like this: foo bar <blockquote> That's one small step for man, one giant leap for mankind A new line and another quote </...
Josh Friedlander's user avatar
0 votes
1 answer
30 views

Can i skip a pattern and also print that pattern along with line by line file comparison using awk?

I have the code below (please see Need to understand below awk command to find missing lines in a file): awk 'NR==FNR{a[$0];next}(!($0 in a)){print}' 1.txt 2.txt Can I add one more condition to skip ...
dbadmin's user avatar
  • 23
3 votes
3 answers
335 views

Find files matching one pattern, but not another, using grep, awk or sed [duplicate]

I need to locate .php and .pl files that do not contain one string (e.g. aaa), but do contain another (e.g. bbb). I'm currently using this command: find /path/ \( -iname '*.php*' -or -name '*.pl*' \) -...
Krackout's user avatar
  • 2,460
-2 votes
1 answer
62 views

Modifying values in a CSV column using awk, issue with using a variable

I have a CSV file composed of several columns of numeric values. I want to replace the 3rd column with abs($3-1), i.e., the absolute value of the number in the third column reduced by 1. I tried using ...
SchroedingersLion's user avatar
0 votes
1 answer
42 views

Awk command not providing output with partial matching in columns 1 and 2 [duplicate]

Please assist with the following script, which is matching variable and providing output. The script is working with exact match in columns 1 and 2, but it is not providing an output when there is a ...
anukalps's user avatar
1 vote
1 answer
38 views

What is the meaning of this pattern arg="${arg//\\/\\\\}"

Please explain the use of pattern arg="${arg//\\/\\\\}" Does it mean exact match?
anukalps's user avatar
0 votes
2 answers
115 views

How to modify an awk command to search and match values in multiple columns of a file?

I have the below script saved in bin, which is using to get output based on a defined variable. Currently, this script is running for column 1 and providing output. How can we update this script so it ...
anukalps's user avatar
4 votes
4 answers
572 views

grep multiple patterns with differing number of matching groups?

I'm trying to tally insertions and deletions from git diffs. I have the following, which when piped one or more strings of the form "4 files changed, 629607 insertions(+), 123 deletions(-)" ...
jawns317's user avatar
  • 143
0 votes
2 answers
53 views

Perform search for text formatting on last row of text using awk or sed

I have output with many rows. The number of rows on the output is inconsistent. Is there a way that the last row can be selected so that a character may be found and replaced? If not using awk or sed, ...
XJMZX's user avatar
  • 35
0 votes
4 answers
79 views

Removing trailing newline from grep output

I'm looking for way to remove new line from grep output, I use grep -oP "changed: \[([^]]+)\] =>|'path': '([^']*)'" /path/to/file The output is like: changed: [hostname1] => 'path': '/...
shlco's user avatar
  • 1
-1 votes
3 answers
84 views

How to include the file name on the output of this command?

How to include the file name on the output of this command? tail -n 1 *.txt |awk -F "|" '{print $2}'
SystemNewbieDeveloper's user avatar
-2 votes
1 answer
66 views

Rule with starting { on next line

I am doing the following, which is repeating MATCH $0 BEG_ERE for every line. Primarily because the starting { starts on the next line. Would this be a bug? match($0, beg_ere, maggr) { print "...
Heime's user avatar
  • 123
2 votes
3 answers
187 views

Replace the last half in every line of a file with corresponding part in another file

I have two files A and B. Every line in both files is considered an item. The format of every item is fixed, consisting of a key and description, separated by a space. as shown in the example below. ...
echo's user avatar
  • 21
0 votes
1 answer
79 views

Setting values through awk function calls

I am using GNU Awk 5.1.0, API: 3.0 and have the following awk script. Testing this has shown that one can never use a function argument to change the variable used as input to it. Thus in the ...
Heime's user avatar
  • 123
0 votes
2 answers
73 views

awk, multiply column value, compare to a constant and find first match before exiting

I have a data2process.tempdata file with data in six columns as follows. # Angle(deg) D1 D2 D3 D4 D5 0.50 0.00E+00 0.00E+00 1.97E-10 3.94E-09 1.97E-10 ...
csnl's user avatar
  • 33
0 votes
3 answers
157 views

Awk function arguments

I am having trouble understanding the use of gawk function arguments. Customarily, arguments define variables that are local to the function. But can arguments be used as input that the function can ...
Heime's user avatar
  • 123
0 votes
1 answer
62 views

Bash script to calculate within a CSV with or without zero

I have a script here that I am working with. I would now like to expand this. POSIXLY_CORRECT=1 awk -F ';' -v OFS=';' ' {print $0, NR == 1 ? "Price" : $1 ? $1 : $3 * 1.2}' < file.csv ...
dima1002's user avatar
3 votes
1 answer
142 views

duplicated entries of an array in awk

I have a file with 4 columns. When I put these 4 columns into an array using NR as the index, the entries are duplicated somehow. See below for an elaboration of the issue. The first 5 lines of the ...
Xuan's user avatar
  • 45
2 votes
1 answer
45 views

Does awk always discard trailing characters when parsing a number?

With awk I need to parse percentage numbers (with a trailing `%´) and check if they exceed a certain percentage. Normal parsing simply ignores any trailing characters like: % echo "99%" | ...
Amaterasu's user avatar
  • 183
2 votes
1 answer
96 views

gawk is treating NF and custom variables differently even though they have the same value

I have the below awk script where I'm using the length of the array passed, as a variable. BEGIN { FPAT = "([^,]*)|(\"[^\"]*\")" len=split(_arr,arr," ") } { ...
Parikshith Raju's user avatar
0 votes
3 answers
72 views

grep from a specific value to specific value

For example, I have a file with data: [2023-03-21 12:27:19 +0000] some_data [2023-03-21 12:27:19 +0000] some_data [2023-03-22 12:27:19 +0000] some_data [2023-03-23 12:27:19 +0000] some_data [2023-03-...
Mexanizm456's user avatar
1 vote
1 answer
53 views

index of an array is not recognized in awk

I have two tab separated files, each with two columns. I want to create a file which contains overlapping elements by column 1 of the two files. To do so, I put file 1 in an array first then scanned ...
Xuan's user avatar
  • 45
-5 votes
2 answers
109 views

How to negate pattern when using awk with double quotes [closed]

I have the following file: $ cat files foo.txt bar.js I am looking to use bash awk to only print files not ending with .js. This works fine when using single quoted awk: $ awk '$1 !~ /.js$/ {print $1}...
Foo's user avatar
  • 178
3 votes
2 answers
586 views

How do you get the individual outputs of grep?

I've seen grep thrown around in plenty of answers but never wrapped my head around it. Now that I am playing with trying to get the content within HTML tags of a locally saved page from the internet, ...
Tim50001's user avatar
3 votes
4 answers
432 views

awk or perl to eval mathematical expressions in each line

I want a script to make awk to become an interactive mathematical calculator, to eval mathematical expressions given in each line. I.e., instead of constructing awk commands to calculate expressions ...
xpt's user avatar
  • 1,093
0 votes
0 answers
44 views

Merging two files based on matching columns with awk

I have two text files with the following structures: File 1: File/Path/DataXXX File/Path/ImageXXX File 2: 1 File/Path/ImageXXX DataGroupXXX In file 2 the first column in every row has been replaced ...
BAROTRAUMA's user avatar

1
2 3 4 5
158