Questions tagged [awk]
A pattern-directed scanning and processing language.
7,888
questions
-1
votes
1
answer
28
views
If the row delimiter is %=$% (there is no \n char) and column delimiter is |#@| how do I find rows with different number of column delimiters in Unix?
I have a delimited file called abc.txt. It has a header row with column names separated by comma (,). The row delimiter used is %=$% (there is no new line character to denote a new line) for all the ...
- 7
-3
votes
1
answer
35
views
How can find special row and replace special strings
I wrote a todo list script with "add","clear" and "find" argument. I need to pass a "done" argument and script find related row and replace its first column ...
0
votes
3
answers
41
views
Linux bash awk print word with special letter
I have a URL:
www.google.com/word/word1/word_2/word3/word4
I would like to use awk to print only the words that contain the _ special character. Sometimes the folder hierarchy may change:
www.google....
-1
votes
1
answer
60
views
Zero pad file names using find and execdir flag
I'm using smplayer and it doesn't sort files by natural order, e.g after playing 1*.sth it will jump to 11*.sth instead of 2*.sth. but as I've checked if I change file names to zero pad them then this ...
- 107
2
votes
3
answers
176
views
Join strings on different lines depending on indentation with awk
I have this multiline string:
foo
foobar
bar
baz
bat
bar
and I'd like to end up with:
foo
foobar.bar
foobar.baz
foobar.baz.bat
bar
My thinking would be that for each line, I'd have to look ...
- 178
0
votes
2
answers
69
views
Add list of words at the end of the 1st line in a loop
I have a list with the names of several files:
file1
file2
I would like to add at the end of the 1st line of each file the name of the file e.g.:
file 1 first row will be
> ATCGCCfile1
file 2 ...
- 684
0
votes
2
answers
59
views
Extracting text preceding a particular string
I'm trying to grep a word per line. For each line, I'd like to print everything that appears before the grep'd word.
Example:
echo help.me.nc.example.com is an alias for fd-nc-001.domain.com.
I'm ...
- 13
2
votes
3
answers
234
views
How to use awk's gensub or alternatives to replace overlapping matches
I am using:
$ echo ".a.b.c." | awk '
{
t=gensub(/.([a-z])./,"[\\1]","g",$1); print t
}'
[a]b[c]
I would like to end up with [a][b][c]. How can I get gensub to work ...
- 178
0
votes
1
answer
54
views
How to escape characters inside variables in awk?
I'm trying to make a script that uses awk to get the recent file names from .config/okularrc and pipe them into dmenu, and back to awk to get the path for the name selected.
However, I'm getting an ...
- 62
0
votes
4
answers
60
views
How to remove commas from a certain field in CSV?
There is a field/column in a CSV file Im working with called name, this is always the second field.
The values in this column are like: "Smith, John" "Brady, Tom" "Manning, ...
- 1
0
votes
3
answers
49
views
Remove hostnames from URL with sed/awk
I got lists of urls with different domains and I want the hostname removed with sed, awk or something similar and only keep the path. There are no urls with port or username@password in it.
input:
...
- 69
0
votes
3
answers
81
views
How can I test nawk on Ubuntu?
I am writing a script which needs to be portable across Apple MacOs and Ubuntu. On the former 'awk' is (I believe) provided by nawk, while on the latter, gawk. There are significant differences ...
- 4,744
-1
votes
3
answers
109
views
How to replace the contents of out.txt with in.txt in a third file?
I have three files, main.txt, out.txt and in.txt. I want to replace every occurrence of the contents of out.txt in main.txt with the contents of in.txt.
Both out.txt and in.txt may contain multiple ...
- 15
1
vote
2
answers
121
views
Combine rows based on shared elements (bash/awk)
I have a following file:
a3 v2c
v5 a7
a9 v2c
v1c a3 a7c
Desired output (without duplicates in each row):
a3 a7c a9 v1c v2c
a7 v5
What I want is to combine the rows sharing at least one element.
In ...
- 13
1
vote
3
answers
42
views
Awk: add modified column to file and keep original one
I have a file with two columns:
path/path/target1.target2 column2
Using awk I want to print:
column1 column2 target1 target2
I tried the following to do the modifications but then I lose the ...
-2
votes
2
answers
52
views
How to group matching data from first two column
I would like to group data from first two fields as per below
Sample Data:
CHECK|checkadm|block1
CHECK|checkadm|block2
CHECK|checkadm|block3
CHECK|checkadm|block4
TEST|testadm|block1
TEST|testadm|...
- 3
2
votes
2
answers
232
views
Why does awk concatenate strings from different lines
Good day.
I'm trying to run a ping6 on IPv6 addresses pulled from /etc/resolv.conf.dnsph. It seems though, as if awk is stringing the IPv6 addresses onto one line.
$ grep ^nameserver /etc/resolv.conf....
- 29
0
votes
1
answer
48
views
is there any way to get the unique word count from a csv file
I have a text file content as below
Notes1,Notes2,Id3,Id4
I'd like to play tennis with you some day with everyone,Mary enjoys cooking,id1234,5678
Some of my friends can speak English well and turkish ...
- 294
-2
votes
1
answer
22
views
How to delete the whole row when it starts with 'H' in a comma separarted csv file
How to delete the whole row when it starts with 'H' in a comma separated csv file for below example,
H,NYR,257,10,11,000, TESTING,055,000,2023-05-10
D,NYR,257,10,11,000, TESTING,055,000,2023-05-10
H,...
- 1
0
votes
3
answers
81
views
Need assistance with awk/sed to identify/mark duplicate IP addresses
Good day.
I have a text file which contains pod/node names and associated IPv6 addresses of which two pods have the same IP address, first pod k8-worker0001c-cif-9d86d6dd4-vf9b9 and last pod k8-...
- 29
0
votes
3
answers
59
views
bash awk how to exclude entries which contain the word 'Mouse'
The following bash command collects the xinput id's of all devices with a name that includes ILITEK ILITEK-TP.
name='ILITEK ILITEK-TP'
ids=$(xinput --list |awk -Fid= "/$name/"'{print $2+0}')
...
- 263
2
votes
2
answers
80
views
How can I select only successful values from leakcheck output?
I have these results that came from leakcheck. How do I properly exclude "false" results and still include the email using bash
test1@email.com
{"success":false,"error":&...
3
votes
6
answers
227
views
How to reverse lines inside repeated blocks in a text file (using sed/awk?)
I have a file like this:
x = {
y = {
z = {
block = {
line1
line2
line3
}
}
}
}
x2 = {
y2 = {
block = ...
- 41
-2
votes
2
answers
64
views
How to find the common occurring values in a column in all rows [closed]
I have a list of images drawn from pdfimages. Each page has many images. I have a csv file of this data
Every page as 3 images in common. Each of 11.0K, 983B, 1613B. I want a bash script to identify ...
-1
votes
3
answers
83
views
Joining multiple lines based on a regex
I have the output of a pandoc conversion to HTML which looks like this:
foo
bar
<blockquote>
That's one small step for man, one giant leap for mankind
A new line and another quote
</...
- 101
0
votes
1
answer
30
views
Can i skip a pattern and also print that pattern along with line by line file comparison using awk?
I have the code below (please see Need to understand below awk command to find missing lines in a file):
awk 'NR==FNR{a[$0];next}(!($0 in a)){print}' 1.txt 2.txt
Can I add one more condition to skip ...
- 23
3
votes
3
answers
335
views
Find files matching one pattern, but not another, using grep, awk or sed [duplicate]
I need to locate .php and .pl files that do not contain one string (e.g. aaa), but do contain another (e.g. bbb).
I'm currently using this command:
find /path/ \( -iname '*.php*' -or -name '*.pl*' \) -...
- 2,460
-2
votes
1
answer
62
views
Modifying values in a CSV column using awk, issue with using a variable
I have a CSV file composed of several columns of numeric values. I want to replace the 3rd column with abs($3-1), i.e., the absolute value of the number in the third column reduced by 1.
I tried using ...
0
votes
1
answer
42
views
Awk command not providing output with partial matching in columns 1 and 2 [duplicate]
Please assist with the following script, which is matching variable and providing output. The script is working with exact match in columns 1 and 2, but it is not providing an output when there is a ...
- 21
1
vote
1
answer
38
views
What is the meaning of this pattern arg="${arg//\\/\\\\}"
Please explain the use of pattern
arg="${arg//\\/\\\\}"
Does it mean exact match?
- 21
0
votes
2
answers
115
views
How to modify an awk command to search and match values in multiple columns of a file?
I have the below script saved in bin, which is using to get output based on a defined variable.
Currently, this script is running for column 1 and providing output.
How can we update this script so it ...
- 21
4
votes
4
answers
572
views
grep multiple patterns with differing number of matching groups?
I'm trying to tally insertions and deletions from git diffs.
I have the following, which when piped one or more strings of the form "4 files changed, 629607 insertions(+), 123 deletions(-)" ...
- 143
0
votes
2
answers
53
views
Perform search for text formatting on last row of text using awk or sed
I have output with many rows. The number of rows on the output is inconsistent. Is there a way that the last row can be selected so that a character may be found and replaced? If not using awk or sed, ...
- 35
0
votes
4
answers
79
views
Removing trailing newline from grep output
I'm looking for way to remove new line from grep output, I use
grep -oP "changed: \[([^]]+)\] =>|'path': '([^']*)'" /path/to/file
The output is like:
changed: [hostname1] =>
'path': '/...
- 1
-1
votes
3
answers
84
views
How to include the file name on the output of this command?
How to include the file name on the output of this command?
tail -n 1 *.txt |awk -F "|" '{print $2}'
-2
votes
1
answer
66
views
Rule with starting { on next line
I am doing the following, which is repeating MATCH $0 BEG_ERE for every line.
Primarily because the starting { starts on the next line. Would this be a bug?
match($0, beg_ere, maggr)
{
print "...
- 123
2
votes
3
answers
187
views
Replace the last half in every line of a file with corresponding part in another file
I have two files A and B. Every line in both files is considered an item. The format of every item is fixed, consisting of a key and description, separated by a space. as shown in the example below.
...
- 21
0
votes
1
answer
79
views
Setting values through awk function calls
I am using GNU Awk 5.1.0, API: 3.0 and have the following awk script. Testing this has shown that one can never use a function argument to change the variable used as input to it. Thus in the ...
- 123
0
votes
2
answers
73
views
awk, multiply column value, compare to a constant and find first match before exiting
I have a data2process.tempdata file with data in six columns as follows.
# Angle(deg) D1 D2 D3 D4 D5
0.50 0.00E+00 0.00E+00 1.97E-10 3.94E-09 1.97E-10
...
- 33
0
votes
3
answers
157
views
Awk function arguments
I am having trouble understanding the use of gawk function arguments.
Customarily, arguments define variables that are local to the function.
But can arguments be used as input that the function can ...
- 123
0
votes
1
answer
62
views
Bash script to calculate within a CSV with or without zero
I have a script here that I am working with.
I would now like to expand this.
POSIXLY_CORRECT=1 awk -F ';' -v OFS=';' '
{print $0, NR == 1 ? "Price" : $1 ? $1 : $3 * 1.2}' < file.csv
...
- 11
3
votes
1
answer
142
views
duplicated entries of an array in awk
I have a file with 4 columns. When I put these 4 columns into an array using NR as the index, the entries are duplicated somehow. See below for an elaboration of the issue.
The first 5 lines of the ...
- 45
2
votes
1
answer
45
views
Does awk always discard trailing characters when parsing a number?
With awk I need to parse percentage numbers (with a trailing `%´) and check if they exceed a certain percentage.
Normal parsing simply ignores any trailing characters like:
% echo "99%" | ...
- 183
2
votes
1
answer
96
views
gawk is treating NF and custom variables differently even though they have the same value
I have the below awk script where I'm using the length of the array passed, as a variable.
BEGIN {
FPAT = "([^,]*)|(\"[^\"]*\")"
len=split(_arr,arr," ")
}
{
...
0
votes
3
answers
72
views
grep from a specific value to specific value
For example, I have a file with data:
[2023-03-21 12:27:19 +0000] some_data
[2023-03-21 12:27:19 +0000] some_data
[2023-03-22 12:27:19 +0000] some_data
[2023-03-23 12:27:19 +0000] some_data
[2023-03-...
- 41
1
vote
1
answer
53
views
index of an array is not recognized in awk
I have two tab separated files, each with two columns. I want to create a file which contains overlapping elements by column 1 of the two files. To do so, I put file 1 in an array first then scanned ...
- 45
-5
votes
2
answers
109
views
How to negate pattern when using awk with double quotes [closed]
I have the following file:
$ cat files
foo.txt
bar.js
I am looking to use bash awk to only print files not ending with .js.
This works fine when using single quoted awk:
$ awk '$1 !~ /.js$/ {print $1}...
- 178
3
votes
2
answers
586
views
How do you get the individual outputs of grep?
I've seen grep thrown around in plenty of answers but never wrapped my head around it.
Now that I am playing with trying to get the content within HTML tags of a locally saved page from the internet, ...
- 73
3
votes
4
answers
432
views
awk or perl to eval mathematical expressions in each line
I want a script to make awk to become an interactive mathematical calculator, to eval mathematical expressions given in each line.
I.e., instead of constructing awk commands to calculate expressions ...
- 1,093
0
votes
0
answers
44
views
Merging two files based on matching columns with awk
I have two text files with the following structures:
File 1:
File/Path/DataXXX File/Path/ImageXXX
File 2:
1 File/Path/ImageXXX DataGroupXXX
In file 2 the first column in every row has been replaced ...
