Questions tagged [awk]
A pattern-directed scanning and processing language.
7,818
questions
0
votes
0
answers
11
views
What is best for multiple word replacement in a large text file, sed or awk?
I have a list of words to be replaced in a large text file. I wanted to know which command sed/awk would be best to do the operation. Also does sed/awk read text file line by line or loads the entire ...
- 1
0
votes
0
answers
47
views
Understanding If and Else If (awk) [closed]
I can't seem to get if and else if statements functioning correctly in awk.
'a' and 'b' will be a 1 or a 0. What I expect is that "ERROR" will never be printed, however when 'a' and 'b' are ...
- 53
0
votes
0
answers
31
views
Using AWK replace command - how to consider space separated words as one word?
Background: Using below awk command which check pipe separated first field - wherever value 10 replaced with "No Files Available" from test.txt files.
Question: It count "No Files ...
- 5
3
votes
3
answers
38
views
Reading a file with two columns and use them to change filenames
I have a text file with two columns, one representing the current file name and another representing the name I want as the final name of the file. The text file is like this:
JALAZR010000001.1 - ...
5
votes
5
answers
237
views
How can I replace multiple substrings from multiple lines in a file matching a pattern from a different file?
I have a file that contains multiple IP addresses and hostnames, and another file that contains some folders with many IP addresses per line.
ip_hostname.txt
host1 10.1.1.1
host2 10.2.2.2
host3 10.3.3....
- 51
0
votes
1
answer
35
views
Awk replace string where specific condition is true using variables
I am trying to replace a string where a specific condition is true in the same line using awk variables.
I found something that could help me a lot but I'm facing a new challenge cause a I need to do ...
-1
votes
1
answer
58
views
Separating CSV (one column) into many columns on delimiter (comma)
I have a CSV with ~50 comma-separated values in one column that I want to separate into separate columns. The header is line 1. This should be really simple, and I've tried a lot surrounding awk and ...
-2
votes
3
answers
183
views
How could awk be timezone aware?
If an input is a timestamp, and the timestamp happens to be in a common format (ISO-8601) that awk could be made to accept, then it is possible to print a seconds after epoch.
For example, with input:
...
0
votes
1
answer
35
views
Linux Column with value print
i tried awk for selecting 2 columns but now, the fourth column is somewhere blank and has value at some places, how to filter only the which has values
a,b,c,
d,e,f,g
g,h,i
j,k,l,m
I need only
f,g
l,...
- 31
2
votes
1
answer
65
views
bash + monitor disk space usage and execute script if used space reached the threshold
We want to follow the used space of /var/hadoop/hdfs partition , by way if used space is more the 50% then as results we run the script – do_action.bash , finally this command should be in crontab and ...
- 11.3k
0
votes
1
answer
31
views
Find the highest number after specific string from mutiple files
Each file has similar content:
number 1:
asdfa11sdfsadf
number 2:
asdfa12sdfsadf
number 1:
asdfa1sdfsadf
number 3:
asdfa33sdfsadf
return
3 or number 3: are equally good.
Have tried:
max="$( awk ...
0
votes
1
answer
31
views
Read from external file, skip lines after a match, print and exit
POSIX awk wizards, I need your help! The question may seem trivial to
you at first glance, however, let me describe my intents in greater
detail.
I have been working on a self-contained POSIX awk ...
- 3
3
votes
1
answer
92
views
AWK return path for files with NF <= 2
I think I'm close here but I'm missing something stupid. I'm trying to make awk print out the filenames for files that are missing extended attributes or are missing an attribute value.
So a file will ...
- 75
-2
votes
1
answer
46
views
Executing find command with a file having directory lists
I have a big file which contains directories paths like below:
$PRJ/fp/t/mxl/lf/
$PRJ/fp/t/mxl/lf/
I want to read this file line by line and list out all the files inside each directory file in a ...
- 109
2
votes
2
answers
95
views
Merge muliple files having N common columns by awk and want to replace column value as 0 if any file does not have common key
I want to merge mulpiple files based on common column and want to add 0 when any file does not have that common column. For example see below:
a1.txt
111,222,444,5.5
121,321,555,1.2
a2.txt
111,222,...
-1
votes
0
answers
21
views
how to find patterns in a text using for loop and awk [duplicate]
I have two strings in a text like this in a text file called asl.txt:
sub1_run-1.nii
sub1_run-2.nii
I need to print them using awk according to their run number. how can I do that?
This is what I ...
- 1
-1
votes
0
answers
43
views
awk group an array with same substrings [closed]
I have a text file:
ses-1:
anat
perf
ses-1/anat:
sub-1_ses-1_run-1_T1w.nii
sub-1_ses-1_run-2_T1w.nii
ses-1/perf:
sub-1_ses-1_run-1_asl.nii
sub-1_ses-1_run-1_m0scan.nii
sub-1_ses-1_run-2_asl.nii
sub-...
- 1
0
votes
4
answers
75
views
sed vs. awk: Read a file line by line and append to a specific line in another file
Consider two files below users.txt:
Arnold.Bosch
David.Ledru
Mehri.Sunny
and docker-compose.txt:
version: '3.9'
services:
strongswan-server:
image: vpn-strongswan
container_name: strongswan-...
- 497
1
vote
2
answers
54
views
Awk -v variable assignment
*nix / awk newbie here. I'm wondering what is wrong with my syntax on the simple awk -v example below.
# this line produces the expected result, 'xyz'
echo "abc xyz" | /usr/bin/awk -v ttl=&...
- 13
4
votes
4
answers
483
views
Convert all small words (2-3 characters) to upper case with awk or sed
I need all 2-3 character words completely capitalized. 1 character and 4+ character words need to be remain untouched.
Input:
cat Example
Dog
I
Fish
su
Su adm
Amd Cat ignore
Expected output:
CAT ...
- 123
-6
votes
4
answers
78
views
Remove the Top Level Domain (TLD) and Second Level Domain (SLD) with awk or sed
I have a list of domains without subdomains in a text file. I need the TLD and SLD removed.
Input
google.uk
example.com
amazon.co.uk
domain.ca.uk
education.edu.it
Expected output:
google
example
...
- 123
-3
votes
1
answer
45
views
Extract an ID that matches the user login from a table on shell script
So, I have all these user login information (service, admin, J009878) and this table you see here comes from another command whose output is this table which I save in file.txt. What I want to achieve ...
0
votes
6
answers
91
views
Remove subdomains with awk or sed
How to remove all subdomains in a text file? I think I need to delete everything before and including the '.'?
input:
unix.stackexchange.com
www.example.org
example.example.tld
Expected output:
...
- 123
0
votes
4
answers
50
views
read file: pickup every text between the pattern including start and end pattern and create new file for each pattern set
I have a file called file.txt on Linux, containing:
sqlplus -s insert into table;
commit;
!
sqlplus -s insert into table;
commit;
!
sqlplus -s insert into table;
commit;
!
.
.
...
- 9
2
votes
3
answers
66
views
Replacing math equations by result
What is the best way to replace, in a file, equations found using some predefined regex with the result.
Let's say each equation formula is compatible with bc -l (basic calculator with float numbers ...
- 23
2
votes
1
answer
58
views
awk print variable value
I would like to print the variable value defined outside awk snippet.
Code Snippet
FailCounter=5
awk -v Fail=${FailCounter} -F, ' BEGIN {
print "<TABLE border=1>"
...
-2
votes
4
answers
70
views
Rename headers of several multifasta files in bioinformation job
I have the 1.files like this:
>YP_008856774.1
MHGTRTSAGWSTQPGKFDVLNLRMTFESSSAYQIPDLQPTEFIPTSLAAWNMPRHREYAAVSGGALHFFLDDYRFETVWS
>YP_008856775.1
...
- 43
2
votes
3
answers
160
views
Trying to find complete string values from one file based on another file using AWK
Hello I have 2 files with the first file containing a few values for example
powershell
vectormaps
JuniperSA
and the second file containing values and and ID
appid uid
SplunkforSnort 340
powershell ...
0
votes
2
answers
36
views
Column number of containing a string in column
Related to this question,
Column number of matching string in column
I want to get the column number in which a certain string is contained.
They did not work below,
string=Bo*
echo 'Hi Boo Jiggy' | ...
- 343
0
votes
3
answers
50
views
How can I convert a table column to a row?
I have a tab-separated (\t) file with two columns:
a_1 T_b
a_2 T_c
a_2 T_d
a_2 T_e
a_3 T_f
a_4 B_a
a_4 B_b
I now want to convert that to a file with one sample per row and all values for each ...
- 43
1
vote
3
answers
48
views
insert text five lines before each pattern match
I have a file in which a pattern (eg: RotX) is repeated many times in a similar context.
I need to insert a specific text (eg: Rot-X) at the start of every line which is located five lines before ...
- 11
0
votes
1
answer
17
views
How to filter rows by one column?
Sorry if this is a basic question, I don't use linux often but I have a 13GB file that I want to filter down.
I have 123 columns and I want to remove rows that have only a . in the 75th column.
I've ...
- 3
0
votes
3
answers
49
views
search for a string
How could I search file with below text and only redirect output where there is a "Msg 208" followed on the 3rd line by a "# name not found"?
File Text:
[
DBCC upgrade_object: ...
- 1
0
votes
3
answers
46
views
Understanding awk ' NR%2 ... `
Hi I am using 'Bash command line and shell scripts pocket primer' to study bash and I don't understand one chapter of awk, in which they illustrate printing even lines.
command:
awk ' NR%2 { printf &...
0
votes
3
answers
84
views
lsblk + how to print only the disks that are without filesystem
we want to print only the disks that are without filesystem by command line
from following example we can see that sde disk is without filesystem
lsblk -o NAME,FSTYPE
NAME FSTYPE
sda
├─...
- 11.3k
0
votes
1
answer
35
views
awk judges and outputs all clumns that meet the conditions according to the specific column [duplicate]
Here is my file which contains three cloumns and were separated by "\t", and the second cloumn was sparated by blank.
1 a b c d e Tom
2 a b Jason
...
- 43
2
votes
4
answers
96
views
Combine columns into one column (stack)
I have multi-column data that I wish to reformat such that each column follow "on top" of each other in one single column.
Example input:
1 2 3 4
1 2 3 4
Desired output: (fixed old ...
- 123
0
votes
2
answers
51
views
split file by awk
Here is my file which contains three columns and separate by "\t", and the delimiter of the second column is a blank
1 a b tom
2 a b c sim
3 a mary
...
- 43
1
vote
1
answer
55
views
Replace newlines with spaces until a limit of characters without breaking text
I am configuring an /etc/hosts file on my router, and have a list of over 135,000 domains that I need to block via this file, which takes up nearly 5MB of space. As it turns out, the available RAM on ...
- 13
-2
votes
1
answer
40
views
Bulk replace multiple variables in multiple files at once
Example: I have 40 different variables like $VAR in 100 .txt files which are located in /home/user/EXAMPLE/. How do I replace all variables in the 100 files?
To tell the system what the variables are ...
- 123
2
votes
4
answers
872
views
Grep excluding line that ends in 0, but not 10, 100 etc
I have a file that has this pattern in Linux and Unix environments.
# file1
text1 98432230
text2 123412
text3 10
line4 0
line5 0
line6 40000
... ...
line10 ...
- 23
0
votes
1
answer
32
views
Bulk overwrite .csv files with AWK or sed
I have a directory with many sub-directories which are all full of .csv files (no files with other extensions).
I want to overwrite all .csv files with a sed/AWK command in once. Moving everything to ...
- 123
2
votes
4
answers
47
views
Pulling Red Hat release level from /etc/redhat-relase
I need to script a way to pull the actual release level of Red Hat boxes into a variable.
I am focusing on the /etc/redhat-release file.
I thought it would be a simple awk '{ print $7 }', but RH8 has ...
2
votes
3
answers
156
views
Output in CSV format using AWK command
I have a input.txt file with following format:
Name age number address
mark 23 89756342192 chennai
david 40 70214789023 delhi
How do I create a CSV file from a TXT file as per the ...
- 21
0
votes
1
answer
31
views
bash: How to add column numbers to the first row
I had many large .txt like:
xx yy ... zz
XX YY ... ZZ
I want to add a header row to them, each with a different number of columns:
1 2 ... 999
xx yy ... zz
XX YY ... ZZ
Since they have a different ...
-3
votes
1
answer
80
views
Awk variable passing error
Below works fine:
awk '{print $1,$2,$3,$4,$5,$6,$7,$8,$57,$9,$55,$10,$11,$12,$56,$13,$53,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,...
- 1
5
votes
2
answers
140
views
Subtract a specific column of a file from another for even lines only
I have two long files that have the repeating general structure,
File1:
AA 100
BB 10
File2:
AA 100
BB 1
I want to subtract column 2 from even lines of File2 from File1, but maintain/print the other ...
- 227
2
votes
5
answers
105
views
how to turn multiple URLs into domains with command line
I have .csv files with multiple columns and ',' as separator. The url's are in the first column. I need to turn all url's into domains without removing the other columns
Example of the data I have:
...
- 123
0
votes
2
answers
56
views
How to calculate percentages of values in specific rows of a file?
I built a file where I have headers with generic IDs that identify multiple alignments and on the next rows the genome IDs that correspond to that alignment ID. Along with the genome IDs I have ...
0
votes
6
answers
85
views
rename the header of multifasta files
I have multi-fast files like this
>aaaaa.1_prot_CAA66754.1_10
MEKLLDAYLSSLRLNRRQVSEQTGLNYTTLQRASDKDALMISPRILWGIAMMVDKTPGQVLDELIELEMK
>bbbbb.1_prot_CAA66753.1_11
...
- 43