Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

What is best for multiple word replacement in a large text file, sed or awk?

I have a list of words to be replaced in a large text file. I wanted to know which command sed/awk would be best to do the operation. Also does sed/awk read text file line by line or loads the entire ...
user avatar
0 votes
0 answers
47 views

Understanding If and Else If (awk) [closed]

I can't seem to get if and else if statements functioning correctly in awk. 'a' and 'b' will be a 1 or a 0. What I expect is that "ERROR" will never be printed, however when 'a' and 'b' are ...
user avatar
0 votes
0 answers
31 views

Using AWK replace command - how to consider space separated words as one word?

Background: Using below awk command which check pipe separated first field - wherever value 10 replaced with "No Files Available" from test.txt files. Question: It count "No Files ...
user avatar
  • 5
3 votes
3 answers
38 views

Reading a file with two columns and use them to change filenames

I have a text file with two columns, one representing the current file name and another representing the name I want as the final name of the file. The text file is like this: JALAZR010000001.1 - ...
user avatar
5 votes
5 answers
237 views

How can I replace multiple substrings from multiple lines in a file matching a pattern from a different file?

I have a file that contains multiple IP addresses and hostnames, and another file that contains some folders with many IP addresses per line. ip_hostname.txt host1 10.1.1.1 host2 10.2.2.2 host3 10.3.3....
user avatar
0 votes
1 answer
35 views

Awk replace string where specific condition is true using variables

I am trying to replace a string where a specific condition is true in the same line using awk variables. I found something that could help me a lot but I'm facing a new challenge cause a I need to do ...
user avatar
-1 votes
1 answer
58 views

Separating CSV (one column) into many columns on delimiter (comma)

I have a CSV with ~50 comma-separated values in one column that I want to separate into separate columns. The header is line 1. This should be really simple, and I've tried a lot surrounding awk and ...
user avatar
-2 votes
3 answers
183 views

How could awk be timezone aware?

If an input is a timestamp, and the timestamp happens to be in a common format (ISO-8601) that awk could be made to accept, then it is possible to print a seconds after epoch. For example, with input: ...
user avatar
0 votes
1 answer
35 views

Linux Column with value print

i tried awk for selecting 2 columns but now, the fourth column is somewhere blank and has value at some places, how to filter only the which has values a,b,c, d,e,f,g g,h,i j,k,l,m I need only f,g l,...
user avatar
2 votes
1 answer
65 views

bash + monitor disk space usage and execute script if used space reached the threshold

We want to follow the used space of /var/hadoop/hdfs partition , by way if used space is more the 50% then as results we run the script – do_action.bash , finally this command should be in crontab and ...
user avatar
  • 11.3k
0 votes
1 answer
31 views

Find the highest number after specific string from mutiple files

Each file has similar content: number 1: asdfa11sdfsadf number 2: asdfa12sdfsadf number 1: asdfa1sdfsadf number 3: asdfa33sdfsadf return 3 or number 3: are equally good. Have tried: max="$( awk ...
user avatar
0 votes
1 answer
31 views

Read from external file, skip lines after a match, print and exit

POSIX awk wizards, I need your help! The question may seem trivial to you at first glance, however, let me describe my intents in greater detail. I have been working on a self-contained POSIX awk ...
user avatar
  • 3
3 votes
1 answer
92 views

AWK return path for files with NF <= 2

I think I'm close here but I'm missing something stupid. I'm trying to make awk print out the filenames for files that are missing extended attributes or are missing an attribute value. So a file will ...
user avatar
  • 75
-2 votes
1 answer
46 views

Executing find command with a file having directory lists

I have a big file which contains directories paths like below: $PRJ/fp/t/mxl/lf/ $PRJ/fp/t/mxl/lf/ I want to read this file line by line and list out all the files inside each directory file in a ...
user avatar
  • 109
2 votes
2 answers
95 views

Merge muliple files having N common columns by awk and want to replace column value as 0 if any file does not have common key

I want to merge mulpiple files based on common column and want to add 0 when any file does not have that common column. For example see below: a1.txt 111,222,444,5.5 121,321,555,1.2 a2.txt 111,222,...
user avatar
-1 votes
0 answers
21 views

how to find patterns in a text using for loop and awk [duplicate]

I have two strings in a text like this in a text file called asl.txt: sub1_run-1.nii sub1_run-2.nii I need to print them using awk according to their run number. how can I do that? This is what I ...
user avatar
  • 1
-1 votes
0 answers
43 views

awk group an array with same substrings [closed]

I have a text file: ses-1: anat perf ses-1/anat: sub-1_ses-1_run-1_T1w.nii sub-1_ses-1_run-2_T1w.nii ses-1/perf: sub-1_ses-1_run-1_asl.nii sub-1_ses-1_run-1_m0scan.nii sub-1_ses-1_run-2_asl.nii sub-...
user avatar
  • 1
0 votes
4 answers
75 views

sed vs. awk: Read a file line by line and append to a specific line in another file

Consider two files below users.txt: Arnold.Bosch David.Ledru Mehri.Sunny and docker-compose.txt: version: '3.9' services: strongswan-server: image: vpn-strongswan container_name: strongswan-...
user avatar
  • 497
1 vote
2 answers
54 views

Awk -v variable assignment

*nix / awk newbie here. I'm wondering what is wrong with my syntax on the simple awk -v example below. # this line produces the expected result, 'xyz' echo "abc xyz" | /usr/bin/awk -v ttl=&...
user avatar
  • 13
4 votes
4 answers
483 views

Convert all small words (2-3 characters) to upper case with awk or sed

I need all 2-3 character words completely capitalized. 1 character and 4+ character words need to be remain untouched. Input: cat Example Dog I Fish su Su adm Amd Cat ignore Expected output: CAT ...
user avatar
-6 votes
4 answers
78 views

Remove the Top Level Domain (TLD) and Second Level Domain (SLD) with awk or sed

I have a list of domains without subdomains in a text file. I need the TLD and SLD removed. Input google.uk example.com amazon.co.uk domain.ca.uk education.edu.it Expected output: google example ...
user avatar
-3 votes
1 answer
45 views

Extract an ID that matches the user login from a table on shell script

So, I have all these user login information (service, admin, J009878) and this table you see here comes from another command whose output is this table which I save in file.txt. What I want to achieve ...
user avatar
0 votes
6 answers
91 views

Remove subdomains with awk or sed

How to remove all subdomains in a text file? I think I need to delete everything before and including the '.'? input: unix.stackexchange.com www.example.org example.example.tld Expected output: ...
user avatar
0 votes
4 answers
50 views

read file: pickup every text between the pattern including start and end pattern and create new file for each pattern set

I have a file called file.txt on Linux, containing: sqlplus -s insert into table; commit; ! sqlplus -s insert into table; commit; ! sqlplus -s insert into table; commit; ! . . ...
user avatar
  • 9
2 votes
3 answers
66 views

Replacing math equations by result

What is the best way to replace, in a file, equations found using some predefined regex with the result. Let's say each equation formula is compatible with bc -l (basic calculator with float numbers ...
user avatar
  • 23
2 votes
1 answer
58 views

awk print variable value

I would like to print the variable value defined outside awk snippet. Code Snippet FailCounter=5 awk -v Fail=${FailCounter} -F, ' BEGIN { print "<TABLE border=1>" ...
user avatar
-2 votes
4 answers
70 views

Rename headers of several multifasta files in bioinformation job

I have the 1.files like this: >YP_008856774.1 MHGTRTSAGWSTQPGKFDVLNLRMTFESSSAYQIPDLQPTEFIPTSLAAWNMPRHREYAAVSGGALHFFLDDYRFETVWS >YP_008856775.1 ...
user avatar
  • 43
2 votes
3 answers
160 views

Trying to find complete string values from one file based on another file using AWK

Hello I have 2 files with the first file containing a few values for example powershell vectormaps JuniperSA and the second file containing values and and ID appid uid SplunkforSnort 340 powershell ...
user avatar
0 votes
2 answers
36 views

Column number of containing a string in column

Related to this question, Column number of matching string in column I want to get the column number in which a certain string is contained. They did not work below, string=Bo* echo 'Hi Boo Jiggy' | ...
user avatar
0 votes
3 answers
50 views

How can I convert a table column to a row?

I have a tab-separated (\t) file with two columns: a_1 T_b a_2 T_c a_2 T_d a_2 T_e a_3 T_f a_4 B_a a_4 B_b I now want to convert that to a file with one sample per row and all values for each ...
user avatar
  • 43
1 vote
3 answers
48 views

insert text five lines before each pattern match

I have a file in which a pattern (eg: RotX) is repeated many times in a similar context. I need to insert a specific text (eg: Rot-X) at the start of every line which is located five lines before ...
user avatar
  • 11
0 votes
1 answer
17 views

How to filter rows by one column?

Sorry if this is a basic question, I don't use linux often but I have a 13GB file that I want to filter down. I have 123 columns and I want to remove rows that have only a . in the 75th column. I've ...
user avatar
  • 3
0 votes
3 answers
49 views

search for a string

How could I search file with below text and only redirect output where there is a "Msg 208" followed on the 3rd line by a "# name not found"? File Text: [ DBCC upgrade_object: ...
user avatar
  • 1
0 votes
3 answers
46 views

Understanding awk ' NR%2 ... `

Hi I am using 'Bash command line and shell scripts pocket primer' to study bash and I don't understand one chapter of awk, in which they illustrate printing even lines. command: awk ' NR%2 { printf &...
user avatar
0 votes
3 answers
84 views

lsblk + how to print only the disks that are without filesystem

we want to print only the disks that are without filesystem by command line from following example we can see that sde disk is without filesystem lsblk -o NAME,FSTYPE NAME FSTYPE sda ├─...
user avatar
  • 11.3k
0 votes
1 answer
35 views

awk judges and outputs all clumns that meet the conditions according to the specific column [duplicate]

Here is my file which contains three cloumns and were separated by "\t", and the second cloumn was sparated by blank. 1 a b c d e Tom 2 a b Jason ...
user avatar
  • 43
2 votes
4 answers
96 views

Combine columns into one column (stack)

I have multi-column data that I wish to reformat such that each column follow "on top" of each other in one single column. Example input: 1 2 3 4 1 2 3 4 Desired output: (fixed old ...
user avatar
0 votes
2 answers
51 views

split file by awk

Here is my file which contains three columns and separate by "\t", and the delimiter of the second column is a blank 1 a b tom 2 a b c sim 3 a mary ...
user avatar
  • 43
1 vote
1 answer
55 views

Replace newlines with spaces until a limit of characters without breaking text

I am configuring an /etc/hosts file on my router, and have a list of over 135,000 domains that I need to block via this file, which takes up nearly 5MB of space. As it turns out, the available RAM on ...
user avatar
-2 votes
1 answer
40 views

Bulk replace multiple variables in multiple files at once

Example: I have 40 different variables like $VAR in 100 .txt files which are located in /home/user/EXAMPLE/. How do I replace all variables in the 100 files? To tell the system what the variables are ...
user avatar
2 votes
4 answers
872 views

Grep excluding line that ends in 0, but not 10, 100 etc

I have a file that has this pattern in Linux and Unix environments. # file1 text1 98432230 text2 123412 text3 10 line4 0 line5 0 line6 40000 ... ... line10 ...
user avatar
  • 23
0 votes
1 answer
32 views

Bulk overwrite .csv files with AWK or sed

I have a directory with many sub-directories which are all full of .csv files (no files with other extensions). I want to overwrite all .csv files with a sed/AWK command in once. Moving everything to ...
user avatar
2 votes
4 answers
47 views

Pulling Red Hat release level from /etc/redhat-relase

I need to script a way to pull the actual release level of Red Hat boxes into a variable. I am focusing on the /etc/redhat-release file. I thought it would be a simple awk '{ print $7 }', but RH8 has ...
user avatar
2 votes
3 answers
156 views

Output in CSV format using AWK command

I have a input.txt file with following format: Name age number address mark 23 89756342192 chennai david 40 70214789023 delhi How do I create a CSV file from a TXT file as per the ...
user avatar
0 votes
1 answer
31 views

bash: How to add column numbers to the first row

I had many large .txt like: xx yy ... zz XX YY ... ZZ I want to add a header row to them, each with a different number of columns: 1 2 ... 999 xx yy ... zz XX YY ... ZZ Since they have a different ...
user avatar
-3 votes
1 answer
80 views

Awk variable passing error

Below works fine: awk '{print $1,$2,$3,$4,$5,$6,$7,$8,$57,$9,$55,$10,$11,$12,$56,$13,$53,$14,$15,$16,$17,$18,$19,$20,$21,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,...
user avatar
  • 1
5 votes
2 answers
140 views

Subtract a specific column of a file from another for even lines only

I have two long files that have the repeating general structure, File1: AA 100 BB 10 File2: AA 100 BB 1 I want to subtract column 2 from even lines of File2 from File1, but maintain/print the other ...
user avatar
2 votes
5 answers
105 views

how to turn multiple URLs into domains with command line

I have .csv files with multiple columns and ',' as separator. The url's are in the first column. I need to turn all url's into domains without removing the other columns Example of the data I have: ...
user avatar
0 votes
2 answers
56 views

How to calculate percentages of values in specific rows of a file?

I built a file where I have headers with generic IDs that identify multiple alignments and on the next rows the genome IDs that correspond to that alignment ID. Along with the genome IDs I have ...
user avatar
0 votes
6 answers
85 views

rename the header of multifasta files

I have multi-fast files like this >aaaaa.1_prot_CAA66754.1_10 MEKLLDAYLSSLRLNRRQVSEQTGLNYTTLQRASDKDALMISPRILWGIAMMVDKTPGQVLDELIELEMK >bbbbb.1_prot_CAA66753.1_11 ...
user avatar
  • 43

1
2 3 4 5
157