Questions tagged [awk]
A pattern-directed scanning and processing language.
7,049
questions
-1
votes
2answers
49 views
How to turn a tab-separated table into tagged sections with command line tools?
That is, turn this:
Column_A Column_B Column_C
-------- -------- --------
value-1a value-1b value-1c
value-2a value-2b value-2c
into this:
Column_A: value-1a
Column_B: value-1b
...
0
votes
2answers
96 views
How to print the number of occurrences of consonants and vowels for each file with awk?
I am trying to count the occurrences of consonants and vowels in multiple files on Linux, but I want the number of occurrences to be separately calculated for each file. I use
awk -v FS=""'{...
3
votes
2answers
137 views
How to print the number of occurrences of consonants for each file separately with awk?
I am trying to count the occurrences of consonants in multiple files,
but I want the number of occurrences to be separately calculated for each file.
I use
awk -v FS="" '{for ( i=1;i<=NF;...
4
votes
2answers
241 views
using awk, how can I find the max value in a column, then print a different field of the row that data was found in?
The specific example I have is finding the max value in column 5, then the associated 2nd field with that value. For example:
1 text1 1 1 5
2 text2 2 2 10
3 text3 3 3 15
4 text4 4 ...
-1
votes
1answer
63 views
Using sed to mass rename files according to contents in those files?
I have a bunch of files which all contain the pattern SIZE1=%d. Is there a way to find that decimal and prefix each filename with %d_?
Further if each of the files contain SIZE1=%d and SIZE2=%e can I ...
0
votes
1answer
45 views
Extract text between two tags and save each result in separate files
I want to retrieve whatever is between these two tags <data ... - </ data> and export each result in a separate text file
This command works very well, but unfortunately output is one file :-(...
0
votes
2answers
67 views
How do you properly format this log?
So I am listening on netcat on a specified port and am saving the output to a logfile. Apparently the log file is a oneliner... How do I properly format it? Extra points if you incorporate grc or ...
-1
votes
1answer
89 views
Counting specific character from file
Could someone explain to me what this part: ~/[bcdfg]/) of the following code:
awk ' {for(i=1;i<=NF;i++) {if ($i ~/[bcdfg]/) count++;
means? I know that it counts how often b,c,d,f appear but I ...
3
votes
3answers
106 views
Find and print the length of the longest line in multiple files?
In my folder, I have 2000 files, which are labeled A0001-A2000. In the second line of each file, there is a long (it should be the longest) line of characters. I would like to print all the file names ...
1
vote
1answer
26 views
Redirect outputs to different columns of the same file
I have a series 5 of commands working on several files located in several sudirectories. Suppose I want to redirect the output of every command as a new column in the same file, how can I do that. ...
0
votes
3answers
68 views
Replace specific lines of a file with values of other file
I would like to replace the 2nd, 6th, and 7th rows of file1 with the values in each row of file2 and generate a series of files equal to wc -l file2
cat file1
w
3
y
G
7
1.2
Q
cat file2
1 1 ...
-1
votes
0answers
40 views
Extracting data from multi line field value pairs
I have a file containing file value pairs in multiple lines.
Line: 1 Field name1: Value1 Field name2: Value2 Field name3: Value3
Line: 2 Field name1: Value1 Field name2: Value2 Field name3: Value3
...
3
votes
5answers
352 views
How to only keep line that start with a character and the line after
I have a FASTA file that has intentionally some sequences with wrong header (i.e absence of >) and some with good header. The file is well-formatted in the sense that the nucleotidic sequence is in ...
1
vote
1answer
43 views
How to add a column based on a regex
I have a file bla.tsv (FS = \t):
>hCoV-19/xxx/xxx-YYY/xxx
>hCoV-19/xxx/xxx-ZZZ/xxx
Precision :
If a character is really written, it's because it will be present in each line as it is
If a ...
0
votes
1answer
27 views
If a sum equals zero how can I get awk to print another known value to the file
I have a numerous files in which I extract the total number of records and total number of missing files in order to compute data averages. I have numerous files where the number of missing records ...
0
votes
3answers
38 views
Unzip and store output name on a variable
I have a zip file with always the same name : toto.zip
Inside there is only 1 file with the name who can change based on a version: totoInstaller-20210321_titi.pkg
I would like to extract the name of ...
0
votes
2answers
96 views
Arithmetic between 2 files generating a series of new files using awk
I have a tab separated model input file i would like to vary for ensemble analysis formatted such as this
cat input.txt
#############################################
### Parameter file for the ...
0
votes
1answer
38 views
How do I fix this syntax error from awk in my shell script?
I have a script with two codes to pull the sum of data of a given file and print to another file and a second pulls the total number of missing data points to another file. Finally the two output ...
-3
votes
0answers
60 views
Merging 2 files in an output based on matching columns using awk
I currently have a problem merging 2 files wherein certain values match. For file1, these are the types of different skus. for file2, these are their groupings. So in file2, skus 2,3,4 have their ...
3
votes
1answer
274 views
Shell script to remove child xml tags conditionally
I have a xml file having following content.
<contracts>
<clients>
<client>
<name>Nicol</name>
<clientRef>123</clientRef>
...
-4
votes
3answers
121 views
How do I calculate sum of numbers from different rows?
I have a file containing numbers in many rows and 5 columns like this:
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
...
If I want to get sums of the ...
2
votes
1answer
68 views
Portable range of integer value in AWK?
According to section 3.2.9 on page 39 on コンパチブル・シェルスクリプティング（第5版）, a reference book of how to make a compatible shell script, there are some AWK implementation with limited range of integer (...
0
votes
6answers
95 views
Remove spaces till the end of each line for each line in a file
I am trying to remove the spaces from each line till the last word for each line in a file.
Example input file:
808 0 C01124 Amazon 45/234Birch 00 YYY
808 0 ...
0
votes
1answer
25 views
to get output from a ssh for a for loop
I need to get result of which process is using most swap space on a remote server. I am using below command which works fine if I log on to remote and run it, but it gives no output if I run it by ...
0
votes
3answers
43 views
Multiple replace in AWK for tsv file [REFACTOR]
I have the followings lines in my file.sh file:
awk '$2 != "$ID1$"' myfile2.tsv > output && mv output myfile3.tsv
awk '$2 != "$ID2$"' myfile3.tsv > output && ...
0
votes
2answers
44 views
deleted records above this and matching primary key in txt file
I am trying to delete some matching records while doing txt processing
list.txt
tag1,slate1,flag1,status0,8,1
tag2,slate2,flag2,status0,2,3
tag1,slate1,flag1,status1,8,1
tag2,slate2,flag2,status1,2,3
...
0
votes
2answers
82 views
Merge lines with the same starting field
I would like to use bash tools to solve the following two related problems.
1. Suppose that I have a file with the following lines
A foo1
B bar
A foo2
A foo3 foo4
C ccc
How could I merge the lines ...
1
vote
1answer
49 views
I want to merge the output of top with the start time from ps
I have several services running and a bash script that displays their current state using top e.g.
top -b -n1 -p `pgrep -f "proc1|proc2|proc3|proc4|proc5" |
tr '\n' ',' | # ...
1
vote
4answers
361 views
Sorting numerically a list
I have a text list with the following structure (all the lines on each entry start with a tab space, there are no blank lines between those lines and there is one blank line between entries):
292G.- ...
0
votes
2answers
73 views
Replace part of a matched regexp with sed or any other tool
I need to replace a character on file portions matching a certain regexp. To be precise, I need to remove spaces with - in ill-formed markdown links, e.g.:
[This is my link](actual bad link.md)
...
2
votes
1answer
72 views
AWK don't match line feed \n on gsub
AWK don't match line feed \n on gsub().
Want change only the 'strawberry' fruit entry below.
Would be great with sed too.
It ignores \n as you can see on this script:
cat << 'EOF' > ~/src
...
0
votes
1answer
35 views
How to print value with a common key from two files but in a specific column in awk
I have a main file bla.tsv (FS=\t):
id hub fil plop
34 ji ji
54 jo jo
46 ja ja
And a info file hub.csv (FS=,):
34,fuse
46,fose
And I want to print the info of hub.csv into the column hub of bla....
-3
votes
2answers
107 views
deduplicate mostly negated awk
do not repeat yourself!, lol, ok, I'm trying to see if I can consolidate my 2 github workflows awk sources into 1. I tried a few things, but for some reason I kept ending up with weird column stuff. ...
0
votes
1answer
47 views
Arithmetic between 2 files generating a series of new files (Pt 3)
I have a tab separated model input file I would like to vary for an ensemble analysis formatted similar to this
cat input.txt
/* Precipitation in mm */
10 30 15 20 ...
1
vote
3answers
97 views
Combining several awk functions
I am modifying several files. I want to combine these three awk functions into one single code:
for i in *.gw; do
awk '/ALPHA_BF :/{sub(/^ *[0-9.]+/," #J #")};{print}' $i > $i....
0
votes
2answers
96 views
How to replace an email in a (PHP) file from the command line without sed?
I have a PHP file which contains an email address as with $to = "example-1_2@example-1_2.com"; and I want to replace the email address to another directly from the terminal.
The following ...
0
votes
1answer
79 views
Extract specific value from a big data matrix
I have a big matrix in which the first row contains the variable_name and the first column contains the name of the objects (variables and objects are the same, it's comparison all vs all). Moreover, ...
0
votes
1answer
68 views
How to use regex to test variable in awk?
I want to test if a script argument is only composed of letters.
here is the script :
BEGIN {
VALUE=ARGV[1];
if (VALUE ~ /[A-Za-z]/) {
print VALUE " : Ok only letters&...
0
votes
4answers
99 views
How do I get a multiline awk script running in bash
To start this script is running it a Github workflow, using shell bash, yaml truncated for readability. I've tried a number of things to make it work as multiline, so I can have comments.
set -x
...
-1
votes
3answers
68 views
Numerating a list of objects
I have a list of objects and the name of these has the same format (*_region_NNN where * represent a variable name and NNN are number). I need to obtain the same list in the same order but modify the ...
-1
votes
1answer
61 views
Print filename, number of line with file size and modification date
I want to print the File Name, Size of file, modification date, Number of line in a file.
At present I use below command to achieve my work but it's hectic to do manually everytime
ls -lrth file1.csv ...
-1
votes
2answers
64 views
get last same kind of record values with primary key
Hi I am trying to get last same kind of record values with primary key
list.csv
tag1,slate1,flag1,check1,1,2
tag2,slate2,flag2,check2,1,2
tag3,slate3,flag3,check3,1,2
tag1,slate1,flag1,check1,3,4
tag3,...
3
votes
4answers
185 views
Extract rows using different info in different columns
I have a big data file containing different information. I need to select and copy only some rows of this file in another one.
my_file.txt (Column are separated by "tab". I reported only the ...
3
votes
2answers
160 views
Merge multiple files by first column
My question is similar to this one: Merge multiple columns based on the first column values
I have multiple files (10+) that I want to merge/join into one output file, for example:
file 1
2000 0....
1
vote
1answer
52 views
Can I use awk to modify FitH values in a PDF?
I used pandoc to convert a Markdown document to a formatted PDF using pdfroff as the engine. The code for this is:
pandoc -s --toc --pdf-engine=pdfroff -o test.pdf test.md
The output file test.pdf ...
0
votes
1answer
57 views
Generate a file that contains the difference of two files
Instead of using the pythonic pandas way to produce such thing from two .csv files I am trying to see if I can do this with awk.
What I have is the following:
file1.csv | file2.csv
name,...
0
votes
2answers
43 views
Sum columns and count different values of another column
I have a file with four columns separated by ";" which looks exemplary like this:
Articles;Qty;Sales;Customers
ArticleA;2;6;Customer1
ArticleA;3;9;Customer2
ArticleA;5;15;Customer1
ArticleA;...
-2
votes
1answer
27 views
merging two files according to a common column; keep the value if the variant is present in file 2, and add 0 to those which are not present in file 2
I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column (3) following the order of file 1 and I want to ...
0
votes
5answers
97 views
How to print in only one column if condition with awk
I have a file bla.tsv (FS=\t):
id hub fil plop drift
3 187 0.91 bis XX
2 245 0.93 XX
4 250 0.70 bis XX
1 245 0.95 bis XX
And I want to print in the column plop the folowing string :
If hub<...
-1
votes
4answers
98 views
Grep for a range of numbers
I have a .txt file with multiple lines that gives amino acid and residue data. The data looks like this:
ARG262-Side ASP368-Side 140,83%
ARG95-Side GLU107-Side 103,73%
ARG474-Side VAL468-Main 94,93%
...
