Questions tagged [awk]
A pattern-directed scanning and processing language.
7,688
questions
0
votes
2
answers
39
views
awk script replace value with increment with if
I have a pipe delimted file
d1000|1000
d1001|100
d1002|10
d1003|1
d1004|
d1005|
I want to modify $2 if length is less than 4 digits, and also keep empty spaces as is
so trying to do it via awk script
...
- 1,603
0
votes
2
answers
61
views
awk print only modified lines
I have file like this
d1000 1000
d1001 100
d1002 10
d1003 1
I want to modify second column where length is not equal 4.
But I want to print only lines that are modified, so original text in column 2 ...
- 1,603
1
vote
4
answers
92
views
awk/sed split a cluster file in to multiple files
I have a cluster fasta file (called file) which looks like:
>1AB2
>1AB2 AA
NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI
NCIDHFR8EHGBVPIWOBGIGRI
>1AB3 AA
WNIOREHUEBRGOUERGHBERGIORBGREUGEGO
...
- 125
0
votes
2
answers
34
views
awk how to set the format output filename for multiple output
I have a file (file.csv) which looks like this:
abc 123
abc 001
abc 014
bos 011
bos 034
bos 079
bos 004
...
PS: column 1 of file.csv has 27297 unique words in total, and file.csv has 789234 rows in ...
- 125
-2
votes
4
answers
115
views
sed or awk: replace only the n-th occurence of a string
How to write a sed (or awk, or both) which will take the following:
echo '1 aa 2 2 3 bb 5 bb 2 5' | sed/awk ...
And only replace the n-th occurrence of a string? For example the 3rd occurrence of 2 ...
- 204
2
votes
8
answers
299
views
awk filter rows that only consist a single letter
I have a file (file1) that looks like this:
ROW 1 AA 120 APFGHKDESFNNJFHGRIHJASFGNSKDHFIXXXXXX
ROW 2 AA 234 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ROW 3 AA 122 XXXXXXXXXXXXXXXXXXXXX
ROW 4 AA 89 ...
- 125
0
votes
1
answer
48
views
Return content between corresponding brackets
I have a file with all type of brackets {}[]()- nested, open and close appropriately. I would like to return the content within the matching square brackets after the string (text:). The content of ...
- 7
3
votes
5
answers
268
views
Bash: sed or awk rewrite of numerical sequence
How to write a sed (or awk, or both) which will rewrite the following:
echo 'v100 v201 v102 v300 v301 v500 v999 v301' | sed/awk ...
to this output:
v1 v2 v3 v4 v5 v6 v7 v5
i.e. each subsequent vx ...
- 204
-3
votes
3
answers
91
views
how to ignore negative numbers
let's consider the following example
1 4.0 3.5 7.2
2 3.2 2.5 5.2
3 1.7 -1.8 2.9
4 4.1 2.5 5.7
How can I extract the 3rd column of the table by ignoring the negative number?
If I use awk '{print $3}' ...
- 19
0
votes
3
answers
64
views
How to add column based in the second line match using SED or another command?
I have the following lines result from zcat command
1 - URL Template: https://www.test.com
2 - Response: <200 OK,{Server=[nginx], Date=[Wed, 11 May 2022 01:05:06 GMT], Content-Type=[text/html; ...
1
vote
1
answer
37
views
Check info in each line on a specific set of columns in one variable against all lines in three specific columns in another variable using awk
I've asked a very similar question today before, however I have since realised that I need to increase parameters for the command. I edited the command for a another parameter okay, but the next ...
- 801
1
vote
1
answer
30
views
Extract field and number of occurrences per line
I have this file:
John Green', 'Age: 32', 'State: New York', 'Total cars: 2', 'Manufacter: General Motor', 'Model: Pontiac', 'Year: 2000', 'Manufacter: Ford Motor', 'Model: Endeavour', 'Year: 2010
...
- 1,101
1
vote
1
answer
79
views
Substitution in html file using awk
My awk script reads records from file1, finds the same records in file2 and substitutes alternate positions (of the record) with a defined symbol in that. But few of the values are not getting ...
- 11
2
votes
2
answers
81
views
Check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk
I need to check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk, keeping lines in the first variable that meet ...
- 801
0
votes
6
answers
83
views
How to ignore all text containing special character more than once?
Sample Input:
apple_ig
rabbit_cat_ig
dog_ig
bird_duck_ig
orange_ig
goat_ig
Expected output:
apple_ig
dog_ig
orange_ig
goat_ig
I need to ignore all data that contains underscore _ more than once in ...
- 1,001
1
vote
1
answer
45
views
awk replace column value with increment values if length equals
I have non-standard data, which I'd like to standardise
file:
d101 11001
e101 9665
f101 9663
d102 11002
e102 11003
f102 11004
g102 11005
desired output:
d101 11001
e101 12001
f101 12002
d102 11002
...
- 1,603
0
votes
2
answers
66
views
How to replace all occurrences of a sequence starting with a given string to the next appearance of a symbol?
I would like to edit a huge single line file with thousands of occurrences of some string 'string_string': until the next appearance of a comma , (including this comma) and removing those occurrences ...
- 171
1
vote
6
answers
105
views
AWK one liner to merge three fields in a single file
I have a file with records (lines) having two types of field delimiters | and ! as given below:
Name|Age|Physics|Chemistry|Maths|English|Batch!Year!AdmisnNo!Grade!Score
Student1|81|65|70|80|88|EWS!...
1
vote
1
answer
56
views
How can I use `find` to find the files and awk to search within the files?
In a folder and its subfolders, I want to search a pattern. For this I am using: grep+cut+awk+tr pipeline.
printf "I am so wide alper world zorro the end\t\t\talper\n" > alper.txt
...
- 230
2
votes
3
answers
551
views
How to set "[}" or "[)" as FS for gawk?
I'm using GNU Awk 5.0.1 and I need to use [} or [) as FS. I can't make it work. Below is what I've tried.
root@u2004:~# echo test | awk -F '[}' '{printf}'
awk: fatal: invalid regexp: Unmatched [, [^, [...
- 793
0
votes
4
answers
92
views
Switch the last two fields with awk
I'm looking for a way to switch the last two fields of a tab-separated file. The number of fields vary among rows, so I can't specify the field number.
Most of what I could find online was field-...
- 559
0
votes
3
answers
72
views
Add carriage return if field does not exist in awk
I am using
awk -F'[":]' '$2=="id"{printf("pri,%s,",$5)}$2=="name"{printf("%s,",$5)}$2=="objectId"{printf$4}$2=="polledName"{print$5}' | ...
1
vote
1
answer
69
views
How can I split a string with awk when FS is already in use?
Let's say I have a file.csv as the following:
id;filename;device
1;118574934-20220503-17h44m20s101;
2;118574934-20220503-17h44m20s101;
3;118574934-20220503-17h44m20s101;DEVICE-0001-33
4;118574934-...
- 319
0
votes
3
answers
86
views
Delete line in each text files if the first field value is greater than 400
I have a large number of txt files. the format of each txt files is similar to this
200 0.2 0.1 0.5 0.4
500 0.4 0.9 0.9 0.1
I am trying to delete each line in each txt file that has the first field ...
- 3
0
votes
3
answers
68
views
awk based solution for summing the rows of multiple files
I have several files which look like
file1.dat:
1 1
1 3 4
5 9 10 11
file2.dat:
3 0
8 9 0
3 9 2 4
typically with many more rows (with each row containing one less column than the previous row). I ...
- 107
1
vote
3
answers
83
views
How to substact or sum 12 depending on the last and following line using awk?
I have this data:
##sequence-region Q75T13 1 641
Q75T13,UniProtKB,Chain,1,641,.,.,.,ID
Q75T13,UniProtKB,Topological domain,1,60,.,.,.,Note=Cytoplasmic
Q75T13,UniProtKB,Transmembrane,61,85,.,.,.,Note=...
2
votes
1
answer
137
views
replace character only when it's not inside curly braces (sed, awk, tr, ...)
I have a string called "desktops" that look like this:
desktops="1 2 %{F#990000}3%{F-} 4 5 6 7 8 9 0"
The 6 digits after the "#" represent an RGB color, so the 3rd ...
- 33
0
votes
1
answer
27
views
Cut last value of a string [duplicate]
My data is like (a.txt)
allheartweb.com
whoisdatacenter.com
domainsanalytics.com
covercian.co.uk
I want to extract all TLD from there
My code is
cut -d "." -f 2 a.txt
its show me
com
com
...
-2
votes
2
answers
46
views
How to extract rows in a file based on common information in another file?
I have file look like: file1:
8189157 7423700 7227471
8189199 7533538 7574425
8189200 7533538 7574425
8189273 7241932 7538338
8191298 6353935 6770138
8191332 7427024 7756709
8192601 6353935 7378544
...
- 1,263
3
votes
1
answer
107
views
Cut or Awk to save lines on array in bash
I have a scenario that I have to put lines of a file .txt inside an array, example of file content;
type of host
|
| first host
| |
| | second host
| | |
| | | same ip ...
- 31
5
votes
5
answers
366
views
If column has multiple values, copy line with each value separately
I have a file with the following format, each column separated by tabs:
C1 C2 C3
a b,c d
e f,g,h i
j k l
...
Now I need to have the number of lines according to the number of values ...
- 11.2k
0
votes
2
answers
61
views
How to replace the character in one column at the position indicated by another column with the string in a different column
I am rather new to bioinformatics (this is my first post!) and I would appreciate some help on task that has me stuck.
I have a Tab-delimited data table with three columns:
AATTCTTGCA 4 [A/T]
...
0
votes
1
answer
45
views
Use awk or sed to capture & parsing text
I've some text output like the following:
+-------------------------+-----+--------------------------------+--------------------------------------------+
| Site | SSL | URL ...
- 511
7
votes
1
answer
519
views
How is awk doing comparison in this case?
I have a text file employees:
Chen Cho 5/19/63 203-344-1234 $76
Tom Billy 4/12/45 913-972-4536 $102
Larry White 11/2/54 908-657-2389 $54
Bill Clinton 1/14/60 654-576-4114 $201
Steve Ann 9/15/71 202-...
- 181
0
votes
0
answers
19
views
awk or sed - find and replace in multiple files according to a key [duplicate]
I have hundreds of files in which I need to make various replacements - and those replacements are not the same in all of the files. For example, I want to replace:
WinEventLog with Windows
SysMon ...
- 1
0
votes
5
answers
129
views
Sed to replace text that comes after match
I was working it with awk, and keying on $1 $2 fields, but awk would act differently depending on whether the FieldSeperator ($match) starts the line, or had a blank before it, or didn't have a blank ...
-3
votes
1
answer
103
views
time difference using sed/awk
I've 2 columns as below with Job submission date.
NAME SUBMIT_TIME
Job1 Apr 25 11:30
Job2 Apr 25 16:30
I need to have an alert which should trigger if the time difference is more than 60 min from ...
0
votes
1
answer
27
views
reload the i3 conf in a script
i am writing a script to quick change the mod key of i3
conf0="/home/myuser/.config/i3/config"
case "${1}" in
(alt) key=Mod1 ;;
(win) key=Mod4 ;;
(*) printf "Invalid ...
- 29
0
votes
2
answers
86
views
split string using a substring as delimiter and get the later part
Here is my string LIBRARY_TRGT_CANV,CANV_MATCH<anything>
I wish to get all the text after last occurance of our delimiter which is _TRGT_:
Desired output:
CANV,CANV_MATCH<anything>
------ ...
- 297
0
votes
1
answer
40
views
Need help Formatting a file having key: value [duplicate]
I have a file having the below values:
cat data.txt
server1: 'calv'
server2: 'anot'
log: '/u/log/1'
server3: 'calv'
server4: 'anot'
server5: 'secd'
server6: 'calv'
LIB_TRGT_calv,anot: '/tmp/hello.txt'
...
- 297
-1
votes
3
answers
72
views
Need to format text file and update based of key value format
I have a file having the below values:
cat data.txt
server1: calv
server2: anot
log: /u/log/1
server3: calv
server4: anot
server5: secd
server6: calv
LIB_TRGT_calv,anot: /tmp/hello.txt
LIB_TRGT_secd: /...
- 297
0
votes
0
answers
43
views
Grouping and aggregating data
Can you please guide me, on how to group and aggregate the below value using a shell script (using awk) or guide me on which is the best option?
Input textfile:
900|1000|currentaccount|usd|66.66|gbp|...
0
votes
3
answers
74
views
How to use crossplatform/standard sed between linux and mac?
Hi I have one script that was running inside makefiles for project deployments.
I use sed inside it to capture environment and project name from input string and in ubuntu works great.
This simple ...
- 103
2
votes
2
answers
101
views
merge multiple rows in specific columns
I have a file like this:
xxx ax1 bx1
xxx ax2 bx2 cx2
xxx ax3 bx3
yyy ay1 cy1
zzz az1 cz1
zzz az1 bz2
...
I want the output like this:
xxx ax1,ax2,ax3 bx1,bx2,bx3 cx2
yyy ay1 ...
- 125
4
votes
1
answer
66
views
GNU awk --traditional vs --posix
So, GNU awk has some extensions that are missing in the macOS awk.
I want to be sure that my awk program also runs on the macOS awk (which I don't have access to).
Now GNU awk has two different ...
- 151
0
votes
6
answers
102
views
How to cut string on last underscore character?
I have list of files like below:
[root@ods1 backup]# ls -l
total 93892810
-rw-r----- 1 root root 651248 Feb 17 08:34 abc_def_g_17-02-2022.sql
-rw------- 1 root root 665248 Mar 23 08:46 bbc_23-03-...
- 103
0
votes
1
answer
90
views
Merge all csv files in directory column wise
Say the directory contained 3 csv files:
The first csv:
Name, John
Age, 18
The second csv:
Name, Jim
Age, 21
The third csv:
Name, Amy
Age, 22
I would want the result to be:
Name, John, Jim, Amy
Age,...
- 25
2
votes
4
answers
190
views
Add a column from one file as last column to another file
I would like to make a loop in which a certain column (in my case column 4) from a text file is added as last column to a new text file. I have in total around 500 text files (V1-V500) from which I ...
- 31
1
vote
5
answers
108
views
replace string x by string y according to string z
(EDITED)
I have a file containing quite some problems. For example as below:
Chr1_RagTag_p AUGUSTUS transcript 393571 396143 . + . transcript_id "TCONS_00000070"; gene_id "...
- 25
5
votes
5
answers
521
views
bash how to insert multiple empty lines into a file according to another file
I am meeting some difficulties when using bash to insert multiple empty lines into a file (called file1) according to an index file(called file2).
(these files can be treated as passed variables)
the ...
- 125