Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
0 votes
2 answers
39 views

awk script replace value with increment with if

I have a pipe delimted file d1000|1000 d1001|100 d1002|10 d1003|1 d1004| d1005| I want to modify $2 if length is less than 4 digits, and also keep empty spaces as is so trying to do it via awk script ...
user avatar
  • 1,603
0 votes
2 answers
61 views

awk print only modified lines

I have file like this d1000 1000 d1001 100 d1002 10 d1003 1 I want to modify second column where length is not equal 4. But I want to print only lines that are modified, so original text in column 2 ...
user avatar
  • 1,603
1 vote
4 answers
92 views

awk/sed split a cluster file in to multiple files

I have a cluster fasta file (called file) which looks like: >1AB2 >1AB2 AA NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI NCIDHFR8EHGBVPIWOBGIGRI >1AB3 AA WNIOREHUEBRGOUERGHBERGIORBGREUGEGO ...
user avatar
  • 125
0 votes
2 answers
34 views

awk how to set the format output filename for multiple output

I have a file (file.csv) which looks like this: abc 123 abc 001 abc 014 bos 011 bos 034 bos 079 bos 004 ... PS: column 1 of file.csv has 27297 unique words in total, and file.csv has 789234 rows in ...
user avatar
  • 125
-2 votes
4 answers
115 views

sed or awk: replace only the n-th occurence of a string

How to write a sed (or awk, or both) which will take the following: echo '1 aa 2 2 3 bb 5 bb 2 5' | sed/awk ... And only replace the n-th occurrence of a string? For example the 3rd occurrence of 2 ...
user avatar
2 votes
8 answers
299 views

awk filter rows that only consist a single letter

I have a file (file1) that looks like this: ROW 1 AA 120 APFGHKDESFNNJFHGRIHJASFGNSKDHFIXXXXXX ROW 2 AA 234 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX ROW 3 AA 122 XXXXXXXXXXXXXXXXXXXXX ROW 4 AA 89 ...
user avatar
  • 125
0 votes
1 answer
48 views

Return content between corresponding brackets

I have a file with all type of brackets {}[]()- nested, open and close appropriately. I would like to return the content within the matching square brackets after the string (text:). The content of ...
user avatar
  • 7
3 votes
5 answers
268 views

Bash: sed or awk rewrite of numerical sequence

How to write a sed (or awk, or both) which will rewrite the following: echo 'v100 v201 v102 v300 v301 v500 v999 v301' | sed/awk ... to this output: v1 v2 v3 v4 v5 v6 v7 v5 i.e. each subsequent vx ...
user avatar
-3 votes
3 answers
91 views

how to ignore negative numbers

let's consider the following example 1 4.0 3.5 7.2 2 3.2 2.5 5.2 3 1.7 -1.8 2.9 4 4.1 2.5 5.7 How can I extract the 3rd column of the table by ignoring the negative number? If I use awk '{print $3}' ...
user avatar
0 votes
3 answers
64 views

How to add column based in the second line match using SED or another command?

I have the following lines result from zcat command 1 - URL Template: https://www.test.com 2 - Response: <200 OK,{Server=[nginx], Date=[Wed, 11 May 2022 01:05:06 GMT], Content-Type=[text/html; ...
user avatar
1 vote
1 answer
37 views

Check info in each line on a specific set of columns in one variable against all lines in three specific columns in another variable using awk

I've asked a very similar question today before, however I have since realised that I need to increase parameters for the command. I edited the command for a another parameter okay, but the next ...
user avatar
  • 801
1 vote
1 answer
30 views

Extract field and number of occurrences per line

I have this file: John Green', 'Age: 32', 'State: New York', 'Total cars: 2', 'Manufacter: General Motor', 'Model: Pontiac', 'Year: 2000', 'Manufacter: Ford Motor', 'Model: Endeavour', 'Year: 2010 ...
user avatar
1 vote
1 answer
79 views

Substitution in html file using awk

My awk script reads records from file1, finds the same records in file2 and substitutes alternate positions (of the record) with a defined symbol in that. But few of the values are not getting ...
user avatar
2 votes
2 answers
81 views

Check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk

I need to check numbers in each line on a specific column in one variable against all lines in two specific columns in another variable using awk, keeping lines in the first variable that meet ...
user avatar
  • 801
0 votes
6 answers
83 views

How to ignore all text containing special character more than once?

Sample Input: apple_ig rabbit_cat_ig dog_ig bird_duck_ig orange_ig goat_ig Expected output: apple_ig dog_ig orange_ig goat_ig I need to ignore all data that contains underscore _ more than once in ...
user avatar
  • 1,001
1 vote
1 answer
45 views

awk replace column value with increment values if length equals

I have non-standard data, which I'd like to standardise file: d101 11001 e101 9665 f101 9663 d102 11002 e102 11003 f102 11004 g102 11005 desired output: d101 11001 e101 12001 f101 12002 d102 11002 ...
user avatar
  • 1,603
0 votes
2 answers
66 views

How to replace all occurrences of a sequence starting with a given string to the next appearance of a symbol?

I would like to edit a huge single line file with thousands of occurrences of some string 'string_string': until the next appearance of a comma , (including this comma) and removing those occurrences ...
user avatar
  • 171
1 vote
6 answers
105 views

AWK one liner to merge three fields in a single file

I have a file with records (lines) having two types of field delimiters | and ! as given below: Name|Age|Physics|Chemistry|Maths|English|Batch!Year!AdmisnNo!Grade!Score Student1|81|65|70|80|88|EWS!...
user avatar
1 vote
1 answer
56 views

How can I use `find` to find the files and awk to search within the files?

In a folder and its subfolders, I want to search a pattern. For this I am using: grep+cut+awk+tr pipeline. printf "I am so wide alper world zorro the end\t\t\talper\n" > alper.txt ...
user avatar
  • 230
2 votes
3 answers
551 views

How to set "[}" or "[)" as FS for gawk?

I'm using GNU Awk 5.0.1 and I need to use [} or [) as FS. I can't make it work. Below is what I've tried. root@u2004:~# echo test | awk -F '[}' '{printf}' awk: fatal: invalid regexp: Unmatched [, [^, [...
user avatar
0 votes
4 answers
92 views

Switch the last two fields with awk

I'm looking for a way to switch the last two fields of a tab-separated file. The number of fields vary among rows, so I can't specify the field number. Most of what I could find online was field-...
user avatar
  • 559
0 votes
3 answers
72 views

Add carriage return if field does not exist in awk

I am using awk -F'[":]' '$2=="id"{printf("pri,%s,",$5)}$2=="name"{printf("%s,",$5)}$2=="objectId"{printf$4}$2=="polledName"{print$5}' | ...
user avatar
1 vote
1 answer
69 views

How can I split a string with awk when FS is already in use?

Let's say I have a file.csv as the following: id;filename;device 1;118574934-20220503-17h44m20s101; 2;118574934-20220503-17h44m20s101; 3;118574934-20220503-17h44m20s101;DEVICE-0001-33 4;118574934-...
user avatar
  • 319
0 votes
3 answers
86 views

Delete line in each text files if the first field value is greater than 400

I have a large number of txt files. the format of each txt files is similar to this 200 0.2 0.1 0.5 0.4 500 0.4 0.9 0.9 0.1 I am trying to delete each line in each txt file that has the first field ...
user avatar
0 votes
3 answers
68 views

awk based solution for summing the rows of multiple files

I have several files which look like file1.dat: 1 1 1 3 4 5 9 10 11 file2.dat: 3 0 8 9 0 3 9 2 4 typically with many more rows (with each row containing one less column than the previous row). I ...
user avatar
  • 107
1 vote
3 answers
83 views

How to substact or sum 12 depending on the last and following line using awk?

I have this data: ##sequence-region Q75T13 1 641 Q75T13,UniProtKB,Chain,1,641,.,.,.,ID Q75T13,UniProtKB,Topological domain,1,60,.,.,.,Note=Cytoplasmic Q75T13,UniProtKB,Transmembrane,61,85,.,.,.,Note=...
user avatar
2 votes
1 answer
137 views

replace character only when it's not inside curly braces (sed, awk, tr, ...)

I have a string called "desktops" that look like this: desktops="1 2 %{F#990000}3%{F-} 4 5 6 7 8 9 0" The 6 digits after the "#" represent an RGB color, so the 3rd ...
user avatar
0 votes
1 answer
27 views

Cut last value of a string [duplicate]

My data is like (a.txt) allheartweb.com whoisdatacenter.com domainsanalytics.com covercian.co.uk I want to extract all TLD from there My code is cut -d "." -f 2 a.txt its show me com com ...
user avatar
-2 votes
2 answers
46 views

How to extract rows in a file based on common information in another file?

I have file look like: file1: 8189157 7423700 7227471 8189199 7533538 7574425 8189200 7533538 7574425 8189273 7241932 7538338 8191298 6353935 6770138 8191332 7427024 7756709 8192601 6353935 7378544 ...
user avatar
  • 1,263
3 votes
1 answer
107 views

Cut or Awk to save lines on array in bash

I have a scenario that I have to put lines of a file .txt inside an array, example of file content; type of host | | first host | | | | second host | | | | | | same ip ...
user avatar
5 votes
5 answers
366 views

If column has multiple values, copy line with each value separately

I have a file with the following format, each column separated by tabs: C1 C2 C3 a b,c d e f,g,h i j k l ... Now I need to have the number of lines according to the number of values ...
user avatar
0 votes
2 answers
61 views

How to replace the character in one column at the position indicated by another column with the string in a different column

I am rather new to bioinformatics (this is my first post!) and I would appreciate some help on task that has me stuck. I have a Tab-delimited data table with three columns: AATTCTTGCA 4 [A/T] ...
user avatar
0 votes
1 answer
45 views

Use awk or sed to capture & parsing text

I've some text output like the following: +-------------------------+-----+--------------------------------+--------------------------------------------+ | Site | SSL | URL ...
user avatar
7 votes
1 answer
519 views

How is awk doing comparison in this case?

I have a text file employees: Chen Cho 5/19/63 203-344-1234 $76 Tom Billy 4/12/45 913-972-4536 $102 Larry White 11/2/54 908-657-2389 $54 Bill Clinton 1/14/60 654-576-4114 $201 Steve Ann 9/15/71 202-...
user avatar
0 votes
0 answers
19 views

awk or sed - find and replace in multiple files according to a key [duplicate]

I have hundreds of files in which I need to make various replacements - and those replacements are not the same in all of the files. For example, I want to replace: WinEventLog with Windows SysMon ...
user avatar
0 votes
5 answers
129 views

Sed to replace text that comes after match

I was working it with awk, and keying on $1 $2 fields, but awk would act differently depending on whether the FieldSeperator ($match) starts the line, or had a blank before it, or didn't have a blank ...
user avatar
-3 votes
1 answer
103 views

time difference using sed/awk

I've 2 columns as below with Job submission date. NAME SUBMIT_TIME Job1 Apr 25 11:30 Job2 Apr 25 16:30 I need to have an alert which should trigger if the time difference is more than 60 min from ...
user avatar
0 votes
1 answer
27 views

reload the i3 conf in a script

i am writing a script to quick change the mod key of i3 conf0="/home/myuser/.config/i3/config" case "${1}" in (alt) key=Mod1 ;; (win) key=Mod4 ;; (*) printf "Invalid ...
user avatar
0 votes
2 answers
86 views

split string using a substring as delimiter and get the later part

Here is my string LIBRARY_TRGT_CANV,CANV_MATCH<anything> I wish to get all the text after last occurance of our delimiter which is _TRGT_: Desired output: CANV,CANV_MATCH<anything> ------ ...
user avatar
  • 297
0 votes
1 answer
40 views

Need help Formatting a file having key: value [duplicate]

I have a file having the below values: cat data.txt server1: 'calv' server2: 'anot' log: '/u/log/1' server3: 'calv' server4: 'anot' server5: 'secd' server6: 'calv' LIB_TRGT_calv,anot: '/tmp/hello.txt' ...
user avatar
  • 297
-1 votes
3 answers
72 views

Need to format text file and update based of key value format

I have a file having the below values: cat data.txt server1: calv server2: anot log: /u/log/1 server3: calv server4: anot server5: secd server6: calv LIB_TRGT_calv,anot: /tmp/hello.txt LIB_TRGT_secd: /...
user avatar
  • 297
0 votes
0 answers
43 views

Grouping and aggregating data

Can you please guide me, on how to group and aggregate the below value using a shell script (using awk) or guide me on which is the best option? Input textfile: 900|1000|currentaccount|usd|66.66|gbp|...
user avatar
0 votes
3 answers
74 views

How to use crossplatform/standard sed between linux and mac?

Hi I have one script that was running inside makefiles for project deployments. I use sed inside it to capture environment and project name from input string and in ubuntu works great. This simple ...
user avatar
2 votes
2 answers
101 views

merge multiple rows in specific columns

I have a file like this: xxx ax1 bx1 xxx ax2 bx2 cx2 xxx ax3 bx3 yyy ay1 cy1 zzz az1 cz1 zzz az1 bz2 ... I want the output like this: xxx ax1,ax2,ax3 bx1,bx2,bx3 cx2 yyy ay1 ...
user avatar
  • 125
4 votes
1 answer
66 views

GNU awk --traditional vs --posix

So, GNU awk has some extensions that are missing in the macOS awk. I want to be sure that my awk program also runs on the macOS awk (which I don't have access to). Now GNU awk has two different ...
user avatar
  • 151
0 votes
6 answers
102 views

How to cut string on last underscore character?

I have list of files like below: [root@ods1 backup]# ls -l total 93892810 -rw-r----- 1 root root 651248 Feb 17 08:34 abc_def_g_17-02-2022.sql -rw------- 1 root root 665248 Mar 23 08:46 bbc_23-03-...
user avatar
0 votes
1 answer
90 views

Merge all csv files in directory column wise

Say the directory contained 3 csv files: The first csv: Name, John Age, 18 The second csv: Name, Jim Age, 21 The third csv: Name, Amy Age, 22 I would want the result to be: Name, John, Jim, Amy Age,...
user avatar
  • 25
2 votes
4 answers
190 views

Add a column from one file as last column to another file

I would like to make a loop in which a certain column (in my case column 4) from a text file is added as last column to a new text file. I have in total around 500 text files (V1-V500) from which I ...
user avatar
  • 31
1 vote
5 answers
108 views

replace string x by string y according to string z

(EDITED) I have a file containing quite some problems. For example as below: Chr1_RagTag_p AUGUSTUS transcript 393571 396143 . + . transcript_id "TCONS_00000070"; gene_id "...
user avatar
  • 25
5 votes
5 answers
521 views

bash how to insert multiple empty lines into a file according to another file

I am meeting some difficulties when using bash to insert multiple empty lines into a file (called file1) according to an index file(called file2). (these files can be treated as passed variables) the ...
user avatar
  • 125

1
2 3 4 5
154