Questions tagged [awk]
A pattern-directed scanning and processing language.
6,939
questions
0
votes
4answers
47 views
How to remove double quotes when it comes within double quotes
I have a data like this
"Apple grape","Banana"orange","Red green"
"Apple grape","Bananaorange","Red green"
I want to remove the single ...
0
votes
3answers
36 views
Combine two files using common column
I have 2 files.
File 1 is:
burnj01,Joe Burns
steves02,Santosh Steve
File 2 is:
burnj01,0001
burnj01,0002
burnj01,0010
burnj,0100
burnj01,2000
steves02,2048
steves02,2049
steves02,2091
and I would ...
-1
votes
1answer
36 views
sed command to replace multiple lines after interval
I would like to replace the first character C with H for 23 lines (say, lines 83 to 106) after every interval of 82 lines. How can I do this using sed or awk?
1
vote
0answers
30 views
How to compare two column from two files with specific condition in awk
I have a data file A.tsv (field separator = \t) :
id clade mutation
243 40A titi,xixi,lolo
254 20B titi,toto,jiji,lala
261
267 20B lala,jiji,jojo
and a template file B.tsv (field separator = \t) :
...
-1
votes
0answers
51 views
Compare 2 Files using awk for multiple fields and process result for matching and not matching lines [closed]
I have 2 CSV files with matching and non matching fields.
I want to compare second, third and fourth column and based on that want to have output as matching(M), non-matching(NM) and also not found(NF ...
-2
votes
0answers
25 views
i want to extract multiple rows from a one field in csv file by using the awk file [closed]
#!/usr/bin/awk -f
BEGIN{
FS=";" }
{print " " $26 " " }
0
votes
2answers
50 views
How to move multiple files to multiple folders in linux?
I have a folder containing *.fastq.gz files (R1 and R2) for many samples. For example, in the folder "Raw_WGS", I have several files like
Sample_1_R1.fastq.gz, Sample_1_R2.fastq.gz, ...
1
vote
7answers
222 views
awk print columns separated by space
I have *.dat files like below in a directory:
$ cat holiday_us.dat
20210101 1 New Year's Day
20210102 2 Labor Day
20210103 1 Independence Day
Output I want:
20210101_New Year'sDay
...
3
votes
4answers
96 views
Find IP addresses in file excluding local IPs
The task: find and print lines from a file that contain any (IPv4) IP address except those on the local LAN (local LAN uses class C addresses of the 192.168.1.x variety). I (having cribbed bits from ...
0
votes
5answers
72 views
Remove block of lines from file with awk
I have a file looking like:
---
some metadata
...
---
Some real content.
...
Important content.
Expected output:
Some real content.
...
Important content.
To keep things simple, let's consider that ...
1
vote
3answers
71 views
How to extract certain columns with AWK from one CSV file to another?
I have a very big CSV log file that contains fields like this:
aaa=somedata1,bbb=somedata2,ccc=somedata3,eee=somedata5,hhh=somedata8
aaa=somedata1,ddd=somedata4,fff=somedata6,hhh=somedata8
aaa=...
9
votes
2answers
763 views
Why is there a syntax error if I don't write 'if' in an END block of AWK?
I'm learning AWK these days for text processing. But I'm very much confused about AWK syntax. I read on Wikipedia that the syntax follows this format:
(conditions) {actions}
I assumed I can follow ...
-4
votes
6answers
90 views
Duplicate lines containing a string with edits
I need to find a way to duplicate lines in a file containing a string with small edits. The dupe would have to show up above the original line. Would sed or awk be my best bet?
Ideally replacing ...
1
vote
1answer
49 views
How to create folders based on part of a filename and also extract & rename specific columns using awk
the following code extracts data from csv files contained in a folder, creates new folders (based on filename and date), creates and stores multiple csv files in the said folders. more info on this is ...
2
votes
2answers
64 views
How to compare two column of two file and print not matching pattern with awk
I have a data file A.tsv (field separator = \t) :
id clade mutation
243 40A siti,toto,mumu
254
267 40B lala,sisi,sojo
and a template file B.tsv (field separator = \t) :
40A toto,xixi,saxa
40B ...
1
vote
4answers
92 views
How to compare two column of two file and print the number of match with awk
I have a data file A.tsv (field separator = \t) :
id clade mutation
243 40A titi,toto,lala
254
267 40B lala,jiji,jojo
and a template file B.tsv (field separator = \t) :
40A lala,toto,xixi,...
4
votes
2answers
147 views
How to split CSV file and create multiple CSV files based on a column
I have a csv file with the format:
Input.csv:
TIMESTAMP,Data1,Data2,Data3,Data4
"2021-01-03 00:00:00",80953,3.243183,2.943338,358.0123
"2021-01-03 00:01:00",80954,2.173187,1.990327,...
0
votes
2answers
43 views
awk paragraph with string and the following paragraph in pgn chess games
How can I get a paragraph with the string and the following paragraph with awk?
The file looks like this:
[Event "Chess game"]
[Date "2021.01.08"]
[White "Player A"]
[...
-1
votes
5answers
84 views
Replace characters on specific lines and specific columns
Consider this sample.txt:
ATO N X B
AT H1 X BT
ATOM H25 X BAA
ATOM H3 X BUTZ
ATOM CA X BAT
I want to replace X's from lines 2-4, ...
1
vote
2answers
59 views
ksh88 AIX merge two files based on partial string match in one column
Using ksh88 on AIX machine, I've tried a whole slew of things that are ultimately not working.
File_A has 2 columns without headers: the user group that created the directory in column 1, and the full ...
0
votes
1answer
48 views
awk: how to extract strings directly into bash array?
let's say I have this:
ARRAY=()
NUM_EXTRACTED=$(cat somefile.txt | awk '/magic/{i++; print > i "_extracted.txt"} END {print i}')
for (( i=1; i<=$NUM_EXTRACTED; i++)); do
LINE=$(cat &...
-1
votes
2answers
51 views
How to merge two files based on the matching of one column?
I have two files, B.csv:
1,AD
2,AB
3,AC
5,AF
7,AE
and C.csv:
1,x
3,z
5,y
How do I get this output:
1,AD,x
2,AB,
3,AC,z
5,AF,y
7,AE,
by matching the common column 1 in both of the files?
0
votes
2answers
43 views
awk to vlookup and print
I got a comma-separated file
# cat data
smartplayer,2222,off
smartplayer,1111,on
scorer,0000,
I want to search for smartplayer in $1, get its state from $3
and print like this
off-smart-player 2222
...
0
votes
2answers
61 views
How to combine data from two file with the header in awk
I have two files, A.tsv and B.tsv:
A.tsv (field separator = \t):
Sample ID Internal Control Result Consensus
4686427 Pass Not Detected Not Available
4666275 Pass Detected Not ...
0
votes
1answer
42 views
How to use BASH variable for AWK filtering results
I have this script, I'm getting first letter of my directory name, and I wan't to get directory names that start with the same letter and write to textfile. The for loop is looping list of directory ...
-1
votes
4answers
101 views
How to split columns of a CSV file into separate files?
I have csv file namely list.csv as shown below,
Pcissicola19,cissicola39,12xbauhiniae
BGDHLHFA_02833,DGDFDEGP_00879,POPGJMOL_04119
BGDHLHFA_01427,DGDFDEGP_03106,POPGJMOL_01558
BGDHLHFA_01618,...
-1
votes
1answer
62 views
grep logs between two timestamps
2021-02-10 08:17:38 dell AptDaemon.Worker: INFO: Processing transaction /org/debian/apt/transaction/08a244f7b8ce4fad9f6b304aca9eae7a
2021-02-10 08:17:50 dell AptDaemon.Worker: INFO: Finished ...
0
votes
2answers
24 views
Removing strings after first space only in first column
The following file is tab-separated. I tried to remove from the first column NbLab330C00 64506568 the number after space in order to get NbLab330C00.
$ head LAB330_TE_annotation.gff3
##gff-version 3 ...
0
votes
0answers
48 views
how to replace a substring of a line after multiple keywords/patterns have matched
I need to go through thousands of files and find lines that start with
# disabledRule:
On these lines I need to look for a series of keywords, namely fileinto or fileinto_copy followed by create and ...
-1
votes
1answer
47 views
extracting multiple columns from multiple files and merge them to a single table
I have nearly 500 files each with more that 120 columns and some of the columns are blank.
hghflf.maf
lnuegflkn.maf
jdfrlbvfl11.maf
jkfbrhw4rkb4.maf
....
here is an example of the head of data for ...
1
vote
1answer
28 views
Repeat the value first column based on key column - Linux
I have a pipe delimited file like this where the second column is the key that groups the ids in the first column.
456566|23233|
456545|23233|
456745|23233|
456456|23234|
456356|23234|
I am trying to ...
0
votes
2answers
71 views
Pass awk variable to command and read output
I would like to sort file in parts delimeted by string, but it is a larger part of processing in the awk.
I have read that it could be possible to store lines between delimeter and then use asort for ...
0
votes
3answers
24 views
Unable to get the nth column data due to changing output of uptime command
Below is the output of uptime from Solaris where I am extracting the third last column:
uptime
8:30pm up 23 day(s), 17:46, 5 users, load average: **2.79**, 1.79, 1.53
I always need to get the ...
2
votes
2answers
79 views
extracting substring
I have the following file
Nt01 maker mRNA 143295 155540 . + . ID=Nitab4.5_0006317g0010.1;Parent=Nitab4.5_0006317g0010;Name=Nitab4.5_0006317g0010.1;_AED=0.08;_eAED=0.08;_QI=0|0.45|0.25|...
-1
votes
4answers
56 views
print all line that match my pattern until last end pattern match
I have a file like below, I need to print all line that match my pattern.
used the below code but end pattern stops after the first pattern match, I need to print until last end pattern match.
Input
...
4
votes
4answers
259 views
Reformat timestamp in a pipe delimited file
I have a pipe delimited file with date time in this format yyyymmddhhmmss as below.
John|Doe|TEST|20210728120821|john@test.com
John|Davis|TEST|20210828120821|john@test.com
John|Smith|TEST|...
0
votes
1answer
64 views
How to concatenate and sum multiple line in a file using awk
I have a file A.csv (field separator = ,):
Lane,SampleID,Index,# Reads,# Perfect Index Reads,# One Mismatch Index Reads,# of >= Q30 Bases (PF),Mean Quality Score (PF)
1,4331099,AACCATAGAA-...
0
votes
0answers
29 views
passing shell variable to awk and get awk output in shell variable [duplicate]
I have passed argument to myscript.sh that returns a value in variable m. Problem is that I am not able to pass shell variable (runvalues) to awk script that search for a pattern. Therefore value of ...
0
votes
1answer
32 views
Arithmetic between 2 files generating a series of new files (Pt 2)
This is a more specific follow-up to a previous question (Arithmetic between 2 files generating a series of new files).
I have a tab separated model input file I would like to vary for an ensemble ...
0
votes
3answers
71 views
Split a row by a delimiter
I have a space-separated text file, e.g.
text1a text2a id1 text4a text5a
text1b text2b id2 text4b text5b
text1c text2c id3,id4 text4c text5c
text1d text2d id5,id6,id7 text4d,text4di text5d
The file ...
-1
votes
0answers
52 views
AWK :string not found to be printed if pattern doesn't match in file
When the file doesn't contain searched strings, my awk program should print "Not Found".
Below is the code, error, and input file. Kindly help me in resolving this.
code
awk 'if
match($0,/...
4
votes
6answers
928 views
how to perform mathematical operations on numbers in a file using perl or awk?
I have a fileA.txt:
RS0255_RS0083:115,124,129,141,143,168,170,180
RS0343_RS0083:112,113,163,175,181
RS0343_RS0255:94,101,107,164,179,183
I would like to perform mathematical operations on the numbers ...
6
votes
5answers
526 views
grep output from long to wide
I have a file of patterns and I want to return all the line numbers where the pattern was found, but in a wide format and not long/spread.
Example:
fileA.txt
Germany
USA
UK
fileB.txt
USA
USA
Italy
...
1
vote
6answers
95 views
Arithmetic between 2 files generating a series of new files
I have a tab separated model input file I would like to vary for an ensemble analysis formatted similar to this
input.txt
/* Preciptation in mm */
10 30 40 50 23
### Species description
*** sp_name ...
1
vote
1answer
171 views
How to change date format in Linux Redhat
Is it possible to change date format from example Fri 12 Feb 18:27:34 +08 2021 to just simply DD-MM-YYYY?
Specifically, to change date format in last command
-3
votes
5answers
90 views
How to read list in JSON file using awk and sed
I have a list in JSON file apps.json:
{
"properites":{
"apps":["apple","orange","mango"]
}
}
I need to read this file and get of objects.
I used:
...
0
votes
1answer
56 views
How to change 1 line of text with existing entries split to multiple lines
RHEL 7 system.
I have an output file that looks like
eDTG=20210210-1540
Sent 74004956
Completed 72185529 (97.54%)
Lost 18119427 (2.46%)
QPS, SERVFAIL QPS 60122 2905
eDTG=20210210-1601
...
-1
votes
4answers
112 views
Count lines where the second field starts with a specific character
I have a file that looks like this
header start
stuff
header end
pos LV file LVG size
1 a1 AAA BBB 100
2 b1 AAC BBB 1000
3 a3 AAB BBB 47
4 b6 AAC BBB 1000
I need to count the lines that ...
-1
votes
1answer
34 views
Extract single field from string in android emulator environment [closed]
I am running android emulator with the adb shell
I try to extract the field _id=number from a query output.
The "raw" result looks like:
Row: 9991 last_time_contacted=0, phonetic_name=NULL, ...
0
votes
4answers
82 views
How to show only line after the matched one?
I have a file A.tsv (field separator = \t):
for Research Use Only
[Header]
Test Name Il
Run ID 2102
...
