Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
-1
votes
4answers
109 views

How to split columns of a CSV file into separate files?

I have csv file namely list.csv as shown below, Pcissicola19,cissicola39,12xbauhiniae BGDHLHFA_02833,DGDFDEGP_00879,POPGJMOL_04119 BGDHLHFA_01427,DGDFDEGP_03106,POPGJMOL_01558 BGDHLHFA_01618,...
-1
votes
1answer
115 views

grep logs between two timestamps

2021-02-10 08:17:38 dell AptDaemon.Worker: INFO: Processing transaction /org/debian/apt/transaction/08a244f7b8ce4fad9f6b304aca9eae7a 2021-02-10 08:17:50 dell AptDaemon.Worker: INFO: Finished ...
0
votes
2answers
28 views

Removing strings after first space only in first column

The following file is tab-separated. I tried to remove from the first column NbLab330C00 64506568 the number after space in order to get NbLab330C00. $ head LAB330_TE_annotation.gff3 ##gff-version 3 ...
0
votes
0answers
49 views

how to replace a substring of a line after multiple keywords/patterns have matched

I need to go through thousands of files and find lines that start with # disabledRule: On these lines I need to look for a series of keywords, namely fileinto or fileinto_copy followed by create and ...
-1
votes
1answer
54 views

extracting multiple columns from multiple files and merge them to a single table [closed]

I have nearly 500 files each with more that 120 columns and some of the columns are blank. hghflf.maf lnuegflkn.maf jdfrlbvfl11.maf jkfbrhw4rkb4.maf .... here is an example of the head of data for ...
1
vote
1answer
30 views

Repeat the value first column based on key column - Linux

I have a pipe delimited file like this where the second column is the key that groups the ids in the first column. 456566|23233| 456545|23233| 456745|23233| 456456|23234| 456356|23234| I am trying to ...
0
votes
2answers
83 views

Pass awk variable to command and read output

I would like to sort file in parts delimeted by string, but it is a larger part of processing in the awk. I have read that it could be possible to store lines between delimeter and then use asort for ...
0
votes
3answers
25 views

Unable to get the nth column data due to changing output of uptime command

Below is the output of uptime from Solaris where I am extracting the third last column: uptime 8:30pm up 23 day(s), 17:46, 5 users, load average: **2.79**, 1.79, 1.53 I always need to get the ...
2
votes
2answers
80 views

extracting substring

I have the following file Nt01 maker mRNA 143295 155540 . + . ID=Nitab4.5_0006317g0010.1;Parent=Nitab4.5_0006317g0010;Name=Nitab4.5_0006317g0010.1;_AED=0.08;_eAED=0.08;_QI=0|0.45|0.25|...
-1
votes
4answers
60 views

print all line that match my pattern until last end pattern match

I have a file like below, I need to print all line that match my pattern. used the below code but end pattern stops after the first pattern match, I need to print until last end pattern match. Input ...
4
votes
4answers
265 views

Reformat timestamp in a pipe delimited file

I have a pipe delimited file with date time in this format yyyymmddhhmmss as below. John|Doe|TEST|20210728120821|john@test.com John|Davis|TEST|20210828120821|john@test.com John|Smith|TEST|...
0
votes
1answer
66 views

How to concatenate and sum multiple line in a file using awk

I have a file A.csv (field separator = ,): Lane,SampleID,Index,# Reads,# Perfect Index Reads,# One Mismatch Index Reads,# of >= Q30 Bases (PF),Mean Quality Score (PF) 1,4331099,AACCATAGAA-...
0
votes
0answers
29 views

passing shell variable to awk and get awk output in shell variable [duplicate]

I have passed argument to myscript.sh that returns a value in variable m. Problem is that I am not able to pass shell variable (runvalues) to awk script that search for a pattern. Therefore value of ...
0
votes
1answer
41 views

Arithmetic between 2 files generating a series of new files (Pt 2)

This is a more specific follow-up to a previous question (Arithmetic between 2 files generating a series of new files). I have a tab separated model input file I would like to vary for an ensemble ...
0
votes
3answers
73 views

Split a row by a delimiter

I have a space-separated text file, e.g. text1a text2a id1 text4a text5a text1b text2b id2 text4b text5b text1c text2c id3,id4 text4c text5c text1d text2d id5,id6,id7 text4d,text4di text5d The file ...
4
votes
6answers
943 views

how to perform mathematical operations on numbers in a file using perl or awk?

I have a fileA.txt: RS0255_RS0083:115,124,129,141,143,168,170,180 RS0343_RS0083:112,113,163,175,181 RS0343_RS0255:94,101,107,164,179,183 I would like to perform mathematical operations on the numbers ...
6
votes
5answers
542 views

grep output from long to wide

I have a file of patterns and I want to return all the line numbers where the pattern was found, but in a wide format and not long/spread. Example: fileA.txt Germany USA UK fileB.txt USA USA Italy ...
1
vote
6answers
102 views

Arithmetic between 2 files generating a series of new files

I have a tab separated model input file I would like to vary for an ensemble analysis formatted similar to this input.txt /* Preciptation in mm */ 10 30 40 50 23 ### Species description *** sp_name ...
1
vote
1answer
183 views

How to change date format in Linux Redhat

Is it possible to change date format from example Fri 12 Feb 18:27:34 +08 2021 to just simply DD-MM-YYYY? Specifically, to change date format in last command
-3
votes
5answers
97 views

How to read list in JSON file using awk and sed

I have a list in JSON file apps.json: { "properites":{ "apps":["apple","orange","mango"] } } I need to read this file and get of objects. I used: ...
0
votes
1answer
56 views

How to change 1 line of text with existing entries split to multiple lines

RHEL 7 system. I have an output file that looks like eDTG=20210210-1540 Sent 74004956 Completed 72185529 (97.54%) Lost 18119427 (2.46%) QPS, SERVFAIL QPS 60122 2905 eDTG=20210210-1601 ...
-1
votes
4answers
119 views

Count lines where the second field starts with a specific character

I have a file that looks like this header start stuff header end pos LV file LVG size 1 a1 AAA BBB 100 2 b1 AAC BBB 1000 3 a3 AAB BBB 47 4 b6 AAC BBB 1000 I need to count the lines that ...
-1
votes
1answer
37 views

Extract single field from string in android emulator environment [closed]

I am running android emulator with the adb shell I try to extract the field _id=number from a query output. The "raw" result looks like: Row: 9991 last_time_contacted=0, phonetic_name=NULL, ...
0
votes
4answers
83 views

How to show only line after the matched one?

I have a file A.tsv (field separator = \t): for Research Use Only [Header] Test Name Il Run ID 2102 ...
0
votes
2answers
50 views

How to compare data from 2 files then create an output file that has the missing data in the order of file 2

I'm new to shell scripting and would like to create the below. I believe I could use something like grep -xvFf file1 file2 but I'm not sure how to create output file with the correct order. I have ...
1
vote
2answers
55 views

AWK pattern-code-block syntax: how to run code blocks by pattern in AWK?

Suppose I have a couple of line formats: this:foo ... that: foo ... And I want to run two different routines for one form or the other: cat text_file \ | awk -F ':' '/:foo/ { # do this } # else print ...
0
votes
3answers
107 views

Parse yaml file without using outside tool [duplicate]

I am new to bash, dragged into a problem where I need to update some dynamic parameters in a yaml file Based on query (search4.subsearch1.name), i need to update name: old_name into name: new_name ...
-1
votes
2answers
71 views

How to print data in a template from multiple file in awk

I have a main file A.txt (field separator = \t) : Sample ID Internal Control Result Consensus Sequence Lane Index Set Index ID 2154686427 Pass Detected Not Available 1,2,3,4 1 ...
5
votes
2answers
282 views

replace lines in one file with lines in another by line number

I have a fileA.txt with many lines and I want to replace specific lines with other specific lines from a second file fileB.txt which also has many lines For example: fileA.txt Italy Korea USA England ...
0
votes
3answers
68 views

Relative log file matching with linux?

I already know how to use grep or awk to match something in a log file, but is there a way to do the following? Match a specific line in a file...and include it in the result. From the matched line ...
0
votes
3answers
76 views

find text in one file and replace with text from another file SED or AWK

Ubuntu 18 build I am trying to find this text, [define( 'AUTH_KEY',] without [] in /var/www/file1.txt, and then copy the line and replace the same line in /var/www/file2.php. I have tried multiple ...
-1
votes
3answers
57 views

awk text processing take values of limits

i have a txt file like this 32 1 32 2 32 3 32 4 32 5 17 8 17 9 17 10 17 11 14 33 14 34 14 35 and i want to get txt file lke this one ! so we take the limits MIN and MAX of a value in field one from ...
-3
votes
3answers
43 views

awk columns based on condition with header

I have a text file like below. variable1 10 20 30 40 50 60 variable2 2 4 40 3 2 1 variable3 2 4 2 3 2 1 If col3>20 the output should be like below with header(variable1/2) values ...
1
vote
0answers
63 views

How to awk only output-lines containing a keyword?

I am in the process of writing a small script, which checks browser-tabs on certain keywords. If the keyword is present, then the script closes the tab. The script is actually a simple "block-...
0
votes
1answer
53 views

Compare three columns of two files

I have two files that have some similar and some unique columns. I'd like to compare the first 3 columns that are similar and print if they match, along with each of their unique columns. File1: A cat ...
0
votes
4answers
84 views

Finding a string and remove all contents between two delimiters

I have searched and I don't know what I'm doing wrong but I can't find an answer to this question. I have a file with all text stored as a single line. I am needing to find a pattern and remove all ...
0
votes
1answer
39 views

How to compare column in two different csv file and replace column from another file

I have a requirement to match column 4 (Grade) from File1 with column 1 (Grade) of File2. If it matches, replace column 4 (Grade) of File1 with column2 (Marks) of File2. If it doesn't match, then ...
1
vote
1answer
99 views

Check number in another file if in range using shell script and output all information for that match

I have two input files File A chr1 1167639 0 chrX 114868305 14 chr1 1167653 0 chr11 62389325 2 chr9 140127768 17 File B chr10 73082492 73082831 SLC29A3 chr11 46332562 ...
0
votes
2answers
41 views

How to merge two files of different number of lines using blank line condition?

I have two files, one have diffrent section separted by blank line as follows a11 a12 a13 b11 b12 b13 c11 c12 c13 a21 a22 a23 b21 b22 b23 c21 c22 c23 d21 d22 d23 a31 a32 a33 b31 b32 b33 the second ...
1
vote
2answers
67 views

How to divide multiple values within a column by a constant?

I am trying to divide the values in 2 columns by a certain constant. However, there are multiple values in each column that are separated by a colon. The file is very large (24 files, 2-3 GB each) For ...
1
vote
2answers
56 views

awk: change line to itself parsed with command. Multiline text

I'm trying to answer a question. It involves using the command recode. A specific line has to be parsed by the command leaving the rest untouched. The file is an xml file. Example: <ITEM_ID>...
-2
votes
4answers
127 views

Replace 0 with 1000 in Port 0 And increment by +1 in following lines

I want to increment port numbers starting from 1000. I know how to make all port 0 when they are assigned numbers with this command sed 's/port [0-9]\{1,5\}/port 0/g' but do not know how to do the ...
1
vote
1answer
54 views

Merge multiple CSV files from subfolders into a unique file and add file name in a new column

I have a bunch of CSV files in many subfolders (only 1 level of subfolders from the parent directory, nothing is nested further). Each CSV file has three columns; i, k, v with a mix of string and ...
-2
votes
1answer
56 views

sed XML parse select text between tags with special characters while deleting other blocks

I am trying to parse an XML which consists of blocks between <JOB> and </JOB> , I am parsing within a do while loop , while reading the block between <JOB> and </JOB> , I am ...
1
vote
1answer
54 views

How does this awk command work?

Consider the file test.txt: pfg025G pfg025T pfg034T pfg039G Now consider the following awk command and its output: awk '(NR>1) {print "s/"p"/"$1"/g"}{p=$1}' test.txt ...
0
votes
4answers
168 views

split a multifasta file into different files with new headers

Input file: 131751_pphA.fasta >ID:NDNDCOEC_02118 |[Genus species]|strain|PANS_1_2_annot.gbk|pphA|855|NODE_3_length_422941_cov_112.146787422941(422941):170566-171420:1 ^^ Genus species strain strain....
-2
votes
2answers
26 views

Move matched line with expr1 before another matched line with expr2 above in a file containing several expr2

I have a folder containing several files. In each of these files there might be one or more occurrences of the word implicit, and in some of the files with implicit there might be the sentence use ...
1
vote
2answers
95 views

How to collapse consecutive numbers into ranges and get the average

I have many .txt files with the following information in 4 columns and thousands of rows: chr10 73121691 18 SLC29A3 chr10 73121692 14 SLC29A3 chr10 73121693 10 SLC29A3 chr10 73120590 15 SLC29A3 chr10 ...
0
votes
2answers
48 views

awk - Remove a line, if a specific column contains as substring an element from another textfile

I have the following tsv-file (extract): File 1: NC_002163.1 RefSeq source 1 1641481 . + . organism=Campylobacter jejuni subsp. jejuni NCTC 11168;mol_type=genomic DNA;strain=NCTC 11168;...
5
votes
8answers
874 views

How to count the lines containing one of two words but not both

I need to count the lines containing the words the and an in a text file (poem.txt), but not those containing both. I've tried using grep -c the poem.txt | grep -c an poem.txt but this gives me the ...