Questions tagged [awk]
A pattern-directed scanning and processing language.
0
votes
1answer
10 views
How to check if all first column fields are unique
I have a file where strings are separated by a special string (not a comma or delimiter) e.g. <vvv>. I want to check if all strings in the first field are unique. If duplicate rows for the same ...
0
votes
1answer
23 views
compare a certain number of lines between columns of two files
I have a thousand files and I need to verify if they all have exactly the same information in the second column up to a certain number of lines. Below an example. I would like to print the names of ...
0
votes
2answers
43 views
Find and replace Pipe using awk command
I am trying to grep column 7th with pipe delimiter (|) data in that I have to removed pipe (|) from middle of text.
Sample file:
col1|col2|col3|col4|col5|col6|col7|col8|col9|col10
value1|value2|...
0
votes
0answers
19 views
Modifying text file using (g)awk [duplicate]
Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
0
votes
3answers
62 views
How to extract first line that matches pattern from a file and if pattern doesn't exist print fixed output?
I have a text file shown below:
Doc_A 123 abc
Doc_A 456 def
Doc_A 789 ghi
Doc_B 123 abc
Doc_B 456 def
Doc_C 123 abc
Doc_C 456 def
Doc_C 789 ghi
Doc_C 101 jkl
And a reference file
Doc_A
Doc_B
Doc_C
...
1
vote
2answers
65 views
find a pair of words that appear the most of the times together
I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:
chapter1:
hello world good boy green sun
...
-1
votes
1answer
31 views
find the word that appears the most at the beginning of a line from entire paragraph
I have a paragraph and I want to know which word appears the most at the beginning of a line from all paragraph
for example:
paragraph:
Hello my name is X
Nice to meet you
Hello my name is Y
so ...
-4
votes
0answers
24 views
Linux script to move item from one folder to another
I want to write a script to read a file which contains the below information.The script needs to process each line.
Folder A goes to Folder AX condition=name starts with RED
Folder B goes to Folder ...
1
vote
1answer
34 views
awk syntax error inside script while assigning it to a variable
Basically memory is a variable. In a script, I am trying to remote login to one server and execute free command to check memory usage and using awk I'm getting the usage in percentage. But I am ...
7
votes
6answers
2k views
How to compare two different files line by line in unix?
File1:
123
234
345
456
File2:
123
234
343
758
Expected output:
File3:
TRUE
TRUE
FALSE
FALSE
so the code should compare two files and print 'TRUE' if it matches otherwise it should print 'FALSE' ...
5
votes
8answers
680 views
Delete nth line (when counted from the bottom)
There are unknown number of lines in a file. How to delete nth line (when counted from the bottom) with one-liner command (you may use more than one if it is necessary) on Unix platform.
0
votes
3answers
33 views
Extracting file name with spaces using awk
I have this line of output:
M file.txt
Getting fed into this awk program:
awk '{printf("\t%s: %s %s\n", FNR, $1, $2)}'
which returns
1: M file.txt
This works and is the usual case. However, if ...
1
vote
1answer
26 views
How to split file based on number of columns?
I have a file containing numeric data. Each line has a varying number of columns. I want to Split the file in to multiple files based on number of columns in a Line. Each Line may have columns varying ...
-1
votes
2answers
24 views
extract specific fields from a file using a character delimiter
I have a 1gb file and would like to get the third and fifth fields.
The file is delimited by (,):
input file:
luis, jose, jesus atoche, mark, john
1,2,3,4,5
aa 1, bb 2, cc3, dd 4, ee5
output file:
...
0
votes
1answer
27 views
Get Lines between pattern 1 and pattern2 which may accure multiple times
I try to print out the lines between two different patterns(including the patterns) which may accure multiple times in a file. The second pattern may accure besides the startpattern-endpattern-rhytm.
...
0
votes
1answer
21 views
modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing
There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file.
...
0
votes
1answer
17 views
Ignoring first record in awk
BEGIN{FS=","; t1 = 0; n = 0; avg = 0; sum=0; printf("%-10s %9s\n%-10s %9s","Name", "Average", "----", "-------") }
{
if ($3 > 0){
t1 += $3;
n++;
}
sum = 0;
if($3 &...
2
votes
2answers
42 views
goto a specific line in awk to start processing [on hold]
I want the awk to go to the beginning of the block of lines to start processing it again once it hit the bottom of that block and processed its last line.
Basically, if block of text is
<START ...
1
vote
1answer
30 views
Element comparison between subsequent lines, assigning a value for each null output
I have an output which looks like this:
ASCII file
I want to pick up the last element from the rows that contain the string "Fe\s2b". But the thing is each occurrence of "H\s\s1\s\s\s\s\s\s4861.33A" ...
2
votes
1answer
40 views
search for single or multiple occurrences of the samestring and edit next line
I have a file that sometimes has one occurrence or multiple occurrences of the same string. I am needing to search for one or more occurrences in the file and go to the next line and keep the values ...
-1
votes
1answer
25 views
Sort based on second column and list from highest to lowest
TGAAACATGCCTTTCACCTCCTGCCATGATTGTGAGGCCT 2
GTGTTGCCCAGGTTGGTTTCAAACTCAGCTCAAGTGATCT 1
GAGACTCCTTCAGGAGACTGGTCCCCTGTCCTCGCCTCTC 3
CATGTTCTCCTGGAAGCCATGTGAGGAAGGCACATCATGG 8
...
2
votes
1answer
171 views
Regex in IF condition in awk
I have awk script file as below. I need to add another condition in the if statement to check if the string contains atleast one alphabet. How can I add the extra condition to the present if statement?...
0
votes
1answer
21 views
Allign file based on field values
I have a file like below with two fields ending with *.alarms.gz and *.values.gz, want to allign the files like all the alarms.gz in first field and all the values.gz in second field. How can i ...
1
vote
1answer
48 views
Bash - How to update what's inside the quotes, but only if it's a number?
For this project, we have hundreds of applications using composer.json. I'm using a Bash script to go through and update the version numbers of any file that contain an application that is getting a ...
2
votes
3answers
178 views
How to get single character after space?
How can I obtain below output? I want the first field as it is and a single character after space.
echo "Hello world"
Hellow
If it also has a 3rd field than the beginning character of the 3rd field ...
0
votes
1answer
30 views
Renaming multiple paired files, deleting varying barcode character string in middle?
I have a bunch of paired files with unneeded barcode tags within the middle of the file, for example:
LIB008983_TRA00020080_TAAGGCGA-TATCCTCT_L001_R1.fastq.gz
LIB008983_TRA00020080_TAAGGCGA-...
0
votes
3answers
53 views
awk or grep? - print keyword and following columns
I did some benchmarks and the output is a single file containing multiple results with the following format
[blaaa]
1st run
T/V N NB P Q Time
==========================
2 3 4 1 2 1....
-1
votes
3answers
79 views
Extract files which have more than 5 lines, and keep same filename [on hold]
I have multipe files containing errors in annual reports. Each file name have .mvt extension. It is not necessary that every file has error. I have found that when errors appear OR not there is always ...
0
votes
4answers
36 views
Match the column heading and print the values of the column using awk
How to print column free using awk or sed command?
Depends on vmstat version column, free may change from column number 5 to something else.
bash-3.00$ vmstat 1 1|tail -2
r b w swap free re mf pi po ...
1
vote
2answers
54 views
Combining two outputs using AWK
I have a bash command (on macOS High Sierra) as:
sudo cat /private/var/log/system.log*| awk 'BEGIN {print "\n"} {print "Month\tDay\tTime\t\tSystemMessage"} /SHUTDOWN_TIME/{print $1,$2,$3,$8}'|sort -...
0
votes
0answers
36 views
How to rearrange a data?
I have Following type of data: each block starting with "Atom An X Y Z", I have 4 rows and 5 columns for each number point. I can arrange somehow for 3 columns and 2 rows with following code. awk '!(...
0
votes
2answers
42 views
Pick up successive lines containing keywords in order
I have a tab-separated file that looks as follows:
$ cat file
GCF_000015405.1_ASM1540v1.dist_nbr_anntn WP_011558474.1 1159543 1160595 -4330977 polyketide synthase [Mycobacterium]
...
0
votes
1answer
14 views
Invalid char error when running .awk file
I am appending a value 100 to every line in a file with the code below. This code works well when I run it in terminal but when I save the code in .awk file and run it I am getting an error. How to ...
0
votes
2answers
46 views
Extracting lines to new files
Say I have a large CSV file with a header and several columns. For the purpose of this question I will consider a small file with just two columns. We can call it use_rep.
user_id,rep
885,500K+
22565,...
1
vote
5answers
59 views
Question about text formatting using sed of awk
I want to format text using awk or sed.
I have text looking like,
H1 (dxfault)
H337 (Unxr)
H1106 (hast)
H1129 (h1129)
H1370 (hlan1370)
I want to remove the letter H and everything after the first ...
1
vote
2answers
52 views
Remove duplicates from particular columns
I have a file in below format, where columns are separated with comma.
[1], Value1, UAC, AB
[2.2], Check1, BOH D2A D2A BOH, SD
[63], name2, MFB MFB, k
...
I want ...
1
vote
3answers
66 views
How to compare two CSV files and display unique records?
I have two csv file as below, i want to compare file1 with file2 and if records present in file2 that complete row should remove from file1. field to compare here is ID and in original file its at ...
0
votes
0answers
18 views
Delete the second line which matches a pattern keeping the first line if it starts with DA [on hold]
I have a file which has following lines
Input
DA0183930283940484088X802004830202**0102937**60
DU0235363283940484088X802004830202**0102937**20
-------------------------------------------
------------...
0
votes
0answers
24 views
Inserting column headings in awk command
I have a raw file (gx.txt) having below sample contents shown below.
TIME;PCS/Main:GxCCAInitialSuccES;PCS/Main:GxCCAInitialSuccSILP;PCS/Main:GxCCATermSuccES;PCS/Main:GxCCATermSuccSILP;PCS/Main:...
0
votes
1answer
18 views
Replacing tabs with spaces from nth column to last column
I have a tab-separated file which looks as follows:
GCF_000014165.1_ASM1416v1.dist_nbr_anntn WP_011558474.1 1155234 1156286 44173 polyketide synthase [Mycobacterium]
GCF_000014165....
1
vote
2answers
71 views
Get logs for last 30 minutes
I want to extract the logs between the current timestamp and 30 minutes before. I developed the below script but it's not working properly it shows only timestamp lines whereas I have to get ...
1
vote
2answers
43 views
awk for filtering the contents on two files
$ file1.txt
12345,865221,APPLE,ZZ,QQ,DD,GG,APPLE-FRUIT
12346,865222,MANGO,ZZ,QQ,DD,GG,MANGO-FRUIT
12347,865222,GRAPE,ZZ,QQ,DD,GG,GRAPE-FRUIT
$file2.txt
APPLE-FRUIT,10KG
MANGO-FRUIT,12KG
I have ...
0
votes
1answer
41 views
Using IDs to extract ID headers and add to second column in file B
I have a file A that contains accession number family name and name and file B that contains ID and sequences.
I want to use the accession number in B to retrieve the family name and virus name in A ...
4
votes
4answers
494 views
extract characters between two commas?
I have a file with ~ 3 million rows, here is the first few lines of my file:
head out.txt
NA
NA
NA
NA
NA
gene85752,gene85753
gene85752,gene85753
gene85752,gene85753
...
0
votes
2answers
62 views
text processing between 2 patterns in awk to give selective unique output
I have below input file,
Policy Name: KE15-LOCALHOST-APP-RADIX-DAILY
Policy Type: Standard
Active: yes
Include: /appussd
/home/ussd2ke
/var/...
0
votes
1answer
38 views
Replace group of similar text in string from nth position with one string else increment the value by +1
I have a text file with following lines for example.
Input
DD0TRANSID000019021210504250003379433005533665506656000008587201902070168304000.0AK 0000L00000.00 N ...
0
votes
2answers
43 views
Add two doubles on bash
I'm trying to add two doubles
y1=0.17580197E-01
y2=0.11979236E-02
sum=`echo $y1+$y2 | bc -l`
the above script gives me sum = -2.704405652. How do i resolve this issue?
1
vote
1answer
56 views
Awk syntax, strange variable?
I have this awk command:
awk -F, 'NR==FNR{_1[$1]++;next}_1[$1]' file1 file2
I actually almost know what the program is doing, but I don't know what _1 means. The intention is to compare two files, ...
0
votes
3answers
81 views
Replace text in lines in a file with increments
I have a file with multiple lines (no. of lines unknown)
DD0TRANSID000019021210504250003379433005533665506656000008587201902070168304000.0AK 0000L00000.00 N 01683016832019021220190212N0000....
0
votes
1answer
32 views
awk filtering one column log file
I'm trying to use the following command to extract a time period forthe current day date and grep for the specific message as you can see below:
awk -v date="$(date +%Y%m%d')" '$1="date" && $...
