Questions tagged [awk]
A pattern-directed scanning and processing language.
7,049
questions
-2
votes
0answers
22 views
How to Print unique IP address and Date with header of Log files in Linux by using a simple script
can anyone please explain me how separate ip address and date of apache access logs in the following way
unique IPs date
15.1.1.0 12/jan/2019
5.1.0.1 13/jan/2019
200.1.1.1 ...
0
votes
3answers
50 views
Using printf with AWK to format my output correctly
I'm trying to format some output using printf with awk. Basically, I want the final format to look like this (everything is tab-separated.)
chr10 100000624 100000625 10:100000625_A_G
...
0
votes
3answers
45 views
Using awk to print part of one column plus additional columns
Good day.
Using only Linux OS tools, how would you convert the following text from this:
cat latest.csv
apsga,alp1acscf0001vm001oam001p1n004v001,192.7.129.7,UVP
apsga,...
0
votes
1answer
17 views
select data conversion based on value range
I have a text file with 3 columns of data. However at random times in the various files there is a change in the in the observed unit from ppn to ppb resulting in the need of a conversion factor and ...
2
votes
2answers
118 views
How to compare two files to get matched records?
I have 2 files with * delimiter, each file with 3k records.
There are common fields in different positions. In file1(count=1590) the position is 1 and in file2(2707) the position is 2. file2 count and ...
0
votes
4answers
59 views
How to replace space with another value?
I am having file with | delimiter 8k records. If column#3 is empty replace with column#2 values. How do we achieve this.
input:
1|100437251|
2|51414204|
3|111651604|111651604
4|8321737|8321737
5|...
0
votes
2answers
40 views
Using awk '{printf…' multiple columns
How do I print multiple columns using awk '{printf...? With awk '{print... it is easy:
$ printf bar\\t5 | awk '{print $2,"baz",$1}'
5 baz bar
But if I want to format the digit to show a ...
-3
votes
1answer
41 views
Adding a values columns in a text file
I have a text file with three columns as below
568234.099421 4497975.785405 0.000000
567992.699441 4498682.885472 0.100000
568738.099488 4499368.184928 0.600000
568029.798936 4499762.584742 0.600000
...
0
votes
2answers
31 views
How to trim off specific parts of multiple directory names?
I have 100 directories within a parent directory (/home/anik/"all my directories") with the following naming structure:
Zt.isolate1.Spades
Zt.isolate2.Spades
Zt.isolate100.Spades
I want to ...
1
vote
1answer
51 views
How to get string between starting with X to ending with Y using Shell
I have dynamic string which are mentioned below, where I want to get ABCXXX.XXX.XXX string till , character. Starting character is fix (ABC)CAPITAL ONLY and ending also fix com or Xcom.
String can be ...
1
vote
2answers
33 views
How to delete duplicate line within a bracket preferably using one-liner?
My intention is to remove duplicate Param within each test. There is multiple tests that might have duplicate Param and I want to remove the 2nd Param duplicate within each test.
My file is as below:
...
1
vote
1answer
52 views
incrementing variables in awk if-else
I have the following code:
TOTAL=0
COUNT=0
# a ratio of 10000 indicates that ILP # rejected was 0.
awk -v c=$COUNT -v t=$TOTAL
'BEGIN { FS=OFS="," }
NR==1 { $(NF+1)="RATIO"...
0
votes
1answer
33 views
awk catch missing columns
I have a file which looks like this:
SOME,DESCRIPTIONS,OF,THE,COLUMNS
0,93,50127,0,49592,1,49592,1
100,195,107336,0,106491,2,107077,2
200,298,178492,3,170578,24,172045,13
...
2590,0
And I'm trying to ...
-1
votes
1answer
49 views
Add and subtract values group by Unique ID
I have records like this: I have to add/subtract the values of columns 2 and 4 if there is add and subtract if it is DEL.
12345)63)ADD)3
12345)40)ADD)2
12345)40)ADD)2
12345)40)DEL)2
67892)30)ADD)5
...
0
votes
0answers
44 views
how to properly match data in awk and fill in missind data based on a known date/time file
I have a data study running from 1/1/2002 00:00 to 12/31/19 23:00. Not all location have the same time ranges thus to make data processing easier I have a script to fill in the missing dates YYYYMMDD ...
-2
votes
3answers
64 views
Filter a word list using a stop words file
So I have a text file with a random text, and I need to list all the words present in that file, but filtering all the words common to my stop words file. What commands can I use for this?
0
votes
1answer
26 views
How to keep blank data or assign missing data when recompiuting data for unit shift
I have a series of data files with format below. I have a script to read just the data to the needed units (ppm to ppb) thus the *1000 of column $3. The issue which I am running into when awk runs and ...
1
vote
1answer
46 views
Concatenating text files based on prefix?
In a directory, I had 9792 files from 1088 groups (1088 x 9 = 9792) each group had a unique ID. I'm interested in concatenating only those files which match group ID as a prefix.
OG00 - prefix for all ...
0
votes
1answer
44 views
Adding and changing columns in a file with awk
I have the file of about 300 hundreds lines
TITLE cargas
REMARK 1 File created by GaussView 5.0.9
HETATM 1 O 0 0.957 -0.000 -0.000 O
HETATM 2 H ...
0
votes
3answers
54 views
Find multiple words and print the next numbers within a single line
I saved a single line sql result to a file.
AA 10 BB 20 DD 14 GG 14
I want to count the numbers next to each words and save it as variables.
And if there's none(like CC) it should be counted as 0.
...
0
votes
2answers
71 views
Creating new tables based on specific parameters from an existing table
I want to create several separate CSV files from a table.
Here is an example table:
gene REF_S1_host REF_S1_FL S1_host1 S1_host2 S1_FL REF_S2_host REF_S2_FL S2_host1 S2_host2 S2_FL
gene1 ...
-4
votes
0answers
59 views
Mutate a particular amino acid coordinates in protein file using grep or awk with sed command
I have a file which contains the atomic coordinates in coloumn. I have to substitute the coordinates (col 7,8,9,10,11,12) of LYS present, id 24, col 6, with PHE (col 4) res id (25, col 6).
I was able ...
-1
votes
1answer
73 views
Extract columns from table using loop
I have this matrix.
#chr st start PM1_r1 PM2_r1 PM3_r1 PM1_r2 PM2_r2 PM3_r2
chr1 - 10625 0 0 0 0 0 0
chr1 + 16616 0.10217712 0 0 1 0 0
chr1 - 120482 ...
1
vote
3answers
115 views
How to extract a substring starting with a word from string
I am writing a bash script and at some point I have
$path="things useless things... path/another/files1/useful/path/here"
So I am trying to remove all words and text before "files1"...
0
votes
3answers
70 views
Adding a column to a three columns file with awk
I would like to add a new column to an existing matrix in file.txt using awk.
I have the file
-1.664405 -0.019498 0.487501
-2.210156 0.326547 1.193717
-2.260318 -0....
-1
votes
5answers
103 views
Find and replace text in bash script terminal
I have a file at /home/count/1/details/info.txt as follow
title1 {
key1 value1
key2 value2
key3 value3
}
info {
name1 text
post1 anything
salary 1
work day1
work day2
work day3
}
work {
department ...
0
votes
3answers
94 views
awk getting rid of blank spaces
I am creating a script that checks to see if the first character in the second line of an output file is " " (blank or not). If it is blank I want to echo 'yes' if not echo 'no'. I am ...
-1
votes
5answers
98 views
Calculating specific average
I have a log with a long variable string and I am trying to get min, max and average from a number.
String examples
date time from Time: 100 ms to status code:
date time Time: 1050 ms status code IP
...
-3
votes
0answers
62 views
how to sum first N numbers of a line
i have a file with a client name like client1.txt with first line represents the salary of client per month, second line represents the costs that he have per month.
> 80
>> 12 10 19 20 18 16 ...
1
vote
3answers
56 views
Error in adding a suffix in a list of identifiers
this is a recurrent topic but still not duplicate since I tried all the solutions and none worked for me.
I am trying to add the suffix "_1" to a list of identifiers imported from a txt file....
-1
votes
2answers
48 views
bash + print all enabled service on linux machine with up-time
I created the following simple script in order to give the list of all systemctl service with the uptime
we run the script on our rhel 7.2 machine
#!/bin/bash
for i in ` systemctl list-unit-files | ...
4
votes
8answers
585 views
Find files with matched whole lines from a file
I have a file with this content:
$ cat compromised_header.txt
some unique string 1
some other unique string 2
another unique string 3
I wanted to find all files that have all the lines of above file ...
2
votes
3answers
240 views
Compare two lines in a file and delete those if pattern matches
I have a file like this.
12345 X678GHR 0 ADD
23445 HGT6787 1 ADD
12345 X678GHR 0 REM
67894 OIY5678 0 ADD
12345 OIY5678 0 ADD
12345 X678GHR 1 ADD
I have to compare the lines in a file to delete the ...
0
votes
2answers
95 views
How to append a missing line only by comparing 2 columns in 2 different files
I have two files
file1 (reference file)
xxx xxxxx 00
xxx xxxxx 01
xxx xxxxx 02
xxx xxxxx 03
xxx xxxxx 04
xxx xxxxx 00
xxx xxxxx 01
xxx xxxxx 02
xxx xxxxx 03
xxx ...
-1
votes
2answers
68 views
Show content of specific line number from a file
So say i have a file with content on each lines, know the number of lines in advance and want to know the content of specific lines, i can do:
awk 'NR==10' file
But while i already know one way (...
-1
votes
1answer
42 views
Replacing numbers in a text file in each iteration with random values
I have a text file with the following content:
number of apples 7
I would like to run the shell script and modify the number with random values for say 5 iterations. So I need my output to look like ...
0
votes
2answers
81 views
How to use awk instead of cmp to store comparison results to variables
Assume I have a file called checkDuplicateDownload.sh to check the downloaded file for duplicate or not, by comparing the new one and the old one.
This is what I have done.
duplicate=`cmp $prevFile ...
0
votes
1answer
61 views
How to sum to one column other column from a diffrent file?
I have something like this
file1:
element1,12
element2,10
element3,5
file2:
IN
element1,8
element3,5
and I want to modify file1 and obtain:
element1,20
element2,10
element3,10
The first line from ...
1
vote
1answer
54 views
Get next line with awk
I am trying to calculate integral from bash.
I have a file "data.log" which contains:
x1 y1
x2 y2
x3 y3
...
xn yn
I have created this file with awk.
Now I'd like to manipulate it to get:
x2 ...
-1
votes
3answers
76 views
Take the lines between two specific values from different columns in a txt file
I have a txt file (as show below) with many columns and I need to get the lines contained between the position of the second column to the position of the third column.
1 10385389 ...
-6
votes
1answer
51 views
find value and write it in 12,2 chars long format
example input:
:20:210301/9944072
:25:PL63213000342008099410720012
:28C:6032563
:60F:C2101010101PLN4138,23
:61:2101010101D13,61
:86:^101
^20Text ANY
^27YTRDTAL co. ltd.
^30
^31
find lines that start ...
0
votes
3answers
71 views
compare the word after char occurrence in each line
I have a file with many lines like below.
123)hfd)jggg)7584)abc
123)))7584)abc
)hfd)jggg)7584)cdf)))abc
)hfd)))abc
I have to output the lines with abc exactly after 4th occurrence of the ) [closing ...
2
votes
3answers
89 views
Grep a pattern in a line from log file and print next n lines until next patter
Suppose the file log.txt contains below content
[12] 03/31/21 08:33:30.080851 T(12581) _DBG message x 1
[12] 03/31/21 08:33:30.080851 T(34897) _DBG message y 1
[12] 03/31/21 08:33:31.241167 T(12344) ...
1
vote
3answers
113 views
Get two values in each row from an existing file
I am trying to make a new file from an existing file, input.txt, using Unix commands.
File input.txt
type ABC=home,3,test,data
type EFG=data,2,test_new,data_load
type PQR=load,2,test_type,data_loader
...
0
votes
2answers
72 views
Need to merge lines for fixed width file based on length
Each line starts with FEPS.
Merge the line which doesn't start with FEPS to previous line and then sort the file based on length i.e; keep the lines whose length is 23 chars
Input:
FEPS xxxx01 ...
0
votes
1answer
23 views
AWK - Select columns in a variable not input records
Assume that you have an AWK program file with a complex procedure: example.awk. You want to assign specific columns of a variable, rather than a whole line of the current record.
#!/usr/bin/awk -f
...
-4
votes
2answers
60 views
Editing text file via awk command
I am very newbie about on that command, so far I tried few things with that but I failed every time..
I have a text file (MWE) like below:
26 6 2020 09:06:23.300 277.430 50.131 ...
0
votes
4answers
52 views
awk hostname and ip address from log files when the column numbers are different
This is my sample file. Actual data has way more data in it.
wolf@linux:~$ more log*
::::::::::::::
log1.csv
::::::::::::::
hostname,sn,type,ip,random
abc,a11,zzz,192.168.168.168,rrr
def,e16,yyy,192....
-1
votes
2answers
91 views
I would like to get the names of entire headers present in another fasta file
I have a fasta file like that :
>TRINITY_DN100_c0_g1_i1 len=242 path=[0:0-241]
...
0
votes
1answer
46 views
return lines containing numbers in a range in a certain column but only if a column contains a certain number
Related to this question...
return lines containing numbers in a range in a certain column
return lines containing numbers in a range in a certain column
I know how to delete lines which contain ...
