Questions tagged [awk]
A pattern-directed scanning and processing language.
0
votes
0answers
16 views
Comparing all combinations of lines within two columns using awk + if statement
I have a file with 11 Columns of data deliminated by a space between each. For each line in column 2, I need to find the closest value in column 9. So below it would take the first entry in column 2 (...
0
votes
0answers
23 views
Howto find the size of a remote http/https directory (non recursively) using a given regexp from the command line
I'm trying to find the size of a remote http/https directory (non recursively) using a given regexp (\.mp4) from the command line, is there a tool to do so or do I need to parse the index.html ?
...
0
votes
2answers
38 views
Add three dot floating numbers in shell
I am trying to append 5.34.03 number to 5.34.04 by using the following command
$ awk '{print 5.34.03 + 0.0.1}'
5.340.030.1
Expected output
5.34.13
I tried with many methods, eg. let, but it didn't ...
0
votes
1answer
28 views
comparison of two sorted files by common first field
I have to compare the IPs of two different /etc/hosts files and show a diff in side-by-side view.
File hosts_sorted
10.244.74.152 REF_UP-4
10.244.74.154 REF_UP-5
10.244.74.156 REF_UP-6
10.244....
0
votes
2answers
38 views
Don't print the same string again if it was found between two files
I'm looking for the string SUCCESS into the file_1 & file_2 which in this case looks for the string and prints it based on whether its there on only file_1 or on both file_1 and file_2 while I'm ...
0
votes
2answers
38 views
adding contents with multiple files using awk, sed or grep
I have to files named file 1 and filed 2 following format:
file 1 with 18000 lines
COL1 COL2 COL3 COL4 COL5
- . ID = tom_fa_10005086
- 0 Parent = tom_fa_10005086
- 0 Parent = ...
0
votes
5answers
60 views
file formatting using awk
My input file contains data in the following format:
1503668542862176 manager=10001|Bounced=999|Analyst=10004|Business Analyst=10005|Programmer=10003
1552024948590636 manager=10001|Bounced=999|...
0
votes
3answers
30 views
Comparing two files
I have two files
file1:
45 789
46 4546
78 4564
71 546
file2:
41 456
46 456
40 465
45 456
I would like to compare these files and get an output: if the number in first column from first file is ...
0
votes
2answers
23 views
Substring without removing field character
I have a string that I want to substring in awk (or cut, or sed, whatever).
But I want to maintain the field separator.
The file has a completely random amount of junk after the field separator, of ...
0
votes
1answer
27 views
sed/awk/python replace all spaces between characters after string match
I want to delete all spaces between characters after an initial match. An example is
exmpl 0 t h i s a r e t he spaces to d e l e t e
exmpl 1 m o r e spaces to d e l
exmpl 2 o r s m t h completely d ...
0
votes
2answers
33 views
How to subtract every adjacent line from the 2nd column and keep the first one
I am creating a file via this command:
awk '{print $2 " "$7" "$8}' REACTOME_EXTENSION_OF_TELOMERES.xls | awk '$8!="No" {print $1 " " $2}' | awk 'NR>1' | awk 'BEGIN { OFS=", "; print "Name" " " "...
0
votes
0answers
12 views
macOS High Sierra “log show ” ouput
The log show command output from a macOS High Sierra is detailed below:
**Command 1**
log show --info --predicate 'eventMessage contains "system boot:"'
log: warning: ./system_logs.logarchive ...
0
votes
3answers
26 views
remove specific characters from a string
I need to replace spaces with comma and then remove certain extra specific chars from a string.
echo "$d"
>>Mon Apr 22 05:06:00 UTC 2019
jent=$(echo $jt1 | sed 's/[[:space:]]/,/g')
echo "$jent"
...
1
vote
1answer
64 views
AWK fields filtering misses the first field
I'm writing AWK code to pick some fields from an input file, and trim columns with pre-defined fields to pick, etc.
The input file looks like:
stop_id,stop_code,stop_name,stop_lat,stop_lon,...
0
votes
4answers
100 views
sed command to replace a value in a config file
I'm working on an automation script and I need to change the value in a config file. The config file that I'm trying to edit has a line like this (including the spaces before):
"peer-port": 23456,...
0
votes
5answers
58 views
How to add a string in column n and first line of a file?
I have a tab-separated file that looks like:
$ cat file
GCF_000014165.1_ASM1416v1.dist_nbr_anntn WP_011558474.1 1155234 1156286 polyketide synthase [Mycobacterium]
GCF_000014165....
0
votes
4answers
78 views
Removing lines whose first column is duplicated
I have a file where strings are separated by a special string (not a comma or delimiter) e.g. <vvv>. I want to check if all strings in the first field are unique. If duplicate rows for the same ...
0
votes
1answer
26 views
compare a certain number of lines between columns of two files
I have a thousand files and I need to verify if they all have exactly the same information in the second column up to a certain number of lines. Below an example. I would like to print the names of ...
0
votes
3answers
61 views
Find and replace Pipe using awk command
I am trying to grep column 7th with pipe delimiter (|) data in that I have to removed pipe (|) from middle of text.
Sample file:
col1|col2|col3|col4|col5|col6|col7|col8|col9|col10
value1|value2|...
0
votes
0answers
20 views
Modifying text file using (g)awk [duplicate]
Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
0
votes
3answers
67 views
How to extract first line that matches pattern from a file and if pattern doesn't exist print fixed output?
I have a text file shown below:
Doc_A 123 abc
Doc_A 456 def
Doc_A 789 ghi
Doc_B 123 abc
Doc_B 456 def
Doc_C 123 abc
Doc_C 456 def
Doc_C 789 ghi
Doc_C 101 jkl
And a reference file
Doc_A
Doc_B
Doc_C
...
0
votes
2answers
71 views
find a pair of words that appear the most of the times together
I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:
chapter1:
hello world good boy green sun
...
0
votes
1answer
31 views
Connecting files like columns
I have this command:
for file in file??.txt; do awk -f vypis_z_dcout.awk "$file" > "$file".csv; done
The result is a column of numbers from each file and I want to connect results from all files ...
-1
votes
2answers
68 views
find the word that appears the most at the beginning of a line from entire paragraph
I have a paragraph and I want to know which word appears the most at the beginning of a line from all paragraph
for example:
paragraph:
Hello my name is X
Nice to meet you
Hello my name is Y
so ...
-4
votes
0answers
26 views
Linux script to move item from one folder to another
I want to write a script to read a file which contains the below information.The script needs to process each line.
Folder A goes to Folder AX condition=name starts with RED
Folder B goes to Folder ...
1
vote
1answer
37 views
awk syntax error inside script while assigning it to a variable
Basically memory is a variable. In a script, I am trying to remote login to one server and execute free command to check memory usage and using awk I'm getting the usage in percentage. But I am ...
12
votes
5answers
3k views
How to compare two different files line by line in unix?
File1:
123
234
345
456
File2:
123
234
343
758
Expected output:
File3:
TRUE
TRUE
FALSE
FALSE
so the code should compare two files and print 'TRUE' if it matches otherwise it should print 'FALSE' ...
6
votes
8answers
742 views
Delete nth line (when counted from the bottom)
There are unknown number of lines in a file. How to delete nth line (when counted from the bottom) with one-liner command (you may use more than one if it is necessary) on Unix platform.
0
votes
3answers
35 views
Extracting file name with spaces using awk
I have this line of output:
M file.txt
Getting fed into this awk program:
awk '{printf("\t%s: %s %s\n", FNR, $1, $2)}'
which returns
1: M file.txt
This works and is the usual case. However, if ...
1
vote
1answer
26 views
How to split file based on number of columns?
I have a file containing numeric data. Each line has a varying number of columns. I want to Split the file in to multiple files based on number of columns in a Line. Each Line may have columns varying ...
-1
votes
2answers
24 views
extract specific fields from a file using a character delimiter
I have a 1gb file and would like to get the third and fifth fields.
The file is delimited by (,):
input file:
luis, jose, jesus atoche, mark, john
1,2,3,4,5
aa 1, bb 2, cc3, dd 4, ee5
output file:
...
0
votes
1answer
29 views
Get Lines between pattern 1 and pattern2 which may accure multiple times
I try to print out the lines between two different patterns(including the patterns) which may accure multiple times in a file. The second pattern may accure besides the startpattern-endpattern-rhytm.
...
0
votes
1answer
22 views
modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing
There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file.
...
0
votes
1answer
18 views
Ignoring first record in awk
BEGIN{FS=","; t1 = 0; n = 0; avg = 0; sum=0; printf("%-10s %9s\n%-10s %9s","Name", "Average", "----", "-------") }
{
if ($3 > 0){
t1 += $3;
n++;
}
sum = 0;
if($3 &...
2
votes
2answers
45 views
goto a specific line in awk to start processing [closed]
I want the awk to go to the beginning of the block of lines to start processing it again once it hit the bottom of that block and processed its last line.
Basically, if block of text is
<START ...
1
vote
1answer
30 views
Element comparison between subsequent lines, assigning a value for each null output
I have an output which looks like this:
ASCII file
I want to pick up the last element from the rows that contain the string "Fe\s2b". But the thing is each occurrence of "H\s\s1\s\s\s\s\s\s4861.33A" ...
2
votes
1answer
40 views
search for single or multiple occurrences of the samestring and edit next line
I have a file that sometimes has one occurrence or multiple occurrences of the same string. I am needing to search for one or more occurrences in the file and go to the next line and keep the values ...
-1
votes
1answer
27 views
Sort based on second column and list from highest to lowest
TGAAACATGCCTTTCACCTCCTGCCATGATTGTGAGGCCT 2
GTGTTGCCCAGGTTGGTTTCAAACTCAGCTCAAGTGATCT 1
GAGACTCCTTCAGGAGACTGGTCCCCTGTCCTCGCCTCTC 3
CATGTTCTCCTGGAAGCCATGTGAGGAAGGCACATCATGG 8
...
2
votes
1answer
176 views
Regex in IF condition in awk
I have awk script file as below. I need to add another condition in the if statement to check if the string contains atleast one alphabet. How can I add the extra condition to the present if statement?...
0
votes
1answer
22 views
Allign file based on field values
I have a file like below with two fields ending with *.alarms.gz and *.values.gz, want to allign the files like all the alarms.gz in first field and all the values.gz in second field. How can i ...
1
vote
1answer
48 views
Bash - How to update what's inside the quotes, but only if it's a number?
For this project, we have hundreds of applications using composer.json. I'm using a Bash script to go through and update the version numbers of any file that contain an application that is getting a ...
2
votes
3answers
180 views
How to get single character after space?
How can I obtain below output? I want the first field as it is and a single character after space.
echo "Hello world"
Hellow
If it also has a 3rd field than the beginning character of the 3rd field ...
0
votes
1answer
35 views
Renaming multiple paired files, deleting varying barcode character string in middle?
I have a bunch of paired files with unneeded barcode tags within the middle of the file, for example:
LIB008983_TRA00020080_TAAGGCGA-TATCCTCT_L001_R1.fastq.gz
LIB008983_TRA00020080_TAAGGCGA-...
0
votes
3answers
54 views
awk or grep? - print keyword and following columns
I did some benchmarks and the output is a single file containing multiple results with the following format
[blaaa]
1st run
T/V N NB P Q Time
==========================
2 3 4 1 2 1....
-1
votes
3answers
79 views
Extract files which have more than 5 lines, and keep same filename [closed]
I have multipe files containing errors in annual reports. Each file name have .mvt extension. It is not necessary that every file has error. I have found that when errors appear OR not there is always ...
0
votes
4answers
36 views
Match the column heading and print the values of the column using awk
How to print column free using awk or sed command?
Depends on vmstat version column, free may change from column number 5 to something else.
bash-3.00$ vmstat 1 1|tail -2
r b w swap free re mf pi po ...
1
vote
2answers
54 views
Combining two outputs using AWK
I have a bash command (on macOS High Sierra) as:
sudo cat /private/var/log/system.log*| awk 'BEGIN {print "\n"} {print "Month\tDay\tTime\t\tSystemMessage"} /SHUTDOWN_TIME/{print $1,$2,$3,$8}'|sort -...
0
votes
0answers
42 views
How to rearrange a data?
I have Following type of data: each block starting with "Atom An X Y Z", I have 4 rows and 5 columns for each number point. I can arrange somehow for 3 columns and 2 rows with following code. awk '!(...
0
votes
2answers
42 views
Pick up successive lines containing keywords in order
I have a tab-separated file that looks as follows:
$ cat file
GCF_000015405.1_ASM1540v1.dist_nbr_anntn WP_011558474.1 1159543 1160595 -4330977 polyketide synthase [Mycobacterium]
...
0
votes
1answer
15 views
Invalid char error when running .awk file
I am appending a value 100 to every line in a file with the code below. This code works well when I run it in terminal but when I save the code in .awk file and run it I am getting an error. How to ...
