Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [awk]

A pattern-directed scanning and processing language.

0
votes
0answers
16 views

Comparing all combinations of lines within two columns using awk + if statement

I have a file with 11 Columns of data deliminated by a space between each. For each line in column 2, I need to find the closest value in column 9. So below it would take the first entry in column 2 (...
0
votes
0answers
23 views

Howto find the size of a remote http/https directory (non recursively) using a given regexp from the command line

I'm trying to find the size of a remote http/https directory (non recursively) using a given regexp (\.mp4) from the command line, is there a tool to do so or do I need to parse the index.html ? ...
0
votes
2answers
38 views

Add three dot floating numbers in shell

I am trying to append 5.34.03 number to 5.34.04 by using the following command $ awk '{print 5.34.03 + 0.0.1}' 5.340.030.1 Expected output 5.34.13 I tried with many methods, eg. let, but it didn't ...
0
votes
1answer
28 views

comparison of two sorted files by common first field

I have to compare the IPs of two different /etc/hosts files and show a diff in side-by-side view. File hosts_sorted 10.244.74.152 REF_UP-4 10.244.74.154 REF_UP-5 10.244.74.156 REF_UP-6 10.244....
0
votes
2answers
38 views

Don't print the same string again if it was found between two files

I'm looking for the string SUCCESS into the file_1 & file_2 which in this case looks for the string and prints it based on whether its there on only file_1 or on both file_1 and file_2 while I'm ...
0
votes
2answers
38 views

adding contents with multiple files using awk, sed or grep

I have to files named file 1 and filed 2 following format: file 1 with 18000 lines COL1 COL2 COL3 COL4 COL5 - . ID = tom_fa_10005086 - 0 Parent = tom_fa_10005086 - 0 Parent = ...
0
votes
5answers
60 views

file formatting using awk

My input file contains data in the following format: 1503668542862176 manager=10001|Bounced=999|Analyst=10004|Business Analyst=10005|Programmer=10003 1552024948590636 manager=10001|Bounced=999|...
0
votes
3answers
30 views

Comparing two files

I have two files file1: 45 789 46 4546 78 4564 71 546 file2: 41 456 46 456 40 465 45 456 I would like to compare these files and get an output: if the number in first column from first file is ...
0
votes
2answers
23 views

Substring without removing field character

I have a string that I want to substring in awk (or cut, or sed, whatever). But I want to maintain the field separator. The file has a completely random amount of junk after the field separator, of ...
0
votes
1answer
27 views

sed/awk/python replace all spaces between characters after string match

I want to delete all spaces between characters after an initial match. An example is exmpl 0 t h i s a r e t he spaces to d e l e t e exmpl 1 m o r e spaces to d e l exmpl 2 o r s m t h completely d ...
0
votes
2answers
33 views

How to subtract every adjacent line from the 2nd column and keep the first one

I am creating a file via this command: awk '{print $2 " "$7" "$8}' REACTOME_EXTENSION_OF_TELOMERES.xls | awk '$8!="No" {print $1 " " $2}' | awk 'NR>1' | awk 'BEGIN { OFS=", "; print "Name" " " "...
0
votes
0answers
12 views

macOS High Sierra “log show ” ouput

The log show command output from a macOS High Sierra is detailed below: **Command 1** log show --info --predicate 'eventMessage contains "system boot:"' log: warning: ./system_logs.logarchive ...
0
votes
3answers
26 views

remove specific characters from a string

I need to replace spaces with comma and then remove certain extra specific chars from a string. echo "$d" >>Mon Apr 22 05:06:00 UTC 2019 jent=$(echo $jt1 | sed 's/[[:space:]]/,/g') echo "$jent" ...
1
vote
1answer
64 views

AWK fields filtering misses the first field

I'm writing AWK code to pick some fields from an input file, and trim columns with pre-defined fields to pick, etc. The input file looks like: stop_id,stop_code,stop_name,stop_lat,stop_lon,...
0
votes
4answers
100 views

sed command to replace a value in a config file

I'm working on an automation script and I need to change the value in a config file. The config file that I'm trying to edit has a line like this (including the spaces before): "peer-port": 23456,...
0
votes
5answers
58 views

How to add a string in column n and first line of a file?

I have a tab-separated file that looks like: $ cat file GCF_000014165.1_ASM1416v1.dist_nbr_anntn WP_011558474.1 1155234 1156286 polyketide synthase [Mycobacterium] GCF_000014165....
0
votes
4answers
78 views

Removing lines whose first column is duplicated

I have a file where strings are separated by a special string (not a comma or delimiter) e.g. <vvv>. I want to check if all strings in the first field are unique. If duplicate rows for the same ...
0
votes
1answer
26 views

compare a certain number of lines between columns of two files

I have a thousand files and I need to verify if they all have exactly the same information in the second column up to a certain number of lines. Below an example. I would like to print the names of ...
0
votes
3answers
61 views

Find and replace Pipe using awk command

I am trying to grep column 7th with pipe delimiter (|) data in that I have to removed pipe (|) from middle of text. Sample file: col1|col2|col3|col4|col5|col6|col7|col8|col9|col10 value1|value2|...
0
votes
0answers
20 views

Modifying text file using (g)awk [duplicate]

Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
0
votes
3answers
67 views

How to extract first line that matches pattern from a file and if pattern doesn't exist print fixed output?

I have a text file shown below: Doc_A 123 abc Doc_A 456 def Doc_A 789 ghi Doc_B 123 abc Doc_B 456 def Doc_C 123 abc Doc_C 456 def Doc_C 789 ghi Doc_C 101 jkl And a reference file Doc_A Doc_B Doc_C ...
0
votes
2answers
71 views

find a pair of words that appear the most of the times together

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e: chapter1: hello world good boy green sun ...
0
votes
1answer
31 views

Connecting files like columns

I have this command: for file in file??.txt; do awk -f vypis_z_dcout.awk "$file" > "$file".csv; done The result is a column of numbers from each file and I want to connect results from all files ...
-1
votes
2answers
68 views

find the word that appears the most at the beginning of a line from entire paragraph

I have a paragraph and I want to know which word appears the most at the beginning of a line from all paragraph for example: paragraph: Hello my name is X Nice to meet you Hello my name is Y so ...
-4
votes
0answers
26 views

Linux script to move item from one folder to another

I want to write a script to read a file which contains the below information.The script needs to process each line. Folder A goes to Folder AX condition=name starts with RED Folder B goes to Folder ...
1
vote
1answer
37 views

awk syntax error inside script while assigning it to a variable

Basically memory is a variable. In a script, I am trying to remote login to one server and execute free command to check memory usage and using awk I'm getting the usage in percentage. But I am ...
12
votes
5answers
3k views

How to compare two different files line by line in unix?

File1: 123 234 345 456 File2: 123 234 343 758 Expected output: File3: TRUE TRUE FALSE FALSE so the code should compare two files and print 'TRUE' if it matches otherwise it should print 'FALSE' ...
6
votes
8answers
742 views

Delete nth line (when counted from the bottom)

There are unknown number of lines in a file. How to delete nth line (when counted from the bottom) with one-liner command (you may use more than one if it is necessary) on Unix platform.
0
votes
3answers
35 views

Extracting file name with spaces using awk

I have this line of output: M file.txt Getting fed into this awk program: awk '{printf("\t%s: %s %s\n", FNR, $1, $2)}' which returns 1: M file.txt This works and is the usual case. However, if ...
1
vote
1answer
26 views

How to split file based on number of columns?

I have a file containing numeric data. Each line has a varying number of columns. I want to Split the file in to multiple files based on number of columns in a Line. Each Line may have columns varying ...
-1
votes
2answers
24 views

extract specific fields from a file using a character delimiter

I have a 1gb file and would like to get the third and fifth fields. The file is delimited by (,): input file: luis, jose, jesus atoche, mark, john 1,2,3,4,5 aa 1, bb 2, cc3, dd 4, ee5 output file: ...
0
votes
1answer
29 views

Get Lines between pattern 1 and pattern2 which may accure multiple times

I try to print out the lines between two different patterns(including the patterns) which may accure multiple times in a file. The second pattern may accure besides the startpattern-endpattern-rhytm. ...
0
votes
1answer
22 views

modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing

There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file. ...
0
votes
1answer
18 views

Ignoring first record in awk

BEGIN{FS=","; t1 = 0; n = 0; avg = 0; sum=0; printf("%-10s %9s\n%-10s %9s","Name", "Average", "----", "-------") } { if ($3 > 0){ t1 += $3; n++; } sum = 0; if($3 &...
2
votes
2answers
45 views

goto a specific line in awk to start processing [closed]

I want the awk to go to the beginning of the block of lines to start processing it again once it hit the bottom of that block and processed its last line. Basically, if block of text is <START ...
1
vote
1answer
30 views

Element comparison between subsequent lines, assigning a value for each null output

I have an output which looks like this: ASCII file I want to pick up the last element from the rows that contain the string "Fe\s2b". But the thing is each occurrence of "H\s\s1\s\s\s\s\s\s4861.33A" ...
2
votes
1answer
40 views

search for single or multiple occurrences of the samestring and edit next line

I have a file that sometimes has one occurrence or multiple occurrences of the same string. I am needing to search for one or more occurrences in the file and go to the next line and keep the values ...
-1
votes
1answer
27 views

Sort based on second column and list from highest to lowest

TGAAACATGCCTTTCACCTCCTGCCATGATTGTGAGGCCT 2 GTGTTGCCCAGGTTGGTTTCAAACTCAGCTCAAGTGATCT 1 GAGACTCCTTCAGGAGACTGGTCCCCTGTCCTCGCCTCTC 3 CATGTTCTCCTGGAAGCCATGTGAGGAAGGCACATCATGG 8 ...
2
votes
1answer
176 views

Regex in IF condition in awk

I have awk script file as below. I need to add another condition in the if statement to check if the string contains atleast one alphabet. How can I add the extra condition to the present if statement?...
0
votes
1answer
22 views

Allign file based on field values

I have a file like below with two fields ending with *.alarms.gz and *.values.gz, want to allign the files like all the alarms.gz in first field and all the values.gz in second field. How can i ...
1
vote
1answer
48 views

Bash - How to update what's inside the quotes, but only if it's a number?

For this project, we have hundreds of applications using composer.json. I'm using a Bash script to go through and update the version numbers of any file that contain an application that is getting a ...
2
votes
3answers
180 views

How to get single character after space?

How can I obtain below output? I want the first field as it is and a single character after space. echo "Hello world" Hellow If it also has a 3rd field than the beginning character of the 3rd field ...
0
votes
1answer
35 views

Renaming multiple paired files, deleting varying barcode character string in middle?

I have a bunch of paired files with unneeded barcode tags within the middle of the file, for example: LIB008983_TRA00020080_TAAGGCGA-TATCCTCT_L001_R1.fastq.gz LIB008983_TRA00020080_TAAGGCGA-...
0
votes
3answers
54 views

awk or grep? - print keyword and following columns

I did some benchmarks and the output is a single file containing multiple results with the following format [blaaa] 1st run T/V N NB P Q Time ========================== 2 3 4 1 2 1....
-1
votes
3answers
79 views

Extract files which have more than 5 lines, and keep same filename [closed]

I have multipe files containing errors in annual reports. Each file name have .mvt extension. It is not necessary that every file has error. I have found that when errors appear OR not there is always ...
0
votes
4answers
36 views

Match the column heading and print the values of the column using awk

How to print column free using awk or sed command? Depends on vmstat version column, free may change from column number 5 to something else. bash-3.00$ vmstat 1 1|tail -2 r b w swap free re mf pi po ...
1
vote
2answers
54 views

Combining two outputs using AWK

I have a bash command (on macOS High Sierra) as: sudo cat /private/var/log/system.log*| awk 'BEGIN {print "\n"} {print "Month\tDay\tTime\t\tSystemMessage"} /SHUTDOWN_TIME/{print $1,$2,$3,$8}'|sort -...
0
votes
0answers
42 views

How to rearrange a data?

I have Following type of data: each block starting with "Atom An X Y Z", I have 4 rows and 5 columns for each number point. I can arrange somehow for 3 columns and 2 rows with following code. awk '!(...
0
votes
2answers
42 views

Pick up successive lines containing keywords in order

I have a tab-separated file that looks as follows: $ cat file GCF_000015405.1_ASM1540v1.dist_nbr_anntn WP_011558474.1 1159543 1160595 -4330977 polyketide synthase [Mycobacterium] ...
0
votes
1answer
15 views

Invalid char error when running .awk file

I am appending a value 100 to every line in a file with the code below. This code works well when I run it in terminal but when I save the code in .awk file and run it I am getting an error. How to ...