Questions tagged [awk]
A pattern-directed scanning and processing language.
0
votes
1answer
28 views
adding contents with multiple files using awk, sed or grep
I have to files named file 1 and filed 2 following format:
file 1 with 18000 lines
COL1 COL2 COL3 COL4 COL5
- . ID = tom_fa_10005086
- 0 Parent = tom_fa_10005086
- 0 Parent = ...
0
votes
2answers
38 views
file formating using awk
My input file contain data as below format:
1503668542862176 manager=10001|Bounced=999|Analyst=10004|Business Analyst=10005|Programmer=10003
1552024948590636 manager=10001|Bounced=999|Analyst=...
0
votes
2answers
24 views
Comparing two files
I have two files
file1:
45 789
46 4546
78 4564
71 546
file2:
41 456
46 456
40 465
45 456
I would like to compare these files and get an output: if the number in first column from first file is ...
0
votes
2answers
23 views
Substring without removing field character
I have a string that I want to substring in awk (or cut, or sed, whatever).
But I want to maintain the field separator.
The file has a completely random amount of junk after the field separator, of ...
0
votes
1answer
27 views
sed/awk/python replace all spaces between characters after string match
I want to delete all spaces between characters after an initial match. An example is
exmpl 0 t h i s a r e t he spaces to d e l e t e
exmpl 1 m o r e spaces to d e l
exmpl 2 o r s m t h completely d ...
0
votes
2answers
32 views
How to subtract every adjacent line from the 2nd column and keep the first one
I am creating a file via this command:
awk '{print $2 " "$7" "$8}' REACTOME_EXTENSION_OF_TELOMERES.xls | awk '$8!="No" {print $1 " " $2}' | awk 'NR>1' | awk 'BEGIN { OFS=", "; print "Name" " " "...
0
votes
0answers
11 views
macOS High Sierra “log show ” ouput
The log show command output from a macOS High Sierra is detailed below:
**Command 1**
log show --info --predicate 'eventMessage contains "system boot:"'
log: warning: ./system_logs.logarchive ...
0
votes
3answers
18 views
remove specific characters from a string
I have a scenario where I need to replace spaces with comma and then remove certain extra specific chars from a string.
echo "$d"
>>Mon Apr 22 05:06:00 UTC 2019
jent=$(echo $jt1 | sed 's/[[:...
1
vote
1answer
62 views
AWK fields filtering misses the first field
I'm writing AWK code to pick some fields from an input file, and trim columns with pre-defined fields to pick, etc.
The input file looks like:
stop_id,stop_code,stop_name,stop_lat,stop_lon,...
0
votes
4answers
99 views
sed command to replace a value in a config file
I'm working on an automation script and I need to change the value in a config file. The config file that I'm trying to edit has a line like this (including the spaces before):
"peer-port": 23456,...
0
votes
5answers
58 views
How to add a string in column n and first line of a file?
I have a tab-separated file that looks like:
$ cat file
GCF_000014165.1_ASM1416v1.dist_nbr_anntn WP_011558474.1 1155234 1156286 polyketide synthase [Mycobacterium]
GCF_000014165....
0
votes
4answers
78 views
Removing lines whose first column is duplicated
I have a file where strings are separated by a special string (not a comma or delimiter) e.g. <vvv>. I want to check if all strings in the first field are unique. If duplicate rows for the same ...
0
votes
1answer
25 views
compare a certain number of lines between columns of two files
I have a thousand files and I need to verify if they all have exactly the same information in the second column up to a certain number of lines. Below an example. I would like to print the names of ...
0
votes
3answers
61 views
Find and replace Pipe using awk command
I am trying to grep column 7th with pipe delimiter (|) data in that I have to removed pipe (|) from middle of text.
Sample file:
col1|col2|col3|col4|col5|col6|col7|col8|col9|col10
value1|value2|...
0
votes
0answers
20 views
Modifying text file using (g)awk [duplicate]
Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
0
votes
3answers
67 views
How to extract first line that matches pattern from a file and if pattern doesn't exist print fixed output?
I have a text file shown below:
Doc_A 123 abc
Doc_A 456 def
Doc_A 789 ghi
Doc_B 123 abc
Doc_B 456 def
Doc_C 123 abc
Doc_C 456 def
Doc_C 789 ghi
Doc_C 101 jkl
And a reference file
Doc_A
Doc_B
Doc_C
...
1
vote
2answers
70 views
find a pair of words that appear the most of the times together
I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e:
chapter1:
hello world good boy green sun
...
0
votes
1answer
31 views
Connecting files like columns
I have this command:
for file in file??.txt; do awk -f vypis_z_dcout.awk "$file" > "$file".csv; done
The result is a column of numbers from each file and I want to connect results from all files ...
-1
votes
2answers
65 views
find the word that appears the most at the beginning of a line from entire paragraph
I have a paragraph and I want to know which word appears the most at the beginning of a line from all paragraph
for example:
paragraph:
Hello my name is X
Nice to meet you
Hello my name is Y
so ...
-4
votes
0answers
25 views
Linux script to move item from one folder to another
I want to write a script to read a file which contains the below information.The script needs to process each line.
Folder A goes to Folder AX condition=name starts with RED
Folder B goes to Folder ...
1
vote
1answer
37 views
awk syntax error inside script while assigning it to a variable
Basically memory is a variable. In a script, I am trying to remote login to one server and execute free command to check memory usage and using awk I'm getting the usage in percentage. But I am ...
12
votes
5answers
3k views
How to compare two different files line by line in unix?
File1:
123
234
345
456
File2:
123
234
343
758
Expected output:
File3:
TRUE
TRUE
FALSE
FALSE
so the code should compare two files and print 'TRUE' if it matches otherwise it should print 'FALSE' ...
6
votes
8answers
739 views
Delete nth line (when counted from the bottom)
There are unknown number of lines in a file. How to delete nth line (when counted from the bottom) with one-liner command (you may use more than one if it is necessary) on Unix platform.
0
votes
3answers
35 views
Extracting file name with spaces using awk
I have this line of output:
M file.txt
Getting fed into this awk program:
awk '{printf("\t%s: %s %s\n", FNR, $1, $2)}'
which returns
1: M file.txt
This works and is the usual case. However, if ...
1
vote
1answer
26 views
How to split file based on number of columns?
I have a file containing numeric data. Each line has a varying number of columns. I want to Split the file in to multiple files based on number of columns in a Line. Each Line may have columns varying ...
-1
votes
2answers
24 views
extract specific fields from a file using a character delimiter
I have a 1gb file and would like to get the third and fifth fields.
The file is delimited by (,):
input file:
luis, jose, jesus atoche, mark, john
1,2,3,4,5
aa 1, bb 2, cc3, dd 4, ee5
output file:
...
0
votes
1answer
29 views
Get Lines between pattern 1 and pattern2 which may accure multiple times
I try to print out the lines between two different patterns(including the patterns) which may accure multiple times in a file. The second pattern may accure besides the startpattern-endpattern-rhytm.
...
0
votes
1answer
22 views
modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing
There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file.
...
0
votes
1answer
17 views
Ignoring first record in awk
BEGIN{FS=","; t1 = 0; n = 0; avg = 0; sum=0; printf("%-10s %9s\n%-10s %9s","Name", "Average", "----", "-------") }
{
if ($3 > 0){
t1 += $3;
n++;
}
sum = 0;
if($3 &...
2
votes
2answers
45 views
goto a specific line in awk to start processing [closed]
I want the awk to go to the beginning of the block of lines to start processing it again once it hit the bottom of that block and processed its last line.
Basically, if block of text is
<START ...
1
vote
1answer
30 views
Element comparison between subsequent lines, assigning a value for each null output
I have an output which looks like this:
ASCII file
I want to pick up the last element from the rows that contain the string "Fe\s2b". But the thing is each occurrence of "H\s\s1\s\s\s\s\s\s4861.33A" ...
2
votes
1answer
40 views
search for single or multiple occurrences of the samestring and edit next line
I have a file that sometimes has one occurrence or multiple occurrences of the same string. I am needing to search for one or more occurrences in the file and go to the next line and keep the values ...
-1
votes
1answer
26 views
Sort based on second column and list from highest to lowest
TGAAACATGCCTTTCACCTCCTGCCATGATTGTGAGGCCT 2
GTGTTGCCCAGGTTGGTTTCAAACTCAGCTCAAGTGATCT 1
GAGACTCCTTCAGGAGACTGGTCCCCTGTCCTCGCCTCTC 3
CATGTTCTCCTGGAAGCCATGTGAGGAAGGCACATCATGG 8
...
2
votes
1answer
175 views
Regex in IF condition in awk
I have awk script file as below. I need to add another condition in the if statement to check if the string contains atleast one alphabet. How can I add the extra condition to the present if statement?...
0
votes
1answer
22 views
Allign file based on field values
I have a file like below with two fields ending with *.alarms.gz and *.values.gz, want to allign the files like all the alarms.gz in first field and all the values.gz in second field. How can i ...
1
vote
1answer
48 views
Bash - How to update what's inside the quotes, but only if it's a number?
For this project, we have hundreds of applications using composer.json. I'm using a Bash script to go through and update the version numbers of any file that contain an application that is getting a ...
2
votes
3answers
180 views
How to get single character after space?
How can I obtain below output? I want the first field as it is and a single character after space.
echo "Hello world"
Hellow
If it also has a 3rd field than the beginning character of the 3rd field ...
0
votes
1answer
35 views
Renaming multiple paired files, deleting varying barcode character string in middle?
I have a bunch of paired files with unneeded barcode tags within the middle of the file, for example:
LIB008983_TRA00020080_TAAGGCGA-TATCCTCT_L001_R1.fastq.gz
LIB008983_TRA00020080_TAAGGCGA-...
0
votes
3answers
54 views
awk or grep? - print keyword and following columns
I did some benchmarks and the output is a single file containing multiple results with the following format
[blaaa]
1st run
T/V N NB P Q Time
==========================
2 3 4 1 2 1....
-1
votes
3answers
79 views
Extract files which have more than 5 lines, and keep same filename [closed]
I have multipe files containing errors in annual reports. Each file name have .mvt extension. It is not necessary that every file has error. I have found that when errors appear OR not there is always ...
0
votes
4answers
36 views
Match the column heading and print the values of the column using awk
How to print column free using awk or sed command?
Depends on vmstat version column, free may change from column number 5 to something else.
bash-3.00$ vmstat 1 1|tail -2
r b w swap free re mf pi po ...
1
vote
2answers
54 views
Combining two outputs using AWK
I have a bash command (on macOS High Sierra) as:
sudo cat /private/var/log/system.log*| awk 'BEGIN {print "\n"} {print "Month\tDay\tTime\t\tSystemMessage"} /SHUTDOWN_TIME/{print $1,$2,$3,$8}'|sort -...
0
votes
0answers
39 views
How to rearrange a data?
I have Following type of data: each block starting with "Atom An X Y Z", I have 4 rows and 5 columns for each number point. I can arrange somehow for 3 columns and 2 rows with following code. awk '!(...
0
votes
2answers
42 views
Pick up successive lines containing keywords in order
I have a tab-separated file that looks as follows:
$ cat file
GCF_000015405.1_ASM1540v1.dist_nbr_anntn WP_011558474.1 1159543 1160595 -4330977 polyketide synthase [Mycobacterium]
...
0
votes
1answer
15 views
Invalid char error when running .awk file
I am appending a value 100 to every line in a file with the code below. This code works well when I run it in terminal but when I save the code in .awk file and run it I am getting an error. How to ...
0
votes
2answers
47 views
Extracting lines to new files
Say I have a large CSV file with a header and several columns. For the purpose of this question I will consider a small file with just two columns. We can call it use_rep.
user_id,rep
885,500K+
22565,...
1
vote
5answers
59 views
Question about text formatting using sed of awk
I want to format text using awk or sed.
I have text looking like,
H1 (dxfault)
H337 (Unxr)
H1106 (hast)
H1129 (h1129)
H1370 (hlan1370)
I want to remove the letter H and everything after the first ...
1
vote
2answers
54 views
Remove duplicates from particular columns
I have a file in below format, where columns are separated with comma.
[1], Value1, UAC, AB
[2.2], Check1, BOH D2A D2A BOH, SD
[63], name2, MFB MFB, k
...
I want ...
1
vote
3answers
68 views
How to compare two CSV files and display unique records?
I have two csv file as below, i want to compare file1 with file2 and if records present in file2 that complete row should remove from file1. field to compare here is ID and in original file its at ...
0
votes
0answers
18 views
Delete the second line which matches a pattern keeping the first line if it starts with DA [closed]
I have a file which has following lines
Input
DA0183930283940484088X802004830202**0102937**60
DU0235363283940484088X802004830202**0102937**20
-------------------------------------------
------------...
