Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [awk]

A pattern-directed scanning and processing language.

0
votes
0answers
9 views

sed command to replace a value in a config file

I'm working on an automation script and I need to change the value in a config file. The config file that I'm trying to edit has multiple lines one of which is this (including the spaces before): ...
0
votes
3answers
27 views

How to add a string in column n and first line of a file?

I have a tab-separated file that looks like: $ cat file GCF_000014165.1_ASM1416v1.dist_nbr_anntn WP_011558474.1 1155234 1156286 polyketide synthase [Mycobacterium] GCF_000014165....
0
votes
5answers
1k views

Sum all file sizes

I'm new to bash and I need to do a little script to sum all file sizes, excluding subdirectories. My first idea was to keep the columns when you do ls -l. I cannot use grep, du or other advanced ...
0
votes
3answers
58 views

Removing lines whose first column is duplicated

I have a file where strings are separated by a special string (not a comma or delimiter) e.g. <vvv>. I want to check if all strings in the first field are unique. If duplicate rows for the same ...
0
votes
2answers
717 views

Convert two rows to two columns

I would like to convert every two rows to two columns using awk. input.txt: # Query: gi|11465907|ref|NC_001872.1| Chlamydomonas eugametos genome, complete genome # 0 hits found # Query: gi|11465922|...
1
vote
1answer
2k views

how concatenate two commands?

I need to concatenate two commands on shell, I'd have the path result and concatenate with the number of column in the first field. EX: #this gives me the path of my path directory like this: `/apps/...
3
votes
2answers
59 views

Iterating fieldA in CSV file, where fieldB has specified value

I've been trying to setup a script to run on gitbash, to process a CSV file so each record that has a specific value, the last field is changed from empty ("") to a value that iterates from 1 to 16. ...
4
votes
2answers
2k views

AWK for filling up rest of the columns with &nbsp in file

I'm trying to get AWK filling up following text file for "empty" columns". Basic idea of finding longest line with AWK NF and checking which line has no value on that column. Then adding x times N/A ...
3
votes
2answers
374 views

Understanding an awk formula that unwraps fasta files

I have just found a formula which can be used to unwrap fasta files. Before I give the formula, I need to explain what unwrapping a fasta file is. In short, the fasta format is like this: >...
0
votes
3answers
139 views

Formatting output from awk command in script

If I run this command: awk -F'[="]+' '/^(NAME|VERSION)=/{printf("%-17s: %s\n",$1,$2)}' /etc/os-release from a terminal, I can retrieve this: NAME : Debian GNU/Linux VERSION : 8 ...
3
votes
1answer
71 views

understanding a part of an awk program

A friend of mine has written a code in unix awk '\ BEGIN { OFS = "\t"; FS = "\t"} ;\ (NR >= 2){\ split($3,a,"|");\ if(a[1]!~ /\?/) {\   print a[1], a[2] > "geneinfo_tumor.txt";\   printf "%s", ...
0
votes
1answer
28 views

Connecting files like columns

I have this command: for file in file??.txt; do awk -f vypis_z_dcout.awk "$file" > "$file".csv; done The result is a column of numbers from each file and I want to connect results from all files ...
0
votes
1answer
943 views

Writing awk to extract cpu utilization from sar command

I am trying to extract the values of the %user,%nice etc from the output of the sar command. sar -P ALL 1 1 Output of this: Linux 2.6.32-358.el6.x86_64 (ftizsldapp009.ftiz.cummins.com) 09/28/...
0
votes
1answer
24 views

compare a certain number of lines between columns of two files

I have a thousand files and I need to verify if they all have exactly the same information in the second column up to a certain number of lines. Below an example. I would like to print the names of ...
2
votes
2answers
44 views

goto a specific line in awk to start processing [on hold]

I want the awk to go to the beginning of the block of lines to start processing it again once it hit the bottom of that block and processed its last line. Basically, if block of text is <START ...
0
votes
2answers
45 views

Find and replace Pipe using awk command

I am trying to grep column 7th with pipe delimiter (|) data in that I have to removed pipe (|) from middle of text. Sample file: col1|col2|col3|col4|col5|col6|col7|col8|col9|col10 value1|value2|...
8
votes
2answers
493 views

How to permanently change a file using awk? (“in-place” edits, as with “sed -i”)

I have an awk file new.awk BEGIN { FS=OFS="," } NR==1 { for (i=1; i<=NF; i++) { f[$i] = i } } NR > 1 { begSecs= mktime(gensub(/[":-]/," ","g",$(f["DateTime"]))) endSecs = begSecs +...
5
votes
8answers
694 views

Delete nth line (when counted from the bottom)

There are unknown number of lines in a file. How to delete nth line (when counted from the bottom) with one-liner command (you may use more than one if it is necessary) on Unix platform.
3
votes
1answer
8k views

awk - gensub function

I need to insert contents of variable to parameter function gensub in awk, specifically instead of regular expression's parameter. I have: gawk ' BEGIN { a = "abc15d56ef"; b = gensub(/.*([0-...
0
votes
0answers
19 views

Modifying text file using (g)awk [duplicate]

Is it possible to make a modification to a text file for example and save those modification to the same file using (g)awk script/command? All I see is only a modification to the printed output and ...
12
votes
6answers
3k views

How to compare two different files line by line in unix?

File1: 123 234 345 456 File2: 123 234 343 758 Expected output: File3: TRUE TRUE FALSE FALSE so the code should compare two files and print 'TRUE' if it matches otherwise it should print 'FALSE' ...
3
votes
2answers
446 views

Looking for duplicate instances of a tag in a file

Mutliple snippets of code exist in a file similar to the following: <blah>Spread the peanut butter <ramout assot="f0123_fun10" bapel="2 or 6"/> on good looking bread <ramout assot="...
0
votes
3answers
66 views

How to extract first line that matches pattern from a file and if pattern doesn't exist print fixed output?

I have a text file shown below: Doc_A 123 abc Doc_A 456 def Doc_A 789 ghi Doc_B 123 abc Doc_B 456 def Doc_C 123 abc Doc_C 456 def Doc_C 789 ghi Doc_C 101 jkl And a reference file Doc_A Doc_B Doc_C ...
-4
votes
0answers
24 views

Linux script to move item from one folder to another

I want to write a script to read a file which contains the below information.The script needs to process each line. Folder A goes to Folder AX condition=name starts with RED Folder B goes to Folder ...
1
vote
2answers
65 views

find a pair of words that appear the most of the times together

I have 10 text files, in each file i have a chapter from a book, i want to find the pair of words that appear the most of the time together in a line i.e: chapter1: hello world good boy green sun ...
-1
votes
1answer
31 views

find the word that appears the most at the beginning of a line from entire paragraph

I have a paragraph and I want to know which word appears the most at the beginning of a line from all paragraph for example: paragraph: Hello my name is X Nice to meet you Hello my name is Y so ...
1
vote
1answer
35 views

awk syntax error inside script while assigning it to a variable

Basically memory is a variable. In a script, I am trying to remote login to one server and execute free command to check memory usage and using awk I'm getting the usage in percentage. But I am ...
1
vote
3answers
2k views

Check if the particular string is present in a output of shell script using shell command

I have a shell script (test1.sh) which returns the following output Employee ID emp Type return type Admin User us321000034006755 ITdept access Itadminuser I wanted to ...
1
vote
1answer
30 views

Element comparison between subsequent lines, assigning a value for each null output

I have an output which looks like this: ASCII file I want to pick up the last element from the rows that contain the string "Fe\s2b". But the thing is each occurrence of "H\s\s1\s\s\s\s\s\s4861.33A" ...
6
votes
5answers
790 views

Using awk to identify the number identical columns

I have a large number of individual files that contain six columns each (number of rows can vary). As a simple example: 1 0 0 0 0 0 0 1 1 1 0 0 I am trying to identify how many ...
0
votes
3answers
34 views

Extracting file name with spaces using awk

I have this line of output: M file.txt Getting fed into this awk program: awk '{printf("\t%s: %s %s\n", FNR, $1, $2)}' which returns 1: M file.txt This works and is the usual case. However, if ...
2
votes
2answers
1k views

add an incrementing identity column to a file

Two questions: 1) Is there a way to create a new column (i.e. ID) to an existing file containing data and the new column ID populate as an auto incrementing number? for example: (Current file has ...
-1
votes
3answers
50 views

AWK Grouping Based On Line Containing Specific Character

The source looks like this: test/snmp/result/08-28-2016_12-30/AAAAA/hostname01_192.168.1.1 161/udp open snmp | snmp-brute: | comstring1 - Valid credentials |_ comstring2 - Valid credentials test/...
1
vote
1answer
26 views

How to split file based on number of columns?

I have a file containing numeric data. Each line has a varying number of columns. I want to Split the file in to multiple files based on number of columns in a Line. Each Line may have columns varying ...
12
votes
4answers
11k views

Concatenate lines by first column by awk or sed

How can I use awk in the following situation? I want to concatenate lines that start with the same column. Only the first column is kept after the join (in this case aaa, www, hhh). The file may be ...
0
votes
2answers
840 views

Awk one-liner to print column of SQL table?

Suppose I have table like: This is first line name minimum maximum config_value ----------------------------------- ----------- ----------- ------------ some ...
19
votes
5answers
16k views

How to select first occurrence between two patterns including them

How can I select first occurrence between two patterns including them. Preferably using sed or awk. I have: text something P1 something content1 content2 something P2 something text something P1 ...
-1
votes
2answers
24 views

extract specific fields from a file using a character delimiter

I have a 1gb file and would like to get the third and fifth fields. The file is delimited by (,): input file: luis, jose, jesus atoche, mark, john 1,2,3,4,5 aa 1, bb 2, cc3, dd 4, ee5 output file: ...
0
votes
1answer
27 views

Get Lines between pattern 1 and pattern2 which may accure multiple times

I try to print out the lines between two different patterns(including the patterns) which may accure multiple times in a file. The second pattern may accure besides the startpattern-endpattern-rhytm. ...
0
votes
1answer
22 views

modify between (only) 1st occurrence of patterns (inclusive of patterns) and keep processing

There are several questions about matching between only the first occurrence of two patterns (e.g., this) — but they all seem to rely on exit, which is no good, as I want to keep processing the file. ...
5
votes
2answers
8k views

use awk to replace parentheses?

How can I use awk to replace the parentheses in these 2 scenarios? Change empty parens to the word "None" 192.168.0.24,Up,() Remove parens to only host.domain.com is left 192.168.0.25,Up,(host....
1
vote
2answers
631 views

Extracting text blocks based on grep output

I'm currently working with a text file that contains the following text blocks: -------------------------------------- Beginning of block Text Random Text keywordA Text End of block ------------------...
1
vote
2answers
580 views

Print command output in separate lines

ip a | egrep '([0-9]{1,3}\.){3}[0-9]{1,3}|enp0s' | grep -e 10 -e 172 -e 192 | awk '{print $2}' | awk '{gsub("/24", "");print}' | xargs The above command gives the output as: enp0s3: 10.0.2.15 enp0s8:...
3
votes
3answers
2k views

How to truncate only given column length?

INPUT: a@notebook:~$ cat in.csv 'XYZ843141'^'ASDFSAFXYVFSHGDSDg sdGDS dsGDSgfa assfd faSDFAS saDFSAFD adFSA343fa sdfSADF'^'BAAAR'^'YYY'^'..... and so on, further columns 'YYZ814384'^'...
6
votes
4answers
881 views

extract lines according to a particular column

I have a CSV file like CK,ck XYZ,xyz,xyzs ABC,abc,abcs PQR,pqr,pqrs LMN,lmn, IJK,ijk, I have to check if something is written in column 3 except whitespace, then display the whole line.the output ...
0
votes
1answer
17 views

Ignoring first record in awk

BEGIN{FS=","; t1 = 0; n = 0; avg = 0; sum=0; printf("%-10s %9s\n%-10s %9s","Name", "Average", "----", "-------") } { if ($3 > 0){ t1 += $3; n++; } sum = 0; if($3 &...
2
votes
1answer
40 views

search for single or multiple occurrences of the samestring and edit next line

I have a file that sometimes has one occurrence or multiple occurrences of the same string. I am needing to search for one or more occurrences in the file and go to the next line and keep the values ...
2
votes
6answers
4k views

How to get the second column from command output?

My command's output is something like: N O D E O U T P U T THE FOLLOWING TABLE IS PRINTED FOR NODES BELONGING TO NODE SET NSET_OUTPUT NODE FOOT- NT11 ...
0
votes
1answer
21 views

Allign file based on field values

I have a file like below with two fields ending with *.alarms.gz and *.values.gz, want to allign the files like all the alarms.gz in first field and all the values.gz in second field. How can i ...
2
votes
3answers
304 views

AWK & Bash: Checking that a User's Home is Owned by the User

I'm trying to cycle through /etc/passwd with a single-line command in order to find those users who have a home directory which is not owned by the user. The following has syntax errors in it, but ...