Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [awk]

A pattern-directed scanning and processing language.

-1
votes
0answers
18 views

Comparision of two columns in two different file should give only approximate match not exact match

My requirement is that I have two txt files like below: $ cat tile_internals.txt workbook_name,tile_name,column_caption /*header*/ Application Actions,Application Aging Alert,Academic Levels ...
0
votes
1answer
18 views

Joining 2 files and excluding word from third file [on hold]

I have three files: file1, file2 and exl. Both file1 and file2 have two columns of content. The first column is the same in both files. I want to join the content of both files using that column. ...
-1
votes
1answer
34 views

cumulative table, create new table in which values in each column are sum of each previous column values [on hold]

I am dealing with script to transform table - in output table each column should be a sum of previous columns (something like cummulative table). Table got unknown number of rows and columns, first ...
0
votes
1answer
23 views

Splitting a file based on values next to matching pattern

I am having a file input.txt which include ~50,000 rows and ~100 column. I want to split is according to matching entry followed by the matching patter. File separator are both space and tab. input....
2
votes
3answers
44 views

How to loop through first file and print each row in the first file next to all the rows in the second file using AWK?

I think the title may be confusing, so here's a small example of what I am trying to accomplish. File 1: Adam John Tim File 2: ,2018-2-1301 ,2018-3-1379 ,2018-1-3809 ,2018-2-5801 Output: Adam,...
0
votes
2answers
26 views

Replace different strings based on a match by a list of strings in another file

I have a file like this with tab delimiter: Chr1 mak gene 120221 120946 . + . ID=spa-h0003.02;Name=spa-h0003.02 Chr1 mak mRNA 120221 120946 . + . ...
0
votes
3answers
24 views

count matching open ports in a network log file

I have network log file like this one: Nmap scan report for 192.168.1.51 Host is up. PORT STATE SERVICE 80/tcp open http 443/tcp open https 8080/tcp open http-proxy 443/tcp ...
6
votes
5answers
841 views

Making summary of sentences

I have data and I want to summarize sentences to generate conclusions. The example below is not related to the data, but just to clarify the idea so I can replicate it. Employee Suzie signed one time....
-2
votes
1answer
29 views

To develope a list of unique names with sysdate [on hold]

I Have a file with ID’s and Names applicable to them as below: 1234|abc|cde|fgh 5678|ijk|abc|lmn 9101|cde|fgh|klm 1213|klm|abc|cde I need a file with only unique Names as a list. Output File: abc|...
1
vote
1answer
44 views

How to remove pattern from line but return what remains?

If I run ip addr sho | grep 9000 it returns the interface that I need quick stats for: eth3: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 9000 qdisc mq state UP qlen 1000 However, I'd like to ...
1
vote
3answers
58 views

extract, cut, process strings in a bash script

I have basically 4 txt files with records such as FILE 1 localhost_access_2018-07-19.tar.gz:13 localhost_access_2018-07-20.tar.gz:17 localhost_access_2018-07-21.tar.gz:12 localhost_access_2018-07-...
0
votes
2answers
49 views

Organizing data by a header

I have two csv files. File 1 contains header. File 2 contains data file 1 format : file1.csv id,abc,xyz,aaa,bbb,ccc file 2 format : file2.csv id,source,value 1,abc,100 1,xyz,...
1
vote
2answers
57 views

How to add seconds or minutes to given timestamp in AWK

Given I have following awk script: #!/usr/bin/awk -f BEGIN { startTimestamp = ARGV[1]; endTimestamp = ARGV[2]; print startTimestamp print endTimestamp print startTimestamp + 60 seconds ...
4
votes
3answers
163 views

Deleting portion of a text file and following lines using sed

I need to edit a file like the following: auto wlx00 allow-hotplug wlx00 iface wlx000 inet dhcp iface wlx000 inet6 auto post-up sysctl -w net.ipv6.conf.wlx000.accept_ra=2 auto wlx000 the goal is ...
5
votes
4answers
534 views

How can I find all lines containing two specified words? [duplicate]

I need to check if two (specified) words exist on any line in a text file. There are no limits for the characters of the words. For example: I want to find lines of a text file that contain the two ...
2
votes
4answers
957 views

How to concatanate the result of two Linux commands

Is there an easy way to concatenate the result of two Linux commands, in one line (i.e., without using variables)? I pull the local outdoor temperature from a nearby weather station. The result ...
0
votes
1answer
23 views

how to copy columns from file 2 to file 1 when some columns matches between 2 files

I have a file head top_candidates 25 elevation_e gene1 20 9 0.0246022994932004 5 8 10.9217937824527 30 elevation_e gene1 59 18 0.0246022994932004 7 12 15.653559774527 31 elevation_e ...
-4
votes
3answers
80 views

File Processing to transpose data [closed]

I have a file with IDs and names applicable to them as below: 1234|abc|cde|fgh 5678|ijk|abc|lmn 9101|cde|fgh|klm 1213|klm|abc|cde I should get the output as different files having IDs matching each ...
0
votes
0answers
30 views

Extracting groups/users allowed in sudoers

I would like to extract all groups/users that have been given permission to "sudo". let us take this as an example. I am using a very basic awk command as shown below. awk '/## Allow root to run ...
0
votes
2answers
31 views

Properly feed paths to executables in bash script

Hello all I have a text file (1.txt) that contains full paths to binaries which I need to execute a command (val_bin) for each path in the file. 1.txt Example: /source/ed/dev/ed/ab/ed_len/comercial/...
0
votes
1answer
26 views

Extract substring after delimiter using GNU awk or awk [duplicate]

I would like to extract substring after colon delimiter for my practical needs using GNU awk utility. For example I can extract the temperature of hdd value using cut as follows hddtemp /dev/sda | ...
-2
votes
2answers
48 views

Find longest stretch of pattern from centre of sequence

How to calculate the longest consecutive pattern of C/T anywhere in sequence and longest consecutive pattern of C/T from centre CCCCTGTTGCCAAACAATGC TTTTCCCGCCTTTGGCCTAC ...
0
votes
1answer
28 views

Extract names from File_B having overlapping intervals with File_A

Two space-delimited files: File_A MT 50000 groupI 7850000 groupI 7950000 groupI 9050000 groupI 21750000 groupII 8750000 groupII 10550000 groupII 16150000 groupII 20850000 groupIII 14750000 groupIII ...
-1
votes
2answers
175 views

Add four lines between sentences [duplicate]

Is there an easy way to add four empty lines between each sentence. For example, this paragraph: This combination of vitamins and selected minerals is used to prevent or treat vitamin deficiency ...
0
votes
5answers
101 views

How to get a count of variables passed to awk [closed]

I am trying to invoke awk inside a bash script and passing some bash variables values to it. Is there any way to get a count of these variables and print all which are passed to it inside awk. No. of ...
0
votes
3answers
65 views

How to split a file into paragraphs and name the resulting pieces based on an identifier present in each paragraph

I have a big file with more than 3264880 lines. I wanted to split that file based on Two strings "BEGIN JOB" and " END JOB" and write it into multiple files and the file name should be based on a ...
2
votes
2answers
152 views

Turning data collected vertically to a table

I have data was collected and saved in file "data.txt", in the following format: <$ MG_nr> </200>company-type:Engine1 </210>company-name:STR </220>address:XX </230>nr:...
1
vote
4answers
55 views

How to move a line to the previous line if a certain character “ : ” is not found? [duplicate]

I have a lot of text files of transcripts. I have cleaned it to an extent. The last bit of cleaning is the following. I have this in certain files *.txt Gary: I said something. Larry: I said ...
3
votes
4answers
58 views

How to compare the 1st column of two files and if the strings match then print “true” [duplicate]

I have two files, say file1 and file2. file1: 91 | 2018-10-11 16:07:14 | bkwdv | 188.0000 | 77 | 2018-10-12 15:05:15 | wvwef | 94.0000 | file2: 77 | 2018-10-12 15:05:15 | wvwef | 94.0000 91 | ...
4
votes
3answers
292 views

Command line argument in awk

I just want to implement if condition in awk. I created one file name : "simple_if" as below. BEGIN{ num=$1; if (num%2==0) printf "%d is Even number.\n",num; else printf "%d is odd Number.\n",num } ...
2
votes
3answers
353 views

Bash Script to remotely collect hostname, IP and host total memory

I am trying to write a scrip that will collect the hostname, IP and total memory installed on remote hosts from a list I feed into the script. The script will collect information from Redhat and ...
-2
votes
1answer
58 views

How to do awk to do sorting between a txt file and a csv file? [closed]

Here is the file that I am going to use: file1.txt: School JUPAS_N Univ ID Working_line_number TST 11-2 CUHK 90544 file2.csv: Date,JUPAS,UniType,DSE,PRA ,PE,SIP,BRA,DDI Group,...
0
votes
2answers
40 views

How to understand the command awk '{for(x=1;x<=NF;x++)if($x~/0.00000/){sub(/0.00000/,++i)}}1' file

As for this Command: awk '{for(x=1;x<=NF;x++)if($x~/0.00000/){sub(/0.00000/,++i)}}1' I know this command will look for "0.00000" in a file and replace it with an incremental number starting from ...
0
votes
2answers
44 views

How to sort the data from CSV file by comparing data from text file using awk output it as a TXT file?

There are two files I am going to extract the data: file1.txt: Type Serial ID Element Hit_Possibility Yasuo 19-2 19623 Hasaki file2.csv: Date,Name,Order,Hit Possibility 12-Aug,Ken,1,256 12-Aug,...
1
vote
2answers
30 views

How to match a column from File1 to get its corresponding fasta sequences in File 2?

May I kindly ask how to match all values of the first column in File with the line text in File 2 so I can copy the fasta sequences of all Object ID in File 1? File 1.csv file Object_ID, Length, ...
0
votes
3answers
45 views

adding a new line character to a row

I'd like to extract the 2nd field and transpose it to become a row cat > temp1 1 73 2 74 3 75 My approach : cut -f2 temp1 | tr '\n' '\t' > temp2 The result : temp2 73 74 75 ...
0
votes
3answers
109 views

Field separator part of a column - incorrect parsing unix

I want to do a check the number of columns in a CSV file before processing it. The problem is that the delimiter (comma) also occurs in the text of some fields and because of that I cannot parse it ...
0
votes
3answers
59 views

How to extract second element from path? [duplicate]

How do I extract such pattern using sed? This is my path: /sample/xyz/logs/test.logs Output should be : xyz How is that possible if the path keeps changing for example like /example/xyz/log/test....
0
votes
3answers
49 views

extract paragraph match with words using sed or awk [duplicate]

I have file called "quota.txt" as below: DISK_info_start:/usr Disk: /usr CURRENT=81 DISK_info_end:/usr DISK_info_start:/usr/var Disk: /usr/tmp CURRENT=1 DISK_info_end:/usr/var I have a variable as ...
0
votes
1answer
38 views

To compare column1 of two files

I need to compare the column 1 of file 1 with the column 1 of file2 in my script, if the column 1 of file1 matches with the column 1 of file2, then only it should proceed further ,else exit. I'm with ...
0
votes
0answers
22 views

aix extract fields from entstat command

I was collecting the SEA adapter and its related etherchannel and Network adapters status from the enstat command below. I want to extract the fields from the below command output. ETHERNET ...
0
votes
2answers
40 views

How to swap two fields inside a JSON file

I have a file like this {"100":["0.00001","10","0.01"], "101":["0.0001","100","0.1"], "102":["1","1000","1"], .... "103":["0.01","10000","10"]} I want to swap the second number with the third, ...
2
votes
3answers
186 views

Replacing just the first occurence of a character in a line

I have a line with thousands of lines like this: {"100K";"0.00001";"10";"0.01"]} {"101K";"0.0001";"100";"0.1"]} {"102K";"1";"1000";"1"]} {"102K";"1";"1000";"1"]} {"103K";"0.01";"10000";"10"]} {"104K";...
4
votes
2answers
229 views

How to reformat text in a file to another direction?

I have table in the current format: Lead:Arrow1:Arrow2:Arrow3 Follow:Arrow4:Arrow5:Arrow6:Arrow7:Arrow8:Arrow9 I want to turn this to the Lead Arrow1 Arrow2 Arrow3 Follow Arrow4 Arrow5 Arrow6 ...
0
votes
2answers
43 views

remove extra symbols and adjust lines

I have text file (test.txt) that looks exactly like this (same number of options for every instance): # a comment first: info about first: option1 \ option2 \ ...
0
votes
1answer
41 views

Normalizing columns in a table using AWK

I have table of numbers, I want to normalize the table as following: 1- Multiplying column 3 and 7 then add the result to column 11 and multiply the last result with column number 1. 2- Then I want ...
1
vote
3answers
113 views

Print text between two patterns

I need to: Print text between "hi" and "hello" patterns. Patterns may be repeated and there may be multiple occurrences of hi..hello. Source file: hi aa bb cc dd ee ff hello xxxxxx aaaa dddd ...
1
vote
3answers
113 views

split string to two parts using sed or awk or perl or bash

I have string like this: Grades ABCDEF-123456 I want to split this string to two sections like this Grades ABCDEF Grades 123456 How can I do that in bash?
1
vote
2answers
71 views

delete specific text based on the next line contents

I need a command to delete specific text based on the contents of the next line, specifically I want to delete "comma" if the next line is "]", and delete that next line "]" as well. example ...
5
votes
6answers
562 views

Extract `m`th (first) column value for line with specific `n`th (second) column value from file [duplicate]

I need to Write an awk command that will return the identification number from the following table for only the lines where the title is Turtle. This table is stored in turtle.txt Id Num. Title ...