awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

10
votes
2answers
851 views

Script to quickly test all keyboard keys

I need to check some notebooks for bad keyboard keys, and so I'd like to speed that up as much as possible. I found nothing for this specific task, so my idea is a script that reads the pressed keys ...
4
votes
3answers
1k views

Summing up an array inside of awk?

I have the following piece of code: sum1= sum2= declare -a a echo $temp | awk '{split($0,a,","); name=a[1] ; for(i=2;i<=4;i++) sum1+=a[i] ; for(i=5;i<=7;i++) sum2+=a[i] }' This code is ...
2
votes
3answers
331 views

How does this awk align the spacing?

Can someone please tell me what does this do? cat infile| awk ' { for(i=3;i<=NF;i++){ $2=$2" "$i } printf "%-4s %s\n", $1,$2 ; }' > outfile.txt infile contains: 1 John Smith 2 ...
4
votes
4answers
475 views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
2
votes
2answers
791 views

How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?

I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below. col0,col1,col2,col3,col4,col5,col6, ...
1
vote
4answers
2k views

How to extract line from the file on specific condition

File: chromosome position ref alt chr1 1398 A T chr1 2980 A C chr2 3323 C T,A chr2 3749 T G chr3 5251 C T,G chr3 ...
1
vote
2answers
5k views

How to select, copy and paste some columns from different files and create one file from them?

I want to create and organize data in a file from a number of files by selecting parts of the columns of the given files. I have more than 10 file to copy the second, third and forth columns of each ...
4
votes
6answers
99 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?
2
votes
3answers
175 views

How to run awk for some number of lines.?

I want to extract some lines with awk. Is it possible to do the following task: ls -l | awk 'BEGIN FOR(i=122;i<=129;i++) FNR==i' How can I display the details from line numbers 122 to 129?
1
vote
3answers
2k views

How to convert from one date format to other format?

I have one problem with my date format. I want to change from one format to the other and vice versa. My date formats are Format 1 YYYY-MM-DD Format 2 MM/DD/YYYY I want to change format 1 to ...
1
vote
2answers
371 views

reading from a file and changing its pattern into an array?

I have a CSV file containging entries like this : ipaddress,VLAN,VLANid 10.192.168.1,vlan-xyz,3 10.192.168.1,vlan-abc,8 10.192.168.1,vlan-mnp,11 10.192.163.24,vlan-llz,3 10.192.163.24,vlan-bbz,5 ...
1
vote
2answers
71 views

how to select part of the row and pasting to form other file?

I want you help me how to organize my data by cutting in the following way. I have the input data as below. input.file: 1 2 1 0.6007 0.1 0.3 0.2 0.7 0.7 0 2 0.3073 0.1 0.1 0.2 0.4 0.7 ...
1
vote
2answers
266 views

How to count the number of words and print the lines that match exactly a given pattern?

I want to count the number of words and print the matched pattern lines which matches exactly with following pattern: abc-ERROR: The input File contains: # abc-ERROR: xyxxkkfgfr # def-Error: ...
0
votes
4answers
2k views

Match multiple regular expressions from a single file using awk

I'm trying to parse a HTML file using shell scripting. There are 4 different regular expressions that i need to catch: name= , age= , classs= , marks=. Using grep "name=\|age=\|class=\|marks=" ...
10
votes
5answers
1k views

Using grep or awk

To capture a particular pattern, awk and grep can be used. Why should we use one over the other. How does this work in the background? Which is faster and why? E.g If I had a log file and I wanted ...
2
votes
3answers
446 views

sed - find string and append

Suppose you have the following text file 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ...
3
votes
3answers
3k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...
1
vote
6answers
3k views

Using cut/awk/sed with two different delimiters

I have the following cases: case@test.com case_1_2@test.com case_1@test.com I'm trying to convert these to case@test.com case@test.com case@test.com So it should remove everything from ...
2
votes
1answer
1k views

Comparing Files based on 5 fields using Awk and Bash

I want to compare File1 and File2 (Separated by spaces) using five fields (Column 1,2,4,5,6). *Logic:*If column 1 and 2 of File1 and File2 match exactly and if the File2 has the same characters as ...
4
votes
1answer
132 views

Looking for lines which is in one file but not in other using Unix and Awk

I have 2 files with 7 fields and I want to print the lines which are present in file1 but not in file2 based on field1 and field2. Logic: I want to print all the lines, where there is a particular ...
2
votes
1answer
1k views

Diff the output of two `awk` commands

I'm trying to compute the difference between the output of two awk commands but my simple attempts at it seem to be failing. Here is what I'm trying: diff $(awk '{print $3}' f1.txt | sort -u) $(awk ...
5
votes
1answer
1k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
1
vote
3answers
430 views

How to limit MAX an MIN length of a string?

"nothing" means no OUTPUT. Q: how can I limit the MIN and MAX length of a piped STRING? (MIN: x>=3 chars, MAX: x<=32 chars) INPUT: echo 12 | MAGIC OUTPUT: nothing INPUT: echo 123 | ...
1
vote
1answer
52 views

How to make URL's out of this text with least resources?

INPUT: nyeeeeeeeeeeeeeee.txt:sadfsadf sadjflsajfd:jasdlfjalsfd foo/asdfsafd.txt:laksjdlfajsfd OUTPUT: <a href="nyeeeeeeeeeeeeeee.txt">nyeeeeeeeeeeeeeee.txt</a>:sadfsadf <a ...
1
vote
1answer
601 views

How to integrate a multiline awk script in a shell script

I am a beginner in shell scripting and my question is a continuation of How to parse a file to extract 3 digits numbers kept in a "group number" I am trying to integrate in a single shell ...
4
votes
1answer
351 views

grep: display filename once, then display context with line numbers

Our source code has error codes scattered throughout. Finding them is easy with grep, but I'd like a bash function find_code that I can execute (eg. find_code ####) which will provide output along ...
2
votes
3answers
95 views

Pulling out keywords using standard command line utilities?

As an example, this is the pronunciation of "when" according to Wiktionary. enPR, IPA and X-SAMPA are different schemes for showing pronunciations. when:* {{a|US}} {{enPR|wĕn|hwĕn}}, ...
0
votes
1answer
77 views

Splitting Apache Vhosts into Individual Files

I have several vhosts in one file that I am looking to break each into a file of their own corresponding to the ServerName. I'm looking to take this file <VirtualHost *:80> ServerName ...
3
votes
2answers
1k views

Removing unwanted lines in text file

All I need is the sed or awk code to remove the even numbered lines in a text file. Nothing fancy I just need to remove irrelevant data on the even lines of a text file.
3
votes
7answers
1k views

Problem in splitting a string using awk or cut command

I have a properties file which has a password field (key value pair) PASSWORD=NDhhHcsOBofXUdUzGw5B0Q== I am trying to get the value in my shell script using the awk command password=`awk -F "=" ...
2
votes
2answers
2k views

Comparing two files using Unix and Awk

I need to compare two files, File1 and File2 (Separated by space) using 4 fields (Field 1, 2, 4 and 5 of File 1 with field1, 2, 4 and 5 of File2). Logic: If column 1, 2 and 4 of File 1 matches with ...
2
votes
6answers
164 views

Bash - Split the previous commands parameters

How would one parse the previous command in bash? Example: root$ ssh foo@1.2.3.4 root$ echo !$ &>/tmp/foo.txt root$ cat /tmp/foo.txt foo@1.2.3.4 Goal: Send just the ipaddress 1.2.3.4 to ...
1
vote
3answers
146 views

Cat 2 files and avoid duplication of lines

I have the file1 with the following content {"name":"clio5", "value":"13"} {"name":"citroen_c4", "value":"23"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"16"} {"name":"golf3", ...
6
votes
3answers
3k views

Why does awk do full buffering when reading from a pipe

I'm reading from a serial port connected to a gps device sending nmea strings. A simplified invocation to illustrate my point: $ awk '{ print $0 }' /dev/ttyPSC9 ...
1
vote
1answer
62 views

How to design the table in awk?

My input file is file1 count month 505 Apr-2010 252 Mar-2010 426 Feb-2010 201 Jan-2010 211 Dec-2009 Same file I want the output like this below +-------+----------+ | count | month ...
1
vote
1answer
86 views

Compare the similar file and not similar files display below output in awk?

I'd like to compare the following 2 files and display a comparison of the 2 files with each other. File #1: DATE DS 2012-08-02 1 2013-06-23 1 2013-06-27 2 2013-06-28 2 2013-06-29 779 File ...
1
vote
2answers
68 views

How to use AWK record separator?

Here is the input: * true, false, yes, no, always, never. * ******************************************************************************* default: su = true ... maxexpired = ...
1
vote
3answers
2k views

Count number of lines with a certain value in a column for all files in directory recursively

I have 40 files in a directory and I want to count the number of times there is a line with a "2" in the first column in each file individually. I am trying something like this, but it prints out ...
1
vote
6answers
2k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
1
vote
2answers
204 views

Are there versions of Unix that don't have awk in default install?

Are there versions of Unix that don't have awk installed by default? I am fine with an answer containing some really old outdated Unix variant which predates awk's creation; but would ideally prefer ...
9
votes
8answers
1k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
-1
votes
2answers
52 views

Line match to trigger specific argument

I have a file with 2 types of contents as below. QJKLJFLHLKJDGFIUD 03:12:23 JHKJFDGDSFGDFGHDF 03:15:45 HJIK ASDSdSADSADSSADSA 03:30:34 DFDF KWQA I have 2 binaries. One is on the local ...
4
votes
3answers
188 views

Check for, and add, missing timestamps to individual lines in a file

I have a file in the following format: ABCD 01206001022T01YA022T01YA022T07SO 09:20:38 ABCD 01206001022ACION 09:24:40 ABCD 04006001021S01UK 09:24:42 ABCD 7878696621321312 23213213213213 ABCD ...
-1
votes
2answers
1k views

How can I print unique line base on particular column value using awk? [closed]

I have a comma separated file with multiple fields. Example: candidate1,12,56,ATTG candidate2,45,90,ATTG candidate3,90,140,ATTG candidate4,2,36,AGCT So, does anyone know the answer?
0
votes
4answers
3k views

Extract substring using regular expression on a Unix file

I have file with below contents . /ABC/RTE/AD_900_VOP_123/OPP /ABC/RTE/TRE/AD_900_VOP_145/BBB /ABC/RTE/AN_900_VFP_124/FBF /ABC/RTE/HD_900_FOP_153/WEW /ABD/RDV/AD_900_VOP_123/OPP ...
1
vote
2answers
605 views

Use FOR Loop inside END block of awk command

I'm trying to find the average number of files per day. The following script works: #!/bin/sh ls -l | grep "^-" | awk '{ key=$6$7 freq[key]++ } END { for (date in freq) printf ...
1
vote
1answer
159 views

Regex to Cisco Interface

I would like to know if have some sed, grep or awk regex to parse Cisco interface section and exclude if interface is down (shutdown), like example below. Example: ! interface FastEthernet0/1 ip ...
4
votes
3answers
782 views

Change the text on a single line number in multiple text files

Suppose that I have ten bash shell scripts: script1.sh, script2.sh, ..., script10.sh. Initially, all ten files are identical. Now, I would like to make two changes in each script: In each file, ...
1
vote
1answer
302 views

awk can't print array element

Lets sat my file1 is: A S B Q file2 is: B 2 I want to match $1 from file2 with $1 file1 and print file1 $2 && file2 $2, for the output like this: Q 2 My awk solution ...
0
votes
1answer
216 views

Regex awk to Cisco Interface

I would like to know if have some Sed/Grep or Awk regex to parse Cisco interface section, with specific attribute, like bellow. Content of file.txt ! interface FastEthernet0/1 no ip unreachables ! ...