awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

4
votes
3answers
2k views

Summing up an array inside of awk?

I have the following piece of code: sum1= sum2= declare -a a echo $temp | awk '{split($0,a,","); name=a[1] ; for(i=2;i<=4;i++) sum1+=a[i] ; for(i=5;i<=7;i++) sum2+=a[i] }' This code is ...
10
votes
2answers
1k views

Script to quickly test all keyboard keys

I need to check some notebooks for bad keyboard keys, and so I'd like to speed that up as much as possible. I found nothing for this specific task, so my idea is a script that reads the pressed keys ...
2
votes
3answers
593 views

How does this awk align the spacing?

Can someone please tell me what does this do? cat infile| awk ' { for(i=3;i<=NF;i++){ $2=$2" "$i } printf "%-4s %s\n", $1,$2 ; }' > outfile.txt infile contains: 1 John Smith 2 ...
4
votes
4answers
838 views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
1
vote
4answers
5k views

How to extract line from the file on specific condition

File: chromosome position ref alt chr1 1398 A T chr1 2980 A C chr2 3323 C T,A chr2 3749 T G chr3 5251 C T,G chr3 ...
2
votes
2answers
1k views

How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?

I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below. col0,col1,col2,col3,col4,col5,col6, ...
1
vote
2answers
9k views

How to select, copy and paste some columns from different files and create one file from them?

I want to create and organize data in a file from a number of files by selecting parts of the columns of the given files. I have more than 10 file to copy the second, third and forth columns of each ...
2
votes
3answers
520 views

How to run awk for some number of lines.?

I want to extract some lines with awk. Is it possible to do the following task: ls -l | awk 'BEGIN FOR(i=122;i<=129;i++) FNR==i' How can I display the details from line numbers 122 to 129?
1
vote
3answers
4k views

How to convert from one date format to other format?

I have one problem with my date format. I want to change from one format to the other and vice versa. My date formats are Format 1 YYYY-MM-DD Format 2 MM/DD/YYYY I want to change format 1 to ...
4
votes
6answers
103 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?
1
vote
3answers
243 views

how to calculate the average value of the column that have the row ID?

I want to calculate the average value of my data when they have the same ID (in my specific case, the same day but different time in a day). This is meant to calculate the daily average value of my ...
1
vote
1answer
67 views

how to clean free row/column spaces after the row of your data

I have a problem in cleaning my file where there are unnecessary free spaces below my last data row. This is created when I am copying and pasting my data from excel to text. I want my text file to ...
1
vote
2answers
80 views

how to select part of the row and pasting to form other file?

I want you help me how to organize my data by cutting in the following way. I have the input data as below. input.file: 1 2 1 0.6007 0.1 0.3 0.2 0.7 0.7 0 2 0.3073 0.1 0.1 0.2 0.4 0.7 ...
1
vote
4answers
3k views

Match multiple regular expressions from a single file using awk

I'm trying to parse a HTML file using shell scripting. There are 4 different regular expressions that i need to catch: name= , age= , classs= , marks=. Using grep "name=\|age=\|class=\|marks=" ...
2
votes
3answers
186 views

how to create a data file with a column of date from day1 to dayN?

I was wondering if I could generate a hourly or daily time step date in a column starting from some year in the past to some day in the near past or today. to be more clear, I want to create one ...
10
votes
5answers
2k views

Using grep or awk

To capture a particular pattern, awk and grep can be used. Why should we use one over the other. How does this work in the background? Which is faster and why? E.g If I had a log file and I wanted ...
2
votes
3answers
683 views

sed - find string and append

Suppose you have the following text file 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ...
1
vote
4answers
3k views

Finding out the common lines in two files using 4 fields using awk and UNIX

I have 2 files. If field 1, 2, 4 and 5 of both file1 and file2 matches, then I want to print the whole line of file1 and file2 one after another in my output file. File1: sc2/80 20 . ...
3
votes
3answers
5k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...
4
votes
1answer
168 views

Looking for lines which is in one file but not in other using Unix and Awk

I have 2 files with 7 fields and I want to print the lines which are present in file1 but not in file2 based on field1 and field2. Logic: I want to print all the lines, where there is a particular ...
2
votes
6answers
6k views

Using cut/awk/sed with two different delimiters

I have the following cases: case@test.com case_1_2@test.com case_1@test.com I'm trying to convert these to case@test.com case@test.com case@test.com So it should remove everything from ...
2
votes
4answers
862 views

Can awk use field identifiers also for shell strings (variables)?

Well, this apparently is not possible the way I'm trying it. This alternate approach to obtain bar as a resulting string works, though: #!/bin/bash path1=/usr/tmp/foo/bar/baz awk -F/ '{print $5}' ...
3
votes
3answers
274 views

Easily get a particular column from output without sed or awk

Is there a quicker way of getting a couple of column of values than futzing with sed and awk? For instance, if I have the output of ls -hal / and I want to just get the file and directory names and ...
2
votes
1answer
1k views

Comparing Files based on 5 fields using Awk and Bash

I want to compare File1 and File2 (Separated by spaces) using five fields (Column 1,2,4,5,6). *Logic:*If column 1 and 2 of File1 and File2 match exactly and if the File2 has the same characters as ...
1
vote
3answers
676 views

How to limit MAX an MIN length of a string?

"nothing" means no OUTPUT. Q: how can I limit the MIN and MAX length of a piped STRING? (MIN: x>=3 chars, MAX: x<=32 chars) INPUT: echo 12 | MAGIC OUTPUT: nothing INPUT: echo 123 | ...
1
vote
1answer
53 views

How to make URL's out of this text with least resources?

INPUT: nyeeeeeeeeeeeeeee.txt:sadfsadf sadjflsajfd:jasdlfjalsfd foo/asdfsafd.txt:laksjdlfajsfd OUTPUT: <a href="nyeeeeeeeeeeeeeee.txt">nyeeeeeeeeeeeeeee.txt</a>:sadfsadf <a ...
1
vote
1answer
148 views

Searching for all the array elements in another array?

This is how I would search for an element in an array. count=${#ARRAY[@]} lsearch() { status=-1 for((i=0;i<count;i++)) do Temp=$1 if [[ $Temp -eq ${ARRAY[i]} ]] then ...
1
vote
1answer
1k views

How to integrate a multiline awk script in a shell script

I am a beginner in shell scripting and my question is a continuation of How to parse a file to extract 3 digits numbers kept in a "group number" I am trying to integrate in a single shell ...
6
votes
1answer
651 views

grep: display filename once, then display context with line numbers

Our source code has error codes scattered throughout. Finding them is easy with grep, but I'd like a bash function find_code that I can execute (eg. find_code ####) which will provide output along ...
5
votes
1answer
2k views

Sorting a file based on one column using Unix and Awk

I need to sort the input file according the 6th column, which is the score. Input File: Sc2/80 20 . A T 86 Pass N=2 F=5;U=4 Sc2/80 20 . A C 80 Pass N=2 F=5;U=4 Sc2/60 55 . G T 90 Pass N=2 F=5;U=4 ...
2
votes
3answers
104 views

Pulling out keywords using standard command line utilities?

As an example, this is the pronunciation of "when" according to Wiktionary. enPR, IPA and X-SAMPA are different schemes for showing pronunciations. when:* {{a|US}} {{enPR|wĕn|hwĕn}}, ...
0
votes
1answer
93 views

Splitting Apache Vhosts into Individual Files

I have several vhosts in one file that I am looking to break each into a file of their own corresponding to the ServerName. I'm looking to take this file <VirtualHost *:80> ServerName ...
3
votes
2answers
2k views

Removing unwanted lines in text file

All I need is the sed or awk code to remove the even numbered lines in a text file. Nothing fancy I just need to remove irrelevant data on the even lines of a text file.
3
votes
7answers
2k views

Problem in splitting a string using awk or cut command

I have a properties file which has a password field (key value pair) PASSWORD=NDhhHcsOBofXUdUzGw5B0Q== I am trying to get the value in my shell script using the awk command password=`awk -F "=" ...
2
votes
2answers
3k views

Comparing two files using Unix and Awk

I need to compare two files, File1 and File2 (Separated by space) using 4 fields (Field 1, 2, 4 and 5 of File 1 with field1, 2, 4 and 5 of File2). Logic: If column 1, 2 and 4 of File 1 matches with ...
2
votes
6answers
176 views

Bash - Split the previous commands parameters

How would one parse the previous command in bash? Example: root$ ssh foo@1.2.3.4 root$ echo !$ &>/tmp/foo.txt root$ cat /tmp/foo.txt foo@1.2.3.4 Goal: Send just the ipaddress 1.2.3.4 to ...
1
vote
3answers
171 views

Cat 2 files and avoid duplication of lines

I have the file1 with the following content {"name":"clio5", "value":"13"} {"name":"citroen_c4", "value":"23"} {"name":"citroen_c3", "value":"12"} {"name":"golf4", "value":"16"} {"name":"golf3", ...
1
vote
1answer
67 views

How to design the table in awk?

My input file is file1 count month 505 Apr-2010 252 Mar-2010 426 Feb-2010 201 Jan-2010 211 Dec-2009 Same file I want the output like this below +-------+----------+ | count | month ...
1
vote
1answer
98 views

Compare the similar file and not similar files display below output in awk?

I'd like to compare the following 2 files and display a comparison of the 2 files with each other. File #1: DATE DS 2012-08-02 1 2013-06-23 1 2013-06-27 2 2013-06-28 2 2013-06-29 779 File ...
1
vote
2answers
77 views

How to use AWK record separator?

Here is the input: * true, false, yes, no, always, never. * ******************************************************************************* default: su = true ... maxexpired = ...
1
vote
3answers
3k views

Count number of lines with a certain value in a column for all files in directory recursively

I have 40 files in a directory and I want to count the number of times there is a line with a "2" in the first column in each file individually. I am trying something like this, but it prints out ...
1
vote
2answers
230 views

Are there versions of Unix that don't have awk in default install?

Are there versions of Unix that don't have awk installed by default? I am fine with an answer containing some really old outdated Unix variant which predates awk's creation; but would ideally prefer ...
-1
votes
2answers
52 views

Line match to trigger specific argument

I have a file with 2 types of contents as below. QJKLJFLHLKJDGFIUD 03:12:23 JHKJFDGDSFGDFGHDF 03:15:45 HJIK ASDSdSADSADSSADSA 03:30:34 DFDF KWQA I have 2 binaries. One is on the local ...
4
votes
3answers
236 views

Check for, and add, missing timestamps to individual lines in a file

I have a file in the following format: ABCD 01206001022T01YA022T01YA022T07SO 09:20:38 ABCD 01206001022ACION 09:24:40 ABCD 04006001021S01UK 09:24:42 ABCD 7878696621321312 23213213213213 ABCD ...
2
votes
6answers
4k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
1
vote
5answers
491 views

delete lines in a csv file older than 7 days

I have a csv that I need to remove all lines that are older than 7 days. This is the format of the csv Person ID VIP CS SS LT FTLT PS Modified Datestamp T001028 1 1 1 0 0 0 ...
3
votes
4answers
2k views

Split long output into two columns

Is there a simple utility or script to columnate the output from one of my scripts? I have data in some form: A aldkhasdfljhaf B klajsdfhalsdfh C salkjsdjkladdag D lseuiorlhisnflkc E sdjklfhnslkdfhn ...
10
votes
8answers
2k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
-1
votes
2answers
2k views

How can I print unique line base on particular column value using awk? [closed]

I have a comma separated file with multiple fields. Example: candidate1,12,56,ATTG candidate2,45,90,ATTG candidate3,90,140,ATTG candidate4,2,36,AGCT So, does anyone know the answer?
0
votes
4answers
5k views

Extract substring using regular expression on a Unix file

I have file with below contents . /ABC/RTE/AD_900_VOP_123/OPP /ABC/RTE/TRE/AD_900_VOP_145/BBB /ABC/RTE/AN_900_VFP_124/FBF /ABC/RTE/HD_900_FOP_153/WEW /ABD/RDV/AD_900_VOP_123/OPP ...