awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
3answers
90 views

How to do a loop to process two input files

I am using awk to do match two files and then multiply the elements of file2 and file1. awk 'NR == FNR{a[$1]=$2; b[$1]=$3; next} /:/ || !NF{print; next} {print $1, $2*a[$1], $2*b[$1]}' ...
1
vote
2answers
81 views

After small code change 'awk' prints header and blank lines, (not header repeated)

data-file content: k = 0.1667 0.0962 0.0000 ( 4922 PWs) bands (ev): -7.1689 -7.1043 -6.1682 -6.0658 1.1309 1.5196 3.0894 3.1087 3.3846 3.7588 4.2143 4.2220 4.3882 ...
0
votes
3answers
70 views

Awk command to get the next line from the matched pattern and crop the string in between two patterns

I need to get the next line after matched pattern and need cut or crop the value in between two patterns from that line. Sample Source file <h2>Commodity Information</h2> <dl> ...
0
votes
3answers
30 views

Extract a part of one line and sort

I have output similar to the following: [test output] my name is bob this is a random string [test output] my name is jim this is a randong string2 [test output] my name is bob this is a randong ...
1
vote
2answers
58 views

How to check if some recurring text is missing in a text file?

I have a text file that looks like this: { "mimeType": "web", "body": "adsfdf", "data_source_name": "abc", "format": "web", "url": "http://google.com/", "urls": "...
1
vote
1answer
64 views

stat(FILENAME) in awk

Title say it all. How can I get file attribute with awk ? or Is there an equivalent to c/php stat() function ? obviously when awk'ing a pipe, file attribute would be undefined. I know I can use ...
-2
votes
2answers
37 views

Data Manupulation required your help

I have a data as mentioned below. host_name Server1.domain.com contacts DL - Desktop contact_groups ravi, raj, rahim host_name Server2.domain.com contact_groups DL-Server host_name Server3....
1
vote
1answer
67 views

Convert vertical array data to horizontal array data

I have a file with 25 matrices in a file, and each matrix is has 128 rows and 5 columns. Each matrix are putted one after the other, in a vertical way. Like this File input(file1): Array1 Array2 ...
1
vote
1answer
47 views

Awk hangs during math operation on Solaris

I'm working on a SunOS 5.11/Solaris 11.3 machine. I have a bash script that calculates and exports CPU frequency because I use it frequently in some test scripts. Here are the two lines of interest: ...
2
votes
6answers
118 views

Comparing 3 lines and 2 lines in BASH

I have a text file which looks like this. 2014-11-24 12:59:42.169 101.0.0.0 source 2014-11-24 12:59:40.375 104.156.80.0 destination 2014-11-24 12:59:36.729 104.219.48.0 destination 2014-11-24 12:59:...
1
vote
3answers
74 views

Creating a new file inside awk but in different directory based on input field value

I am trying to work with awk datetime=`date +\%Y\%m\%d` logdatetime=`date +\%Y\%m\%d` foldermonth=`date +\%B_\%Y` folderday=`date +\%d` inputdir="~/sboper/Standalone/fsplit/GLMS" ...
1
vote
2answers
62 views

Merging Two Files Columns

I want to add 6 field of File2 in last field of File1, i'm using following command but unable to get output. awk 'FNR==NR{a[$1]=$6; next} {print $0,a[$1]}' File2 File1 File1: zehriscollection.co....
2
votes
0answers
54 views

how to capture the latest kernel version from grub.conf

how to capture the latest kernel version from grub.conf ( in case default=0) for example the latest kernel from the file is 2.6.18-409.el5 # all kernel and initrd paths are relative to /...
2
votes
1answer
50 views

How to choose the highest number element in many files with many arrays

I have many files which each one contain many arrays(like input 1 and 2, below). Each array has two element numbers in the second row. I need to use a logical criterion to choose the highest number ...
0
votes
1answer
41 views

How to get the data between specified Starttime and Endtime in a log file?

This is my LogFile(Test.Log): May 10 01:06:05 A May 10 01:06:06 B May 10 01:06:07 C May 10 01:07:01 D May 10 01:07:02 E May 10 01:07:09 F May 10 01:07:11 G May 10 01:07:11 H May 10 01:07:12 I May 10 ...
0
votes
2answers
40 views

Sum column numbers elements in many arrays

I need to sum the the numbers located in the same column in an array. The input is: 1: 6.1703 44 3.5 0.3 46 7.3 2.3 2: 6.1932 44 5.29379 2.9 46 1.72261 7.2261 3: 6.5664 45 4.756 5.6 46 1.6 2.5 4:...
0
votes
0answers
33 views

awk or sed Commands to Print Columns at desired locations [duplicate]

I have a source file which contain 15 columns and so may rows: 2973695.500 633952.188 2440.000 255 0 255 1 1 OP UNK opt TIME SKZ meter ms 2973695.500 633952.188 2440.000 0 255 255 1 1 ...
3
votes
1answer
61 views

awk or sed command to start each column of file from desired locations or Byte Position

I have a source file format like below containing 15 columns and so many rows: 23.5 31.5 5.00 255 0 255 2 1 sal unknown sa time sps meter ms 25.5 32.5 6.00 0 255 0 2 2 sal unknown sa time sps ...
-1
votes
0answers
27 views

How to copy data from a text file to a particular Excel row (or) column using shell script

I am facing the problem in copying text data into a particular Excel column using a shell script. I have used the following command awk -v cols=6 '{printf "%s%s", $1, (NR % cols == 0 ? "\n" : "\t")}...
0
votes
0answers
32 views

get user input in terms of size (GB or MB)

I created script to calculate shmmax shmall shmmini Total_Memory_M=$(dmidecode -t 17 | grep Size | awk '{print $2}' | grep -o '[0-9]\+' | paste -sd+ | bc) Total_Memory_G=$(echo $Total_Memory_M /1024 |...
2
votes
1answer
35 views

How join two files by one common column (1ª) between two file (File1 and file2)?

I am trying join two files by one common column (1ª) between two file (File1 and file2), resulting in file 3, but the resultant file is not working. Can someone help me? I used this command: awk 'NR=...
3
votes
3answers
52 views

Grep/awk/sed for lines composed of only two letters, and lines that start with a letter and meet a certain length

Not sure how many people are familiar with DNA sequencing data, but if this is part of my file (lines starting with ">" are IDs, and lines starting with a letter are DNA sequences): >NB501013:9:...
1
vote
4answers
64 views

Extract X number of minutes from the current time from a logfile

I have a logfile with the following timestamp format 2016-06-02 13:59:58.069 The log I am working with spans about an hour. 2016-06-02 13:05:06.144 2016-06-02 13:05:06.144 2016-06-02 13:05:06.160 ...
1
vote
1answer
46 views

How to sum specific rows of a matrix

I need to sum the matrix elements of specific rows for each column. An example of the matrix is given below(input). So the matrix has 4 columns and 6 rows of numbers. I need to sum the numbers of ...
3
votes
1answer
51 views

awk + how to perform OR in awk

by the following command I capture available size from df echo "$DF" | awk '$NF == "/var" { print $(NF-2) }' how to set the syntax if I want to set NF == "/var" or NF == "/" because I ...
0
votes
1answer
41 views

How to use TWO “for statements” in a shell script?

Writing script to create user accounts. Script is written to echo the useradd command (so I can make sure it will work as expected). I have a file newemploy.txt with Employee Names (first and last, ...
2
votes
3answers
68 views

Mysql output format in Bash Script

I'm writing a bash script where I'm using mysql query in that script where the format which I'm getting is quite different. Query: root@debian:~# mysql -u root -ptoor super_market -h 0 -e "select * ...
0
votes
2answers
58 views

Comparing two lines in BASH using awk

I have text file like this. 2015-11-24 12:59:37.112 128.206.6.136 source 2014-11-24 12:59:36.920 8.8.8.8 source 2014-11-24 14:59:38.112 23.234.22.106 destination 2014-11-24 13:59:37.113 23.234.22.106 ...
1
vote
4answers
64 views

How to use Unindented line as the Record Separate in awk cli

I have a log file that look like: 2016-05-31 09:54:36 (16667) heritage_w? From: ip68-8-49-100.sd.sd.cox.net User: user1wizard (wizard) Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/...
0
votes
1answer
43 views

sum of numbers located in different rows using awk

The following lines sum numbers located in the row #2 and row #3 , row #4 and row #5, row #6 and row #7 and so on (so, sum in a consecutive way). awk ' NR==1{$1=$1; print; next} !(NR%2){split($0,a);...
1
vote
2answers
41 views

How to append some string combined with the pattern at the end of line with the pattern

I want to append .com at the end of line for every line matching the pattern in /etc/hosts file. Sample file content: 127.0.0.1 localhost 1.2.3.4 hostname1 hostname1.xyz hostname1.xyz. 1.2.3....
-1
votes
2answers
164 views

How to sum many numbers inside 2D array using awk

I need to sum numbers located in a file like this one: column1 column2 column3 row1 a(1,1) a(1,2) a(1,3) row2 a(2,1) a(2,2) a(2,3) row3 a(3,1) a(3,2) a(3,3) row4 a(4,1) ...
1
vote
1answer
46 views

completely ignore lines that start with a specific pattern

I have a script which analyzes the output of a computation software. sometimes the output comes with some extra data that are irrelevant to my script. This data can be huge and makes running my simple ...
0
votes
1answer
27 views

How to move lines between patterns to the end of a file

I want to move several lines to the end of the file. Example: 1 2 3 4 5 6 blkn XYZ indf 7 8 9 10 11 lnkn ABCYDF knnfd 12 13 14 15 I want to move lines 6 to 11 to the end of the file. The patterns ...
3
votes
6answers
825 views

How do I remove a words ending in the letter S if duplicates without an S on the end are on the same list?

I have a large list of words. Many of the words are only different because they have the letter s on the end. If a word on the list is the exact same as another word on the list, except one of the ...
-5
votes
1answer
42 views

What happen if record separator and field separator are the same

In awk, what happen if record separator RS and field separator FS are the same? How can I tell fields from records?
1
vote
1answer
45 views

Need to filter the data on the basis of two column having many to one relationship using awk

I have a big file having 50s columns and 100K of rows delimited by |. Now $2(col 2) has multiple type of $1(col 1) value which means col 2 will be repeated. So I have sorted the file. I need now to ...
2
votes
1answer
58 views

on-line tally of unique lines

uniq -c is very useful for counting the number of times the same line appears consecutively $ seq 1 1000 | awk '{ if ($1 > 100 && $1 <= 200) { print "hi" } else {print "bye"} }' | uniq -...
0
votes
1answer
32 views

Understanding sed expression

I am a beginner and would appreciate help on the interpretation of the sed+awk command below, sed -e '1,$s/ /:/g' -e '1,$s/;/:/g' |awk -F ":" '{print $1,$2,$3,$4,$12,$14,$18,$20,$22,$24,$26,$28,$30,$...
2
votes
4answers
65 views

Can Awk read escaped newlines? I.e. read past newlines

I have input looking like this: entry1line1 entry2line1\ entry2line2\ entry2line3 entry3line1 I would like to read such input into an array in AWK, to then process it together with the embedded ...
11
votes
3answers
756 views

How do you remove dot character from string without calling sed or awk again?

I have a file called hostlist.txt that contains text like this: host1.mydomain.com host2.mydomain.com anotherhost www.mydomain.com login.mydomain.com somehost host3.mydomain.com I have the ...
3
votes
2answers
86 views

Organizing three dimensional data with awk/sed/perl

I have this file (a sparse matrix): PC.354 OTU1 6 PC.354 OTU2 1 PC.356 OTU0 4 PC.356 OTU2 7 PC.356 OTU3 3 I want an output like this (dense matrix -classic .biom table): OTU_ID PC.354 PC.355 PC....
1
vote
3answers
58 views

Awk/sed: add line numbers to text from within a range

So I have a range, say 2-4. And I have three lines: first second third I need my output to be: 2 first 3 second 4 third I'm trying this on BSD (Mac) awk/sed, which seems to be making it harder.
1
vote
2answers
31 views

replace the content of one field based on the content in another field

I would like to divide the first column based on the delimiter, here -. Based on the last value, last column, here column 2, should be populated. If the value is 01 or 99 then replace with 2 or 1 ...
0
votes
3answers
64 views

How to extract logs between the current time and the last 15 minutes

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help ...
3
votes
2answers
106 views

Question about concatenation instructions

I have a semicolon-delimited csv file and a Korn shell script that do some elaboration. I'm searching to understand what these two instructions do in Korn shell: num_rec=`cat test.csv |wc -l|tr -d " "...
-1
votes
2answers
49 views

Check if a column is float

I have a requirement that i want to check if a column is float or not. If its a float then do nothing else concatenate with .00. File: 2017 47.0000 0.783333 0.2500 1663 1920.0000 32 8.0000 ...
1
vote
1answer
32 views

Need help with sed/awk with deleting string

i have a txt file contenting the following information -Xmx4096m -Xms4096m -verbose:gc -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -XX:+ScavengeBeforeFullGC -Dsun.net.inetaddr.ttl=3600 -XX:...
1
vote
1answer
25 views

Wrong output in comparing floats

I'm trying to compare two floats in bash and something is going wrong. Here is the code sample based on solution here num1=0.502E-01 num2=0.01 echo $num1'>'$num2 | bc -l echo $num2'>'$num1 | bc ...
0
votes
1answer
36 views

have a log file that I would like to break up into several files based on content

I have a log file that I would like to break up into several files Table of Contents *** Configuration Database *** Logs *** Configuration Logs *** Query Logs . . and so on The issue ...