awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
1answer
58 views

Solaris: PKG - Script To Verify All Packages Except for a Few

I'm trying to verify the all packages except for a pre-defined list of packages that I know are going to fail for known reasons. This script is going to be run on all Solaris systems within our ...
3
votes
1answer
50 views

Filter out inconsistent data blocks

I would like to filter out rows that has inconsistent geno values in line-part blocks. In case of duplicate consistent values , I would just keep the first row. As example, 'gf345 part1' has more ...
2
votes
1answer
47 views

I want to update a INI file using awk

I have an ini file which looks like this [backup] [persistence] log_backup_timeout_s = 900 log_mode = normal I want update this file to [backup] data_backup_parameter_file = ...
2
votes
1answer
501 views

extracting data from a log file and plotting it using Gnuplot

I have a logfile with the following data in it: Sat Sep 20 11:25:01 BST 2014 -- temp=53.0'C -- message from script /usr/src/scripts/wifi_test_2.sh Sat Sep 20 11:26:01 BST 2014 -- temp=53.0'C -- ...
1
vote
1answer
28 views

Syntax error removing duplicate lines via awk '!x[$0]++' file

The file contains cat file a b c b d Trying to remove duplicate lines on SunOS via awk '!x[$0]++' file (as found in another postings) results in a syntax error awk: syntax error near line 1 ...
1
vote
1answer
25 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
1
vote
1answer
23 views

How to see the lines whose date column is less than x day from today

My file ID=29524 02/12/2015,23:59:34,'PAt, 0, ID=29628 02/12/2015,23:59:37,'PAb, 1.9, ID=30816 03/12/2015,00:00:20,'PAc, 4.15, I would like to print files with X days less than from today. My try ...
1
vote
1answer
36 views

how to set a variable to multiple values with awk?

I have a file similar to the following : x 0 1 x 1 1 and I am essentially trying to see if both fields are equal to one another or not for every row. But the problem is that this file contains x ...
1
vote
1answer
59 views

Replace awk and tr with just awk for converting to lowercase

Not a duplicate question, as I am asking for awk specifically. I have the following command to convert everything to lowercase, and then reprint out substrings in a certain order. cat words |tr ...
1
vote
1answer
30 views

display users that logged in since a date using last

I am trying to display the user name of the accounts that have logged in during the past month using the last command. At the moment I have this last | awk '{print $1, $4 ,$5 ,$6}' | grep -B 10 Jul ...
1
vote
1answer
45 views

Find matches from an index file without exact matching and print the last field

I have a large file (~6mill rows) with 2 columns which looks like this: 1111.aaaaabbb.b.cccc.c ValueA 2222.dddddeee.e.ffff.f ValueB 3333.gggghhhh.h.iiii.i ValueC I want to use that as my index ...
1
vote
1answer
39 views

Form a output by combining multiple lines

I have a log file as below: /export/home/got/logs/o2877612.01:job_1432787863184_159408 Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: ...
1
vote
1answer
77 views

How to count the frequency of a word >20

Using exclusively AWK, I need to count the occurrence of a word only when greater than 20 characters. I've already found the way to count the frequency of a word, but I have some problems in setting ...
1
vote
1answer
210 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
1
vote
1answer
102 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
60 views

Unable to replace variable in shell script using awk

I have been struggling to replace every occurence of "<KEY>" in a file with ${encryptedkeyValue} I wrote this line but it is not doing ...
1
vote
1answer
108 views

awk array using number as value did not work

I have a reference file: Refrence File Dpse\GA30012 FBgn0000447 chr2 26607738 26607962 -1 Dpse\GA19764 FBgn0085819 chrX 28571020 28571736 -1 Dpse\ttk FBgn0000100 chr2 ...
1
vote
1answer
99 views

Multiply rows of a column in different chunk of data, add them together and get the average?

I have a chunk of data that looks like this: 924 777 777 777 8445.65349 P 9.44657 -0.840283 -0.394467 C 4.0555 -0.683104 -1.70917 P 7.5169 3.54172 2.77681 C 3.21536 2.67146 2.20242 P 4.33641 7.62012 ...
0
votes
1answer
43 views

Add column to a CSV file as a function of other two columns, one from other file

Here is the situation: a.csv Contains values from some sensors captured every 1 second, the last column is the epoch (they actually looks like 1453998477.044).(*s are the sensors values). Example: ...
0
votes
1answer
34 views

How to paste over column by separate matching column

I have these two files: file1.txt 0 rs3094315 0 0 C T 0 rs12562034 0 0 A G 0 rs3934834 0 0 C T file2.txt Chr,Pos,snp_name 1, 742429,rs3094315 Where the ...
0
votes
1answer
29 views

Format the output of awk command

I have to format the output of the awk command which gets the input from a file Sample Input File 123 ABC 12 23 34 45 56 67 78 13 234 DEF 23 34 45 56 67 78 89 345 GHI 34 45 56 67 78 90 10 01 99 456 ...
0
votes
1answer
30 views

Make new line above last line and add something there

Like a reverse of awk 'NR==2 {print ""} 1 which adds a new line under the first and prints whatever there. I can't replace the number after NR== with a fixed value because I want to be able to do this ...
0
votes
1answer
104 views

Shell script to process service restart procedure

We're running a 3rd party TTY service on our servers, to support some thin clients manufactured by the 3rd party. From time to time, connection to a thin client can get locked up. Procedure is to ...
0
votes
1answer
52 views

How do I sort a file and output the latest record in case duplicates exist?

I save all the requests, received via API, in a file. I wish to process only the latest request if duplicate requests arrive. Sort command is not helping, I am using : sort -r -k 1.69,1.88 ...
0
votes
1answer
53 views

Need help in writing awk to extract cpu utilization from sar command

I am trying to extract the values of the %user,%nice etc from the output of the sar command. Command : sar -P ALL 1 1 Output of this : Linux 2.6.32-358.el6.x86_64 (ftizsldapp009.ftiz.cummins.com) ...
0
votes
1answer
77 views

Wanted: nonassociative array in AWK

I am looking at speeding up some code in AWK by serially processing an internal string representation instead of serially processing an array representation. (For Examples 1 and 2 below assume ...
0
votes
1answer
47 views

Fetch second and fourth column from a file

Problem Statement I am not able to get the values in second and fourth column. Tried grep -i "AUGC\|AACN\|AACY" output.csv | cut -d ',' 2,4 DATA ...
0
votes
1answer
31 views

Print specific FULL column only by range

I have a file which have contents like this. Column1 Column2 Column3 Column4 USER1 25 Years xyz zyz 123 abc ...
0
votes
1answer
28 views

sum over floating points of the form 3.016e-06 using awk

I want to sum over floats of the form: 3.016e-06 The floats are saved in a files called S_3x3.txt S_4x4.txt ... S_8x8.txt My code sums only the first integer. Instead of 3.016e-06 + 4.022e-06 it ...
0
votes
1answer
40 views

Print concat vars inside loop

I have a code like this: cat input | while read p; do lynx -dump "www.xyz.com/p?=$p" | grep -m 1 -A 1 "Hello" | awk -F: '{print $2}' >> output done Where the content looks like this: [$p ...
2
votes
0answers
46 views

How to use awk-it in Emacs?

I'm a pretty new Emacs user and I want to call awk from Emacs. After googling around I found awk-it at http://www.emacswiki.org/emacs/AWK_It. However I still can't understand how to use it after I ...
1
vote
0answers
53 views

Is there a reference for the assembler-like output produced by `awk -W dump`?

The command awk -W dump ... produces a dump of "assembler-like" code (as shown here [link], or by reading the man). Some of the instructions are easy to guess at their meaning such as jz, which is ...
1
vote
0answers
33 views

How to extract weather data from a html doc using linux string manipulation

I need to extract these info from this weather forecast site, but I don't know how much about string manipulation in linux: acm_RecentLocationsCarousel.push({name:"Irvine, CA", daypart:'day', ...
1
vote
0answers
36 views

Unix: C-shell:Disk space checking in Hp-UX

i have 10 devices that using hp-ux and i want to check the disk space in each devices. my requirement is if the space more than 90%, the info of device ans space will be save to a log. this is list of ...
1
vote
0answers
104 views

Find and delete lines containing multiple patterns in a file

I have a list of strings and want to find and delete lines containing these strings in a file. A short example of the list of strings is listed as below. File S 1Mo 32,33 2Mo 7,26 2Mo 7,27 2Mo 7,28 ...
1
vote
0answers
28 views

tab-delimiting a file in shell

I have a file that I want to put in a table in SQL with 10 columns. I am trying to tab-delimit the file so that the file can go in the table. Here is part of my txt file: Chr12 MSU_osa1r7 exon ...
0
votes
0answers
74 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
0
votes
0answers
25 views

Canvas(blackboard)-like program

I have to make a program that has most of the functions those websites have and I am stuck at the point where I have to get the grades from a file and use a formula to calculate the students final ...
0
votes
0answers
20 views

awk - how to reorder the contents of a file based on another pattern input file

I have a huge file with a list of names/characters in each line. The good thing is each line is unique, not sorted, but the order makes some sense and same needs to be retained. This is the file to be ...
0
votes
0answers
24 views

How to restore sbatch array job output file names to their original names

In order to submit array jobs using Slurm, I've renamed hundreds of files with incremental numbers using the following command: COUNT=1 for file in ./*; do mv "$file" "${COUNT}" let COUNT++ ...
0
votes
0answers
33 views

Issue with netstat script

I am trying to run the following script to capture the netstat data and put it in to log file. But this is giving error while running the netstat command, however if i directly run the netstat ...
0
votes
0answers
49 views

Sort Checkpoint Firewall log into easier output

I am trying to sort out this log: Aug 27 01:23:38 1.1.1.1/1.1.1.1/1.1.1.1 fw1-loggrabber[0000]: loc=12569|time=2015-08-27 ...
0
votes
0answers
85 views

Convert vnstat log file to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface. I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to ...
0
votes
0answers
40 views

Subtract a Number from variable which contains a sentence

This is extended question from http://stackoverflow.com/questions/32081168/reading-a-log-file-with-different-sequences-using-shell-script I'm new to shell. I've written a shell script a code with ...
0
votes
0answers
28 views

Output a particular character if the same column consists of a few defined characters with AWK

Let say I have a text file as below: G W G A A T G A A T R A T A R A G - G A And I already know that if the same column consist of "A", "G" and "T", I would like to output a "D". For column 2, ...
0
votes
0answers
132 views

How to search for columns that contain specific values

I have a large tab-delimited text file (~40000 columns x ~20000 rows) that I am trying to search through. Each column in the file has a unique identifier in row 1. I have a separate list of a subset ...
0
votes
0answers
193 views

Output of wc | awk is different in shell script than from cli

When issued via command line, this returns "2": pgrep someprocess | wc | awk '{print $1}' When run in a script like this, it returns "4" #!/bin/bash SOME_PROCESS=`pgrep someprocess | wc | awk ...
0
votes
0answers
38 views

to check fields in configuration file

i have configuration file in which first field is filename, second field is the column if we find the value of that column is null in the filename that is given in first column. if any of value of ...
0
votes
0answers
478 views

wget: Retrieving a list of URLs when modifying input data file on the fly

This issue is currently driving me up the wall. It just does not work as it should. I have a file inp with audio samples to download where I preserved the internal ID number by parsing some other ...
-1
votes
0answers
27 views

dump non formated output of commands to csv file

I have some commands with which i can get list of VMs running on hypervisors. I redirect that output to some file.txt and use while loop to read each line from file.txt and execute two more commands ...