awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
1answer
61 views

Solaris: PKG - Script To Verify All Packages Except for a Few

I'm trying to verify the all packages except for a pre-defined list of packages that I know are going to fail for known reasons. This script is going to be run on all Solaris systems within our ...
3
votes
1answer
55 views

Filter out inconsistent data blocks

I would like to filter out rows that has inconsistent geno values in line-part blocks. In case of duplicate consistent values , I would just keep the first row. As example, 'gf345 part1' has more ...
2
votes
1answer
49 views

on-line tally of unique lines

uniq -c is very useful for counting the number of times the same line appears consecutively $ seq 1 1000 | awk '{ if ($1 > 100 && $1 <= 200) { print "hi" } else {print "bye"} }' | uniq ...
1
vote
1answer
88 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
1
vote
1answer
46 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
1
vote
1answer
123 views

how to set a variable to multiple values with awk?

I have a file similar to the following : x 0 1 x 1 1 and I am essentially trying to see if both fields are equal to one another or not for every row. But the problem is that this file contains x ...
1
vote
1answer
69 views

Replace awk and tr with just awk for converting to lowercase

Not a duplicate question, as I am asking for awk specifically. I have the following command to convert everything to lowercase, and then reprint out substrings in a certain order. cat words |tr ...
1
vote
1answer
36 views

display users that logged in since a date using last

I am trying to display the user name of the accounts that have logged in during the past month using the last command. At the moment I have this last | awk '{print $1, $4 ,$5 ,$6}' | grep -B 10 Jul ...
1
vote
1answer
62 views

Find matches from an index file without exact matching and print the last field

I have a large file (~6mill rows) with 2 columns which looks like this: 1111.aaaaabbb.b.cccc.c ValueA 2222.dddddeee.e.ffff.f ValueB 3333.gggghhhh.h.iiii.i ValueC I want to use that as my index ...
1
vote
1answer
41 views

Form a output by combining multiple lines

I have a log file as below: /export/home/got/logs/o2877612.01:job_1432787863184_159408 Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: ...
1
vote
1answer
84 views

How to count the frequency of a word >20

Using exclusively AWK, I need to count the occurrence of a word only when greater than 20 characters. I've already found the way to count the frequency of a word, but I have some problems in setting ...
1
vote
1answer
290 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
1
vote
1answer
113 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
89 views

Unable to replace variable in shell script using awk

I have been struggling to replace every occurence of "<KEY>" in a file with ${encryptedkeyValue} I wrote this line but it is not doing ...
1
vote
1answer
109 views

awk array using number as value did not work

I have a reference file: Refrence File Dpse\GA30012 FBgn0000447 chr2 26607738 26607962 -1 Dpse\GA19764 FBgn0085819 chrX 28571020 28571736 -1 Dpse\ttk FBgn0000100 chr2 ...
1
vote
1answer
120 views

Multiply rows of a column in different chunk of data, add them together and get the average?

I have a chunk of data that looks like this: 924 777 777 777 8445.65349 P 9.44657 -0.840283 -0.394467 C 4.0555 -0.683104 -1.70917 P 7.5169 3.54172 2.77681 C 3.21536 2.67146 2.20242 P 4.33641 7.62012 ...
0
votes
1answer
32 views

Redirect output of: yum repolist all

If you run the command: yum repolist all you will see there are basically four columns of data. repo id, repo name, status, install count. If however your run something like: yum repolist all > ...
0
votes
1answer
34 views

Using Perl for Even/Odd numbers

Currently, I'm using a process to create a very specific set of four files. The Process I'm using is shown in the attached Process Image, below: The first line is part of a solution, given to a ...
0
votes
1answer
44 views

How to make sed as fast as grep when replacing values

If I search with grep in a 30 gig database for some values it finds all of them in no time, if I run the same command with sed to replace those values it takes ages. Why is grep so much faster than ...
2
votes
0answers
52 views

Awk /sed extract information when a pattern match from a paragraph

I want to search a pattern "FROM" in paragraph that begins with CREATE VIEW and ends with ";" and save the result in a csv file. for example if I have the following file : CREATE VIEW view1 AS ...
1
vote
0answers
27 views

Paging with follow less +F and colorize with awk

I would like to paginate a file with follow, i.e. $less +F and also colorize the output on the screen. I want to use less because I can leave the follow mode, browse the file and then re-enter the ...
1
vote
0answers
17 views

Find and redirect to capture pattern filename

I'm looking a way to do something like: find /path -name "201[2-6]*" | awk -F"/" '{print $5"/"$6"/"$7"/"$8",0,0,0"}' > archives_201\1.csv where archives_201\1.csv could be archives_201[2-6].csv ...
1
vote
0answers
178 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
1
vote
0answers
61 views

Is there a reference for the assembler-like output produced by `awk -W dump`?

The command awk -W dump ... produces a dump of "assembler-like" code (as shown here [link], or by reading the man). Some of the instructions are easy to guess at their meaning such as jz, which is ...
1
vote
0answers
46 views

How to extract weather data from a html doc using linux string manipulation

I need to extract these info from this weather forecast site, but I don't know how much about string manipulation in linux: acm_RecentLocationsCarousel.push({name:"Irvine, CA", daypart:'day', ...
1
vote
0answers
47 views

Unix: C-shell:Disk space checking in Hp-UX

i have 10 devices that using hp-ux and i want to check the disk space in each devices. my requirement is if the space more than 90%, the info of device ans space will be save to a log. this is list of ...
1
vote
0answers
34 views

tab-delimiting a file in shell

I have a file that I want to put in a table in SQL with 10 columns. I am trying to tab-delimit the file so that the file can go in the table. Here is part of my txt file: Chr12 MSU_osa1r7 exon ...
1
vote
0answers
59 views

How to use awk-it in Emacs?

I'm a pretty new Emacs user and I want to call awk from Emacs. After googling around I found awk-it at http://www.emacswiki.org/emacs/AWK_It. However I still can't understand how to use it after I ...
0
votes
0answers
35 views

find awk log print error

I'm trying to find one year old directories named in yyyymmdd format and print it in the log file placed at different location. find ${LOG} -maxdepth 1 -type d -mtime +${DAY} -name ...
0
votes
0answers
33 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...
0
votes
0answers
41 views

awk, how to split on special characters

Background Using awk {GNU Awk 3.1.7} Must split log file with awk on special characters: (id= To illustrate my problem This works: echo "aa_bc" | awk '{{split($0,var,"_")} {print ...
0
votes
0answers
73 views

Replace column in one file with column from another using awk? where column from another may have null values

I have two files: f1: 111|aaa|444 222|bbb|555 333|ccc|666 f2: 111 333 replacing column 2 of f1 with column 1 of f2 required output new: 111|111|444 222||555 333|333|666 I am using ...
0
votes
0answers
60 views

Need some help on awk/grep command on shell script

This is in reference to Need some help on grep command on shell script I have a log file and I am trying to write a shell script to find pause/play happening or not (Success /Failure). The script ...
0
votes
0answers
21 views

Order Columns in CSV file

I am trying to order all columns by column name in a csv file. What I have is something like this: name ,adress ,mobile-number Ane ,USA ,12121212 Joane ,England ,234234 and the output I ...
0
votes
0answers
43 views

Standard deviation of number next to a pattern

I have a file with lots of tags with a number next to them, set out like this: <Score>4 ... other data <Score>2 ... other data <Score>3 and so on... I have used grep to get the ...
0
votes
0answers
16 views

Loop through directory and sum value next to specific pattern producing an average

I want to loop through all the files in the directory. the files are set out like this: <Overall>4 other data <Overall>2 other data ...... I have the code: for file in .dat; do awk ...
0
votes
0answers
29 views

Canvas(blackboard)-like program

I have to make a program that has most of the functions those websites have and I am stuck at the point where I have to get the grades from a file and use a formula to calculate the students final ...
0
votes
0answers
24 views

awk - how to reorder the contents of a file based on another pattern input file

I have a huge file with a list of names/characters in each line. The good thing is each line is unique, not sorted, but the order makes some sense and same needs to be retained. This is the file to be ...
0
votes
0answers
35 views

How to restore sbatch array job output file names to their original names

In order to submit array jobs using Slurm, I've renamed hundreds of files with incremental numbers using the following command: COUNT=1 for file in ./*; do mv "$file" "${COUNT}" let COUNT++ ...
0
votes
0answers
54 views

Issue with netstat script

I am trying to run the following script to capture the netstat data and put it in to log file. But this is giving error while running the netstat command, however if i directly run the netstat ...
0
votes
0answers
72 views

Sort Checkpoint Firewall log into easier output

I am trying to sort out this log: Aug 27 01:23:38 1.1.1.1/1.1.1.1/1.1.1.1 fw1-loggrabber[0000]: loc=12569|time=2015-08-27 ...
0
votes
0answers
120 views

Convert vnstat log file to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface. I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to ...
0
votes
0answers
44 views

Subtract a Number from variable which contains a sentence

This is extended question from http://stackoverflow.com/questions/32081168/reading-a-log-file-with-different-sequences-using-shell-script I'm new to shell. I've written a shell script a code with ...
0
votes
0answers
30 views

Output a particular character if the same column consists of a few defined characters with AWK

Let say I have a text file as below: G W G A A T G A A T R A T A R A G - G A And I already know that if the same column consist of "A", "G" and "T", I would like to output a "D". For column 2, ...
0
votes
0answers
215 views

How to search for columns that contain specific values

I have a large tab-delimited text file (~40000 columns x ~20000 rows) that I am trying to search through. Each column in the file has a unique identifier in row 1. I have a separate list of a subset ...
0
votes
0answers
255 views

Output of wc | awk is different in shell script than from cli

When issued via command line, this returns "2": pgrep someprocess | wc | awk '{print $1}' When run in a script like this, it returns "4" #!/bin/bash SOME_PROCESS=`pgrep someprocess | wc | awk ...
0
votes
0answers
38 views

to check fields in configuration file

i have configuration file in which first field is filename, second field is the column if we find the value of that column is null in the filename that is given in first column. if any of value of ...
0
votes
0answers
605 views

wget: Retrieving a list of URLs when modifying input data file on the fly

This issue is currently driving me up the wall. It just does not work as it should. I have a file inp with audio samples to download where I preserved the internal ID number by parsing some other ...
-1
votes
0answers
35 views

how to pass multiple column value in a command

I have a file oldnewmap.txt. This file has hundreds of records like the following, where column one represents oldnumber and column two represents newnumber. 21382,45678 34567,45663 44245,42455 ...
-4
votes
0answers
41 views

Join column with columns of next two line in shell script

I want to join value of the first line with the value of next two line. If some value includes number (G, G1, G2) in input data, It should be considered for all combinations. For example, input ...