awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

4
votes
1answer
65 views

Solaris: PKG - Script To Verify All Packages Except for a Few

I'm trying to verify the all packages except for a pre-defined list of packages that I know are going to fail for known reasons. This script is going to be run on all Solaris systems within our ...
3
votes
1answer
59 views

Filter out inconsistent data blocks

I would like to filter out rows that has inconsistent geno values in line-part blocks. In case of duplicate consistent values , I would just keep the first row. As example, 'gf345 part1' has more ...
2
votes
1answer
29 views

Merging specific columns of 2 files based on field match

I know I'm not the first one asking this but my code still does not work: File 1: gi|1283| tRNAscan exon 87020 88058 . - . transcript_id "Parent=tRNA-Tyr5.r01"; gi|3283| tRNAscan exon 97020 97058 . +...
2
votes
1answer
141 views

Read from and append to file at the same time while preserving end-of-line

Conceptually I have an easy task... I have [loosely] structured data in a file: Testing: debug, default CXXFLAGS <100's of additional output lines> Testing: release, default CXXFLAGS <100's ...
1
vote
1answer
122 views

I need to merge multiple .csv files together side by side with unequal number of rows

I have anywhere from 3-5 .csv files and need to be able to merge them together while keeping everything in its respective columns below is an easy example with files with varying number of rows. file1 ...
1
vote
1answer
52 views

how can i comment out/in a section in source code with sed or awk

Currently i have files, which contain sections like this code statement1 code statement2 # BEGIN SOMENAME some code some other code # END SOMENAME code statement n +1 code statement n +1 What i ...
1
vote
1answer
130 views

how to set a variable to multiple values with awk?

I have a file similar to the following : x 0 1 x 1 1 and I am essentially trying to see if both fields are equal to one another or not for every row. But the problem is that this file contains x ...
1
vote
1answer
72 views

Replace awk and tr with just awk for converting to lowercase

Not a duplicate question, as I am asking for awk specifically. I have the following command to convert everything to lowercase, and then reprint out substrings in a certain order. cat words |tr '[:...
1
vote
1answer
41 views

display users that logged in since a date using last

I am trying to display the user name of the accounts that have logged in during the past month using the last command. At the moment I have this last | awk '{print $1, $4 ,$5 ,$6}' | grep -B 10 Jul |...
1
vote
1answer
64 views

Find matches from an index file without exact matching and print the last field

I have a large file (~6mill rows) with 2 columns which looks like this: 1111.aaaaabbb.b.cccc.c ValueA 2222.dddddeee.e.ffff.f ValueB 3333.gggghhhh.h.iiii.i ValueC I want to use that as my index ...
1
vote
1answer
41 views

Form a output by combining multiple lines

I have a log file as below: /export/home/got/logs/o2877612.01:job_1432787863184_159408 Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: /user/got/...
1
vote
1answer
85 views

How to count the frequency of a word >20

Using exclusively AWK, I need to count the occurrence of a word only when greater than 20 characters. I've already found the way to count the frequency of a word, but I have some problems in setting ...
1
vote
1answer
311 views

sort file based one colum and group by another column

I have a file of three columns: 1 A 0.5 2 B 0.7 3 A 10 4 C 4 5 B 4 I want to sort the file by increasing order of column 3 and group by column 2 1 A 0.5 3 A 10 ...
1
vote
1answer
113 views

Shell intervals using awk

I have a file a 10:10 10:30 b 12:15 13:15 c 16:54 17:54 and I need to enter interval for example 10:15 so my output should be a 10:10 10:30 How I need to do that using awk?
1
vote
1answer
102 views

Unable to replace variable in shell script using awk

I have been struggling to replace every occurence of "<KEY>" in a file with ${encryptedkeyValue} I wrote this line but it is not doing ...
1
vote
1answer
111 views

awk array using number as value did not work

I have a reference file: Refrence File Dpse\GA30012 FBgn0000447 chr2 26607738 26607962 -1 Dpse\GA19764 FBgn0085819 chrX 28571020 28571736 -1 Dpse\ttk FBgn0000100 chr2 ...
1
vote
1answer
124 views

Multiply rows of a column in different chunk of data, add them together and get the average?

I have a chunk of data that looks like this: 924 777 777 777 8445.65349 P 9.44657 -0.840283 -0.394467 C 4.0555 -0.683104 -1.70917 P 7.5169 3.54172 2.77681 C 3.21536 2.67146 2.20242 P 4.33641 7.62012 ...
0
votes
1answer
40 views

How to get the data between specified Starttime and Endtime in a log file?

This is my LogFile(Test.Log): May 10 01:06:05 A May 10 01:06:06 B May 10 01:06:07 C May 10 01:07:01 D May 10 01:07:02 E May 10 01:07:09 F May 10 01:07:11 G May 10 01:07:11 H May 10 01:07:12 I May 10 ...
0
votes
1answer
41 views

How to use TWO “for statements” in a shell script?

Writing script to create user accounts. Script is written to echo the useradd command (so I can make sure it will work as expected). I have a file newemploy.txt with Employee Names (first and last, ...
2
votes
0answers
54 views

how to capture the latest kernel version from grub.conf

how to capture the latest kernel version from grub.conf ( in case default=0) for example the latest kernel from the file is 2.6.18-409.el5 # all kernel and initrd paths are relative to /...
2
votes
0answers
55 views

Awk /sed extract information when a pattern match from a paragraph

I want to search a pattern "FROM" in paragraph that begins with CREATE VIEW and ends with ";" and save the result in a csv file. for example if I have the following file : CREATE VIEW view1 AS ...
1
vote
0answers
29 views

Paging with follow less +F and colorize with awk

I would like to paginate a file with follow, i.e. $less +F and also colorize the output on the screen. I want to use less because I can leave the follow mode, browse the file and then re-enter the ...
1
vote
0answers
17 views

Find and redirect to capture pattern filename

I'm looking a way to do something like: find /path -name "201[2-6]*" | awk -F"/" '{print $5"/"$6"/"$7"/"$8",0,0,0"}' > archives_201\1.csv where archives_201\1.csv could be archives_201[2-6].csv ...
1
vote
0answers
196 views

How to redirect linux command outputs to specific row and column in csv?

I'm working on Unix Build post documentation process, I have a CSV file to be filled in and uploaded for every new build, to ensure the servers are build as per the original request. The CSV file ...
1
vote
0answers
61 views

Is there a reference for the assembler-like output produced by `awk -W dump`?

The command awk -W dump ... produces a dump of "assembler-like" code (as shown here [link], or by reading the man). Some of the instructions are easy to guess at their meaning such as jz, which is ...
1
vote
0answers
51 views

How to extract weather data from a html doc using linux string manipulation

I need to extract these info from this weather forecast site, but I don't know how much about string manipulation in linux: acm_RecentLocationsCarousel.push({name:"Irvine, CA", daypart:'day', ...
1
vote
0answers
48 views

Unix: C-shell:Disk space checking in Hp-UX

i have 10 devices that using hp-ux and i want to check the disk space in each devices. my requirement is if the space more than 90%, the info of device ans space will be save to a log. this is list of ...
1
vote
0answers
61 views

How to use awk-it in Emacs?

I'm a pretty new Emacs user and I want to call awk from Emacs. After googling around I found awk-it at http://www.emacswiki.org/emacs/AWK_It. However I still can't understand how to use it after I ...
0
votes
0answers
30 views

get user input in terms of size (GB or MB)

I created script to calculate shmmax shmall shmmini Total_Memory_M=$(dmidecode -t 17 | grep Size | awk '{print $2}' | grep -o '[0-9]\+' | paste -sd+ | bc) Total_Memory_G=$(echo $Total_Memory_M /1024 |...
0
votes
0answers
43 views

find awk log print error

I'm trying to find one year old directories named in yyyymmdd format and print it in the log file placed at different location. find ${LOG} -maxdepth 1 -type d -mtime +${DAY} -name '20[0-9][0-9][0-9][...
0
votes
0answers
36 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...
0
votes
0answers
42 views

awk, how to split on special characters

Background Using awk {GNU Awk 3.1.7} Must split log file with awk on special characters: (id= To illustrate my problem This works: echo "aa_bc" | awk '{{split($0,var,"_")} {print substr(var[1],1)}}...
0
votes
0answers
88 views

Replace column in one file with column from another using awk? where column from another may have null values

I have two files: f1: 111|aaa|444 222|bbb|555 333|ccc|666 f2: 111 333 replacing column 2 of f1 with column 1 of f2 required output new: 111|111|444 222||555 333|333|666 I am using ...
0
votes
0answers
60 views

Need some help on awk/grep command on shell script

This is in reference to Need some help on grep command on shell script I have a log file and I am trying to write a shell script to find pause/play happening or not (Success /Failure). The script ...
0
votes
0answers
22 views

Order Columns in CSV file

I am trying to order all columns by column name in a csv file. What I have is something like this: name ,adress ,mobile-number Ane ,USA ,12121212 Joane ,England ,234234 and the output I ...
0
votes
0answers
43 views

Standard deviation of number next to a pattern

I have a file with lots of tags with a number next to them, set out like this: <Score>4 ... other data <Score>2 ... other data <Score>3 and so on... I have used grep to get the ...
0
votes
0answers
16 views

Loop through directory and sum value next to specific pattern producing an average

I want to loop through all the files in the directory. the files are set out like this: <Overall>4 other data <Overall>2 other data ...... I have the code: for file in .dat; do awk '...
0
votes
0answers
29 views

Canvas(blackboard)-like program

I have to make a program that has most of the functions those websites have and I am stuck at the point where I have to get the grades from a file and use a formula to calculate the students final ...
0
votes
0answers
24 views

awk - how to reorder the contents of a file based on another pattern input file

I have a huge file with a list of names/characters in each line. The good thing is each line is unique, not sorted, but the order makes some sense and same needs to be retained. This is the file to be ...
0
votes
0answers
37 views

How to restore sbatch array job output file names to their original names

In order to submit array jobs using Slurm, I've renamed hundreds of files with incremental numbers using the following command: COUNT=1 for file in ./*; do mv "$file" "${COUNT}" let COUNT++ ...
0
votes
0answers
57 views

Issue with netstat script

I am trying to run the following script to capture the netstat data and put it in to log file. But this is giving error while running the netstat command, however if i directly run the netstat ...
0
votes
0answers
75 views

Sort Checkpoint Firewall log into easier output

I am trying to sort out this log: Aug 27 01:23:38 1.1.1.1/1.1.1.1/1.1.1.1 fw1-loggrabber[0000]: loc=12569|time=2015-08-27 01:23:37|action=block|orig=192.168.1.1|i/f_dir=outbound|i/f_name=eth1-04|...
0
votes
0answers
123 views

Convert vnstat log file to csv

My goal is to log daily network activity (rx/tx) from each ethernet interface. I installed vnstat, and while it reports daily and monthly stats, it doesn't offer a nice way to convert to csv to ...
0
votes
0answers
45 views

Subtract a Number from variable which contains a sentence

This is extended question from http://stackoverflow.com/questions/32081168/reading-a-log-file-with-different-sequences-using-shell-script I'm new to shell. I've written a shell script a code with ...
0
votes
0answers
30 views

Output a particular character if the same column consists of a few defined characters with AWK

Let say I have a text file as below: G W G A A T G A A T R A T A R A G - G A And I already know that if the same column consist of "A", "G" and "T", I would like to output a "D". For column 2, ...
0
votes
0answers
251 views

How to search for columns that contain specific values

I have a large tab-delimited text file (~40000 columns x ~20000 rows) that I am trying to search through. Each column in the file has a unique identifier in row 1. I have a separate list of a subset ...
0
votes
0answers
265 views

Output of wc | awk is different in shell script than from cli

When issued via command line, this returns "2": pgrep someprocess | wc | awk '{print $1}' When run in a script like this, it returns "4" #!/bin/bash SOME_PROCESS=`pgrep someprocess | wc | awk '{...
0
votes
0answers
38 views

to check fields in configuration file

i have configuration file in which first field is filename, second field is the column if we find the value of that column is null in the filename that is given in first column. if any of value of ...
0
votes
0answers
631 views

wget: Retrieving a list of URLs when modifying input data file on the fly

This issue is currently driving me up the wall. It just does not work as it should. I have a file inp with audio samples to download where I preserved the internal ID number by parsing some other ...
-1
votes
0answers
24 views

How to copy data from a text file to a particular Excel row (or) column using shell script

I am facing the problem in copying text data into a particular Excel column using a shell script. I have used the following command awk -v cols=6 '{printf "%s%s", $1, (NR % cols == 0 ? "\n" : "\t")}...