awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
335 views

awk date identification from input file

I currently have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new ...
-3
votes
2answers
76 views

How to get a device details only using awk commands

Friends, I have fdisk -l output in one file called test1.txt. Now I want to get the device name only using awk. How to get a device name? I want output like this: /dev/sda1 /dev/sda2 /dev/sda3 ...
4
votes
3answers
7k views

Adding a Column of values in a tab delimited file

How can I add a Column of values in a file which has a certain number of rows. I have a input file like this: Input file: SPATA17 1 217947738 LYPLAL1 1 219383905 FAM47E 4 77192838 SHROOM3 4 ...
0
votes
2answers
115 views

How to extract submatches from a string and write those to a file?

I want to parse a string for specific submatches and output those - one submatch per line - into a file. Essentially I would like to extract the value of each href-attribute: $ cat Fin <a ...
0
votes
3answers
399 views

removing spaces from first column

My input file has positions in first column with different number of spaces (or no space) 16504 16516 1650811 16520 1651 16524 16516111 16528 165204 16532 ...
1
vote
3answers
2k views

How can I remove the text from a line after a certain character with awk

How can I use awk to remove all text after a certain character ";" that appears on every line of my text file? (I then need to run for loops on the text) Jenny,Sarah,John;North Dakota ...
4
votes
2answers
150 views

How can I delete all unordered lines from a text file?

Overview Consider an ordered list interspersed with unordered elements, e.g.: Alligator Ant Falcon <-- Baboon Badger Armadillo <-- Caiman Cat How can this list be processed so that all ...
2
votes
1answer
328 views

Awk regular expression syntax with number of repetition - different handling between gawk 3 and gawk 4

I am having trouble understanding or explaining why the following expression fails in gawk 3.1.x and yet works in gawk 4.1.x: (Minimum working example) echo ";#ADCDE#" | awk '/#.{5}#$/' -> produces ...
2
votes
2answers
101 views

/etc/shadow & gawk

I ran this small script on /etc/shadow: awk '{ FS = ":" } { print $1, $2 }' /etc/shadow It does what it should on all lines except for the root's record I.e. it just prints the entire record. Know ...
2
votes
3answers
94 views

How to optimize this oneliner? (if second column empty, put the line in another file)

I have a little "oneliner": while read ONELINE; do if [[ -n "`echo $ONELINE | awk '{print $2}'`" ]]; then echo "$ONELINE" >> LIST-ok.txt else echo "$ONELINE" >> ...
2
votes
1answer
408 views

Awk field separator bug?

This is the expected way for the awk field separator to work: $ echo 'fooXbar' | awk 'BEGIN {FS="X"} {print $1}' foo $ echo 'fooXbar' | awk 'BEGIN {FS="X"} {print $2}' bar $ but if the FS is ...
1
vote
3answers
81 views

parsing file from ls -laLR

ls -laLR /home/tools > all_site1.txt I need the state of all folders and files in subfolders of /home/tools. With this command I put all in this file. What I need now is a solution how to parse ...
5
votes
3answers
179 views

Increment numbers inside strings by an offset

I want to search a text file for strings like these. [task0] [task1] ... [task1613] and increment each by a given offset, in this case by 3640. So, with the example I gave above, I'd get the ...
1
vote
2answers
2k views

How do I multiply and sum column data using awk and or sed?

I have several columns of data. I always have the same number of rows (say 5). In the 2nd column, I want to multipy the first value by 5, then the second value by 4, the third value by 3, etc. I then ...
2
votes
3answers
1k views

How to show CPU time for processes via top without 'root' procs

Okay, So I've been attempting this for about three hours now without success. How do you search/display/use the top command (or ps command if it works...) to output a list of all procs, sorted by CPU ...
5
votes
4answers
2k views

number parsing in awk

I have a text file with numbers in second column and the numbers have thousand-separator and decimal-separator: 2014-01 2,277.40 2014-02 2,282.20 2014-03 3,047.90 2014-04 4,127.60 ...
3
votes
2answers
109 views

Command in variable one are affecting output of other variable when displayed together in UNIX

I am having two file naming Extra_info and Class_info Class_info contains data : stuId stuName stuClass S01 jagdeep First S02 harsh First S03 Roop Second S04 Kali Third ...
2
votes
4answers
5k views

awk + print awk output in one line

When I run the following hasys command with awk , I get output with machines names as the following: hasys -display|grep Shutdown | awk '{print $1}' machine1a machine1b please advice how to ...
2
votes
4answers
280 views

Count rows with specific integer in a column

I have 6 columns, each with multiple rows. I want to count the number of rows which have the integer 4 or 5 in the fifth column. A1 jhfj jdhfjkhd kdkfjjh 5 jhsdjkfh A2 ujhf jhdfhsd dsfkks 4 ...
2
votes
1answer
313 views

What is an array in awk?

I'm very new to unix and am trying to learn basic awk scripting - during which I keep coming across the word "array", and "associative arrays". I have no clue as to what this is, and don't tell me to ...
4
votes
4answers
1k views

Print line after nth occurrence of a match

I am looking to display the line 4598 in the following file. Effectively I want to display the line AFTER the nth occurrence of a match. In this case, the line after the 3rd occurrence of <Car>. ...
4
votes
5answers
113 views

Reformat each two-line sequence into two columns

I would like to tab my printout: 1 2 3 4 5 6 7 8 to 1 2 5 6 3 4 7 8 How to adjust it using simple command line (awk or sed)??
-2
votes
3answers
96 views

Deleting Lines, which match a particular Identifier from another file

I have 2 files. File 1, has an identifier (eg. D7MHBF:11:1449:1988) and every new entry starts with @. It has few more fields, which are not important in our analysis. File 2 consists of a column of ...
3
votes
3answers
236 views

Easily get a particular column from output without sed or awk

Is there a quicker way of getting a couple of column of values than futzing with sed and awk? For instance, if I have the output of ls -hal / and I want to just get the file and directory names and ...
1
vote
2answers
231 views

comparing user given time for log file time

Apologies straight away for bad-styled-question-and-heading.....learning unix and its command these days, m trying to figure out a solution for below scenario : Lets say i have a log file which ...
0
votes
4answers
95 views

What are alternative tools for searching files in full content?

I realized that most of the text related tools I use, grep/egrep/awk/sed are line-oriented. I guess that is a simplified approach to search in files but I was wondering if there is another set of ...
1
vote
1answer
138 views

pcregrep fails due to segmentation fault

I am extracting the HTML source of a web page using the below command. curl http://www.linkedin.com/pub/ramesh-venkataraman/63/621/b42 -s | w3m -dump -T text/html > foo.txt The foo.txt has the ...
2
votes
3answers
1k views

Print Column if values is between two other values

I am fairly new to data analysis and I have a problem with filtering some of my values. I have data arranged in a file with four columns like this: A 1 10 5 B 10 100 120 C ...
1
vote
1answer
280 views

Is Perl ever a better tool than awk for text processing [closed]

This question was inspired by a discussion on stackoverflow - see in particular the comment thread under Ed Morton's answer. I figured this was the site where I could get a definitive answer: In ...
1
vote
1answer
240 views

How to get the JSON STRING from the given string

I have a string given below. I just need to get the first accountHeader (bolded) JSON string, I tried everything.. It didn't work. Can you help me to get the data? <START ...
0
votes
4answers
843 views

Removing certain fields from a CSV file

I have a large CSV file (~15GB) which a client has sent over as an export of a table. The format is a follows: field, ... , field, field which sometimes contains commas, another field which sometimes ...
5
votes
2answers
259 views

how to add command line calculator to bash that uses comma as decimal mark?

How can I add a command line calculator to my bash? I have found some, but all of them use the full stop as decimal mark, but I want to have it to use the comma as decimal mark as most of the world ...
2
votes
2answers
1k views

Do I need to encapsulate awk variables in quotes in order to sanitize them?

As per an answer on stackoverflow, it's my understanding that encapsulating bash variables in double-quotes is a fairly safe way of sanitizing user input. What about awk variables? For example, if ...
5
votes
2answers
915 views

Add thousands separator in a number

In python re.sub(r"(?<=.)(?=(?:...)+$)", ",", stroke ) To split a number by triplets, e.g.: echo 123456789 | python -c 'import sys;import re; print re.sub(r"(?<=.)(?=(?:...)+$)", ",", ...
2
votes
3answers
8k views

How to merge two files based on the matching of two columns?

I have file1 likes: 0 AFFX-SNP-000541 NA 0 AFFX-SNP-002255 NA 1 rs12103 0.6401 1 rs12103_1247494 0.696 1 rs12142199 0.7672 And a file2: 0 AFFX-SNP-000541 1 0 ...
1
vote
4answers
3k views

Finding out the common lines in two files using 4 fields using awk and UNIX

I have 2 files. If field 1, 2, 4 and 5 of both file1 and file2 matches, then I want to print the whole line of file1 and file2 one after another in my output file. File1: sc2/80 20 . ...
2
votes
1answer
72 views

Why does AWK not like dots in one of two substitutions?

I'm trying to use AWK in a bash script to look for a placeholder string in a template file and replace it with the contents of a variable that (can) contain various special characters. Example: ...
3
votes
1answer
143 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
4
votes
2answers
82 views

awk pairwise differences

I run into this problem a lot with data, if I have some integers, I frequently want to look at differences between adjacent integers, I usually solve this in Ruby or Python, but I think such a thing ...
1
vote
1answer
44 views

Unable to unregister vms

I have the following command to extract the name of virtual machines currently available in the host system. vmslist=$(vboxmanage list vms | awk '{print $1}') I pass this list to a for loop in ...
0
votes
3answers
660 views

Awk/grep/sed get comma separated list of numbers from lines of text

I have a log file that has lines that look like this: blah blah blah Photo for (702049679 - blah blah blah Now I know I could get all the lines like that from the file by doing: grep "Photo for" ...
5
votes
2answers
550 views

Gawk: Passing arrays to functions

Stuck with GNU awk 3.1.6 and think I've worked around its array bugs but still have what looks like a scope problem in a 600-line awk program. Need to verify understanding of array scope in awk to ...
1
vote
2answers
2k views

what is awk '{print $1+0.45 “ ” $2 “ ” $3 }' file1> file2?

What does awk '{print $1+0.45 " " $2 " " $3 }' positionXYZ > positionX0.45YZ mean? Does it mean changing a row in the first file and put the new data into the second one?
0
votes
2answers
1k views

Using AWK to delete line and output result

I have a text file and manage to find the line through output comparison. With the book title and author name. awk -F ':' '/KungFu Feet|Chuck Norris/' test.txt Kungfu Feet:Chuck Norris:12:1:1 Is ...
1
vote
7answers
3k views

Sed / Awk - Find string exactly n characters long

Looking through the man pages for sed, awk and grep, I'm unable to find a way to search for a string that is exactly n characters long. Given the following text file, I want to extract just ...
3
votes
2answers
821 views

Bash Script To Find Debian Release Number from /etc/debian_version

The following is what I'm currently using for testing the version of Debian. Instead of doing it this way, can I make it more succinct? #!/bin/bash VERSION=NONE if [ $(cat /etc/debian_version) == ...
2
votes
3answers
397 views

To assign variable from a file separated by delimiter

I have a text file with datas as below separated by delimiter: 1000:Tom:user1 1002:Jerry:user2 1003:Victor:user3 1004:Jason:user4 I need to run a while loop (any loop) to fetch details from first ...
5
votes
3answers
201 views

The ERE regex to split() string between a delimiter and end-of-word

I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem... Sample ASCII input... [[link|label label]] [[link]] ...
4
votes
2answers
2k views

System calls, AWK, and Bringing in external inputs

awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this? What I'm aiming to ...
-1
votes
1answer
249 views

extracting ip address from a file [closed]

I'm trying to extract all the IP address from a file. FILE *>i85.95.2.160/29 172.16.24.4 0 200 0 64515 4445 15412 i *>i85.112.80.0/28 172.16.24.4 0 200 ...