awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
85 views

Duplicate specific text in text file, using perl or grep

I need to duplicate part of a plain text file as follows: The text file consists of a sequence of "pages", of the form: <page [lots of lines of text] /page> <page [lots of ...
3
votes
2answers
80 views

how to grep complex hierarcy?

I have a file as:- start apple 1 a 2 b 3 c start orange 4 a 5 b start mango 1 a start a/b/c 5 z end 4 b end 6 c end start banana 3 c end 4 d 5 e end I want the output as :- 1 apple/a 2 apple/b 3 ...
-3
votes
1answer
156 views

how to select specific range of line & count the specific occurence of the first charcter w.r.t. each unique second charcter? [closed]

Hi I have a file like this : #0 0:() 1b:cg*b 1c 0:cg xe #4 0:() 0b:cg*b xc 0:cg 1e #8 0:() 0b:cg*b xc 0:cg xe #12 1b:cg*b xc 0:cg 0e #16 xb:cg*b 1c xe #20 1:() xb:cg*b xc 1:cg 1e #24 x:() xb:cg*b xc ...
2
votes
3answers
595 views

How to copy or read the return value (exit status) from AWK script into the shell script (bash) to be used in if statement to compare

I want to read the error codes or the exit status of awk script into shell to use in if condition to compare for error handling my shell script looks something like: My script: #!/bin/sh awk -v ...
4
votes
4answers
644 views

How to remove duplicate lines with awk whilst keeping empty lines?

Below awk command removes all duplicate lines as explained here: awk '!seen[$0]++' If the text contains empty lines, all but one empty line will be deleted. How can I keep all empty lines whilst ...
0
votes
2answers
76 views

Formatting numbers using awk / print

I found this very cool code to get upload/download speed: awk '{ if (l1) { print "↓"($2-l1)/1024"kB/s ","↑"($10-l2)/1024"kB/s" } else { l1=$2; l2=$10; } }' <(grep wlan0 ...
0
votes
1answer
53 views

change column with awk scriting

I am trying to make a script B that uses another script A to change a specific column in a specific line. The problem is that when I run the script A alone it runs perfectly but when I try to use the ...
2
votes
4answers
171 views

sed remove end of line for specific lines

I'm trying to find the sed command to remove the end of the line (meaning, get the next line to the current), but only if line starts with the string NOTOK. example input: NOTOK something ...
3
votes
6answers
384 views

Add a symbol before any letter

I have a text file with each line having some defined number of fields but they can differ from line to line. All I want to do is to add "=" sign in front of every value in the field in that line. ...
1
vote
2answers
77 views

Renaming a whole folder content

I'm on OSX 10.9.2 (BASH 3.5) and I wanted to rename a whole folder content names. So I run a script and somehow ended with this: s01eDavid.Letterman.2014.02.27.Stupid.Human.Tricks.x264-SWOLLED.mp4 ...
-4
votes
1answer
115 views

Write a script file 'account.awk' that from a file 'orders' creates the formated printout of account

Write a script file account.awk that from a file orders creates the formatted printout of account. The file orders is as follows: http://pastebin.com/kPF5B0c0 Beside the column Cost the account ...
1
vote
4answers
318 views

Pattern Matching and Delete the whole line

I want to delete all the Lines of File 1, if Column1 of File1 matches exactly with Column 1 File2. File 1: r001:21:10 21 AAAAAATTTGC * = XM:21 r002:21:10 21 YAAAATTTGC * ...
1
vote
3answers
1k views

Delete lines from a file using awk or sed based on the variable from previous command

I have a CSV file (say temp.csv) with strings and numbers. I need to delete the lines which contain a particular pattern of string at the beginning of the lines in the file. For example, here is my ...
-2
votes
3answers
69 views

Can anyone please explain me below AWK script

format of flat file : ;metier_code ; ;-------------------------; (0 rows affected) ;CRDS_Ptf_No; ; ; ; ; ...
0
votes
3answers
85 views

Copying specific text from one file into a schema

I have two files. One being a schema, and the other a list of data. The schema looks like: line_name: shot_edit: channel edit: cable_edit and repeats regularly for the number of lines that I ...
3
votes
1answer
212 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
2
votes
5answers
244 views

Can the string “abcedf” be matched to string “bafcde” in a single line command?

I am planning to implement an indexing structure in my program. For example, if I have 100 rows in the table, I will number these rows from 1 to 100 in another column by appending an _ to the end of ...
1
vote
1answer
39 views

Tile logfile into pieces and search for errors

I want to tile a logfile into pieces and then search in this pieces for errors. If there is an error, I want to put the relevant logfile-piece into another file. Little example: Logfile looks like ...
1
vote
2answers
112 views

How can I format a flat file using awk?

Below is the format of some table data contained within a single file: ;NULL;ABCD;ABHJARS;;ABCD;ABCD;Y;;;;;;;;;A; ;NULL;XEU-ANKD;XEU-AJKD;;ABCD;ABCD;Y;;;;;;;;;A; . . ;11744;AMKDIONSKH;AMKDJ AN DJ ...
0
votes
2answers
72 views

Method to paste specific column when in terminal

I'd like to paste only the third column of data from within the terminal. I originally have something like: 4 SF0086 SF0086 12 SF0087 SF0087 17 TSF141 TSF141 29 TSF070 TSF070 30 TSF028 TSF028 31 ...
4
votes
4answers
563 views

remove blank lines in awk display

I was trying to figure out a solution for this question. I wanted to use awk for the solution. My input file is something like below. -bash-3.2$ cat file ramesh ramesh_venkat ramesh3_venkat3 ...
0
votes
1answer
71 views

Please explain the Awk script provided below

Can anyone please expain me step by step below AWK script written. I have the below code written in my script to format the flat file data. Just wanted to understand so that i can reuse -- I am not a ...
6
votes
3answers
520 views

Is there a perl or awk solution for this problem?

I have an input file (input.txt) like below. id1 id2 name weight 53723848 12651711 timburnes 1.36667 53530214 12651711 timburnes 1.51191 53723848 53530214 timburnes 1.94 764157 ...
-1
votes
1answer
56 views

Unix Awk group by

Can anyone shed me light how to group by based on the username and sum the values divide it by number of rows of that specific username. for example below. root 25 oracle 25 batch 30 griduser 20 ...
0
votes
2answers
134 views

what does getline do inside awk?

I have this following awk script that takes the following input file, input.txt and produces the output below. Can someone please take the time to break down how this awk script works? I've spent a ...
4
votes
4answers
736 views

How to remove character and space from a string

Hello I have one file with output Name : ABC Name : CDE Name : ZYS I want to get the result ABC CDE ZYS How can i get it through SED command. I have tried. sed s/'Name' but ...
1
vote
1answer
91 views

Using awk for data manipulation

I have an input file in the following format. #track chr11 61731756 61735132 FTH1 - chr12 6643584 6647537 GAPDH + chr11 18415935 18429765 LDHA + chr12 21788274 21810728 ...
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...
1
vote
1answer
118 views

Parse/Manipulate in awk

I am new to awk programming. I have a question on manipulating text file, which is required to draw certain network based images in a visualization software (Circos http://circos.ca) I have input ...
2
votes
1answer
92 views

Pipe hex characters to awk

This command works as expected $ awk 'BEGIN {print "\x41"}' A As does this $ printf '\x41' | awk '{print}' A However when you rely on awk to recognize the hex characters, it fails $ echo '\x41' ...
6
votes
5answers
1k views

Find images by size: find / file / awk

I've been trying to find png image files a certain height (over 500px). I know that file will return image dimensions. Example: $ file TestImg1a.png TestImg1a.png: PNG image data, 764 x 200, ...
1
vote
2answers
72 views

Parse,Manipulate Data In linux/Unix

I want to have exactly the same output in this post(http://unix.stackexchange.com/questions/126139/extract-data-in-linux-unix)as above,but I have a new input file,which is below: ABCB11 4 ACE 11 ...
1
vote
2answers
167 views

Extract data in linux/unix

I have input data which I want to parse and extract values using awk/grep/sed: group-2 9 10 text color=black,from=ABCB11,fromid=4,order=2,thickness=3,to=ACE,toid=11,use=1,z=1 group-3 0 1 text ...
7
votes
5answers
556 views

SED : Case Match Pattern Replacement

I have a source code spread across several files. It has a pattern abcdef which I need to replace with pqrstuvxyz. The pattern could be Abcdef (Sentence Case) then it needs to be replaced with ...
2
votes
1answer
277 views

Regex doesn't work with grep

I have a string like this TEST="/dev/sda1 on /media/sda1 type vfat (rw,relatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,quiet)" (Result from mount | grep sda1) ...
1
vote
2answers
194 views

How to multiply two columns in awk?

I want to multiply column 1 with column 2 (till end of file) in input file and should output 1 column and multiplied 3 column in separate file. input.txt 1 677679866 2 121867616 3 49413198 4 ...
0
votes
1answer
46 views

How can I print a specific occurence of a range pattern?

I am trying to extract several data fields out of somewhat peculiarly formatted strings. Each string has a format similar to the following (spaces added for emphasis): 1 abcd 2 1 efg 2 hij 3 ...
0
votes
2answers
177 views

Compare first column in different files

I want to compare one file with other files in the same directory. file1.txt contains: move 34#123# get 11#278 put 33#29#567#23 file1.txt should compare with files 1.txt and 2.txt. 1.txt ...
3
votes
2answers
615 views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...
6
votes
3answers
1k views

How to subtract rows (lines) with AWK

I'm trying to figure out how I can use AWK to subtract lines. For example, imagine the input file is: 30 20 The output would be: 10 Now, as a test I am trying to calculate the "Used" memory ...
3
votes
2answers
91 views

Printing in awk up to specific line and 5 more results

I'm trying to print out all the lines up to the appearance of a particular song and 5 more songs with that. The problem i'm new to the shell what I have to fix in the bellow commands? RANK NAME ...
2
votes
1answer
49 views

I'm trying to count and print song file names using awk as following

RANK NAME BAND YEAR GENERE DOMESTIC/INTERNATIONAL 206:Reach Out, I'll Be There:The Four Tops:1978:Pop:3/2 207:Bye Bye Love:The Everly Brothers:1950:Classic:3/2 ...
3
votes
4answers
147 views

Identifying genes from a list of genes

I have a gene list file. Some thing like this SWT21 SSA1 NRP1 EFB1 TFC3 MDM10 I have another file which also contains the names of these genes in my list along with other ...
3
votes
3answers
188 views

Rearrange text file by prepending data rows with header row

I have a file whose content looks like this: 2009 150 0 0 0 0000 75.316 0.0400390625 0.00007 0.00000 0.8980 76.216 0.0400390625 0.00007 1.00000 0.9046 77.217 0.0400390625 ...
0
votes
2answers
76 views

Co Record Verification

I have a task requiring some awk to verify a few records quickly. Let' say: A1,A2 B1,B2 C1,C2 C2,C1 A1,C1 A1,B1 B1,A1 Which would be the best way to check for reciprocity only between the A# the ...
3
votes
3answers
134 views

Append columns in a text file to after the final row

A smaller version of many data files I have looks like this: 0 0 0 0.05 9.6877884e-06 0.0024898597 0.1 4.2838688e-05 0.0049595502 0.15 0.00016929444 0.0074092494 0.2 0.00036426881 0.009839138 0.25 ...
4
votes
3answers
3k views

Filter a .CSV file based on the 5th column values of a file and print those records into a new file

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , ...
1
vote
1answer
132 views

Searching for all the array elements in another array?

This is how I would search for an element in an array. count=${#ARRAY[@]} lsearch() { status=-1 for((i=0;i<count;i++)) do Temp=$1 if [[ $Temp -eq ${ARRAY[i]} ]] then ...
3
votes
2answers
631 views

Using grep/sed/awk to classify log file entries

I need to process a very large log file with many lines in different formats. My goal is to extract unique line entries who have the same starting pattern, e.g. '^2011-02-21.*MyKeyword.*Error', ...
1
vote
3answers
64 views

awk want to total by hour and variable

I am able to run this and it works fine - note: field $1 is a time/date stamped field.. gawk -F ":" "{ print $1 }" /cygdrive/c/counting/ourlog | sort | uniq -c | sort -r 57339 2014-03-21 09 54290 ...