awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

-1
votes
3answers
45 views

Insert array of years into a list of URLS

I have a list of URLS like the example below. /test/test I also have an array of years [1999, 2000, 2001]. I am trying to find a way to automate the task of printing out a list of URLS with ...
0
votes
1answer
49 views

How to Tail -f output with colors using only Awk and show the rest of the output?

I was able to do a tail -f output with colors using the Awk command. But I'm unsure how to output the rest of the log as it seems to be filtering only those 2 lines. I also want to see the other lines ...
2
votes
4answers
393 views

How can I split a line into two lines if the length is greater than 7 using awk? [closed]

For example I just wanted to print something like this in the command line. Let say I have a file called file.txt. What is life? how are you? hi whatup this is more than And I want to print it ...
0
votes
3answers
89 views

insert variable text after another variable text on a specific line

I am trying to insert variable text after another variable text on a specific line. Below is my attempt so far, as far as I can tell, if I can escape the quotations in awk so that the variable can be ...
7
votes
6answers
7k views

How to remove all comments from a file?

I have a file with a few lines commented out: foo bar stuff #Do not show this... morestuff evenmorestuff#Or this I just want to print all the uncommented code: foo bar stuff morestuff ...
1
vote
2answers
48 views

sed + add last word on each string only if word not appears

I have the following file: linuxA.mtt.corp linux3V linux4B linux2A linux5v.mtt.corp how to add the .mtt.corp on each string that not have this word?
6
votes
5answers
6k views

Print line after nth occurrence of a match

I am looking to display the line 4598 in the following file. Effectively I want to display the line AFTER the nth occurrence of a match. In this case, the line after the 3rd occurrence of <Car>. ...
2
votes
2answers
61 views

find duplicate in first, second columns and sum numbers in third column

I have this input: Fred apples 20 Susy oranges 5 Mark watermellons 12 Robert pears 4 Robert oranges 17 Terry oranges 9 Lisa peaches 7 Susy oranges 12 Mark grapes 39 Anne mangoes 7 Greg pineapples 3 ...
1
vote
2answers
44 views

awk + how to append list in file

I have the following list in file more file.ex efifc1a hygg4a hdy5d nhdw4s wesf3a fjfhyr jfhg gsfar how to append any second list after the first list and any third list after the second list ...
1
vote
1answer
45 views

extract time range from log file using AWK

I am trying to extract time range from a logfile by passing shell variables in awk. This is how my log file is. 2014-11-24 12:58:59.290 1.1.1.1 etc.. 2014-11-24 13:58:59.290 2.2.2.2 etc.. 2014-11-...
4
votes
4answers
496 views

Print only the Nth line before each line that matches a pattern

I'm trying to print only the <N>th line before a search pattern. grep -B<N> prints all the <N> lines before the search pattern. I saw the awk code here that can print only the <N&...
6
votes
2answers
11k views

Remove lines based on duplicates within one column without sort

I have large 3-column files (~10,000 lines) and I would like to remove lines when the contents of the third column of that line appear in the third column of another line. The files' sizes make sort a ...
4
votes
4answers
145 views

Replace strings in a file based on a list of strings and a list of corresponding replacements

I am trying to replace strings in a file A: Hello Peter, how is your dad? where is mom? where the strings to be replaced are in file B: Peter dad mom and their corresponding replacements are in ...
0
votes
1answer
39 views

subtract previous line based on another column

I'm trying to figure out how to subtract previous line based on another column using AWK. Here is the input: site year count 040130013 1997 34 040130013 1998 55 040130019 2006 79 040130019 2007 135 ...
1
vote
4answers
84 views

Impose character between repeated characters

Given a string of characters ,,a,,,b,,,, I want to impose the character - at the beginning, if no character is preceded comma, and between every comma that has no character. Meaning the result is -,-,...
1
vote
2answers
60 views

Process several files and print filename without extension and the value of a specific key

I have ASCII file and I want to output the file name and a specific field from this file beside each other. For example: The contents of the ASCII file "data1.txt" is as follow: Name: John Age: 90 ...
-3
votes
2answers
51 views

linux + convert file format columns to one column

I have the following file (Linux.txt) linux-test01 linuxER LinuxCV LinuxWE LinuxPL LinuxDF LinuxXC LinuxSD LinuxAQ how to print using awk or perl one-liner to produce the following ...
0
votes
1answer
56 views

Compare strings with specialized charcters

If I have a lookup table cat tmp1 [//rtwttwtr*fgg]\\\erw``~ 1 ^774574574565665f[[[//]\] 2 ()42543^[[D^[[D^[[D^[[D^[[D353535345****@3242- 3 and a number of strings to be looked up from the first ...
21
votes
9answers
28k views

How to print certain columns by name?

I have the following file: id name age 1 ed 50 2 joe 70 I want to print just the id and age columns. Right now I just use awk: cat file.tsv | awk '{ print $1, $3 }' However, this ...
3
votes
1answer
36 views

Awk Search - Combined Character Classes in Same String

Good Evening, I am attempting to utilize multiple character classes at the same time without success. Given a column that contains the URI of HTTP traffic, I want the REGEX to identify lines that ...
0
votes
2answers
60 views

How to determine what IP addresses are available in a hosts table?

I'm working on a script that will read from the /etc/hosts file and determine what IPs are available in the hosts file. The way the hosts file will look is as followed: 172.17.164.2 System1 172.17....
3
votes
1answer
43 views

Splitting a large file by column with values in the header as file names

I have a large tab-delimited text file which contains a header and data. I want to split the file into smaller files by column, with the header excluded, and the files named as the column name. pos ...
0
votes
2answers
50 views

Awk: Print last N columns, where N is passed through a variable

My input variable $dirPath contains directory path. The action is to be able to retrieve the last N values in the directory path, where N is passed through a variable $depth. For a fixed N value (say ...
0
votes
1answer
71 views

Redirect output of: yum repolist all

If you run the command: yum repolist all you will see there are basically four columns of data. repo id, repo name, status, install count. If however your run something like: yum repolist all > ...
3
votes
2answers
55 views

Simple awk wrapper function or alias

I had seen someone use a bash alias or function based on awk, which I liked for its simplicity, but can't remember how to implement it (in bash). Lets say you have a large formatted file, from which ...
0
votes
3answers
79 views

Right syntax for awk usage in combination with other command inside xargs sh –c

How to make this command work: ls * | xargs -I {} sh -c 'echo {}; awk '{print $1}' {} | uniq' It should do the simple thing: print for each file in the folder its name and uniq values in the first ...
1
vote
1answer
52 views

Filter out duplicate lines of tail -f [duplicate]

I often need to tail -f apache access logs for websites to troubleshoot issues- one thing that makes it annoying is that anyone loading a page once may cause 12+ lines to get written to the log, and ...
2
votes
2answers
79 views

Justifying text using awk

I am trying to format my output file so that text in lines is spreaded evenly, to align it both left and right. To do left and right justifying is trivial, but how can one do both left and right with ...
0
votes
1answer
18 views

Log from a specific line to another specific line [duplicate]

I am doing a data monitoring and I am doing a cron job to get the data everyday. But I am receiving many junk data with my actual data that makes the file huge. For example my data is, eg.hh.jk.ll.mm ...
0
votes
0answers
47 views

find awk log print error

I'm trying to find one year old directories named in yyyymmdd format and print it in the log file placed at different location. find ${LOG} -maxdepth 1 -type d -mtime +${DAY} -name '20[0-9][0-9][0-9][...
1
vote
0answers
30 views

Paging with follow less +F and colorize with awk

I would like to paginate a file with follow, i.e. $less +F and also colorize the output on the screen. I want to use less because I can leave the follow mode, browse the file and then re-enter the ...
280
votes
6answers
544k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
0
votes
1answer
27 views

sed - match on IP octet and remove line

I am in need of a solution, preferably sed, that can match on an IP octet that contains any number combination and remove the line. For example: My file: 192.168.100.12 192.168.200.12 10.10.20....
0
votes
1answer
26 views

Linux red hat + replace PATH or word on scripts

Under folder: /var/userTN/applo folder I have around ~100 shell scripts that include the following PATH: GLOBAL_PATH=/export/pop/PLW5 I want to replace the PATH on all the scripts under folder: ...
2
votes
2answers
145 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
0
votes
1answer
22 views

Using awk to change an entire column of both positive and negative values, to just negative? [duplicate]

I use awk alot to manipulate xyz text files (lat,long,elevation). I have a specific file that has both positive and negative values in the z column, I would like all of these to be negative. So {$3*-1}...
2
votes
4answers
6k views

Split a file into two

I have a big file and need to split into two files. Suppose in the first file the 1000 lines should be selected and put into another file and delete those lines in the first file. I tried using split ...
1
vote
4answers
137 views

How to add a new text line at the first line of a file? [duplicate]

File: TABLE1 1234 9555 87676 2344 Expected output: Description of the following table: TABLE1 1234 9555 87676 2344
6
votes
8answers
7k views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
-1
votes
2answers
35 views

errors using alias for an awk command

I'm attempting to add this alias to my .zshrc but run into errors. I've tried escaping the ==, ", and && values and still no luck. What am I missing? alias broke="ssh -t server@5.555.555.555 ...
2
votes
1answer
68 views

Can I determine type of an awk variable?

I have the gawk version of awk. In this part of gawk manual, it is stated that awk variables have "attributes", which are used to determine how to treat them in various operations. For example, a ...
0
votes
0answers
38 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...
2
votes
5answers
475 views

How to find what device a file is on (and use that in a script)?

I want to find out what device my file is on so that I can use it in a script. I can get this far: $ df . Filesystem 512-blocks Used Available Capacity Mounted on /dev/disk0s2 498438976 ...
-4
votes
2answers
37 views

How to process only certain columns and ignore header and footer in delimited text file using awk?

I have an input text file called text.txt: Header Column1|Column2|Column3 C11|C12|C13 C21|C22|C23 C31|C32|C33 Footer My output should be: Column1|Column3 C11|C13 C21|C23 C31|C33 My command that ...
-1
votes
1answer
38 views

Unix/Linux - File Formating

We have a source file which needs to be formatted into a fixed width file. Source File Format: Hdr1 Dtl1.1 Dtl1.2 Dtl1.3 Dtl1.4 Trailer0004 Hdr2 Dtl2.1 Dtl2.2 Dtl2.3 Dtl2.4 Dtl2.5 Trailer0005 . . . ...
9
votes
7answers
18k views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
3
votes
1answer
36 views

Replace a word after matched word with incrementing values

I have a file like this - ! interface csa2549abA xxxx yyyy zzzz ! interface aso23apwoeAe aaa bbb ccc ! interface FastEthernet0/1 ppp qqq rrr ! and so ...
1
vote
1answer
17 views

How do I interpolate a text file of 3D coordinates (X,Y,Z) onto a regular grid?

I have a 3D surface in a text file which I need to plot on a regular X/Y grid. However, the values for X and Y are not regularly spaced and are not necessarily in ascending order. I need to regularly ...
2
votes
3answers
51 views

linux + ssh + perform two tasks in one ssh and print to CSV

the following command print the two values in one ssh shut ssh test01 "hostname && cat /etc/sysconfig/rhn/rhnsd " test01 INTERVAL=240 but how to change the syntax in order to print the ...
0
votes
2answers
45 views

return first instance of characters from a list

Lets say I have the output of this command saved to a file. cat /dev/urandom | tr -dc '[:graph:]' | fold -w 1000 | perl -pe 's/(.)(?=.*?\1)//g' | head -n 50 I would like to compare only the first n ...