awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
33 views

Wrong output in comparing floats

I'm trying to compare two floats in bash and something is going wrong. Here is the code sample based on solution here num1=0.502E-01 num2=0.01 echo $num1'>'$num2 | bc -l echo $num2'>'$num1 | bc ...
1
vote
4answers
310 views

How to split text file into records by first non-blank character using awk and print only records containing string

I have a text file with information separated in blocks. I would like to separate these blocks into records so that I can handle them separately. This is my text file (file.txt): Alarm Stats: com....
0
votes
1answer
36 views

have a log file that I would like to break up into several files based on content

I have a log file that I would like to break up into several files Table of Contents *** Configuration Database *** Logs *** Configuration Logs *** Query Logs . . and so on The issue ...
3
votes
1answer
45 views

Modify values in a column based on number of colons it contains

I would manipulate the contents of the second column based on the number of colons it contains. If the second field contains more than one colon then I would require the content before first colon ...
1
vote
2answers
87 views

Print text between multi-line delimiters

I am trying to extract data between 2 matching patterns, only if the content is not empty and I have issues doing it: here is an example: ============================== Queue Manager is : ...
4
votes
7answers
734 views

How to split a filename list in 5GB sets?

I have a pretty big list of files which I like to group in smaller list of 5GB each for further processing. The file list is in the following form: "size" (from du) "tab" "absolute path", like this (...
1
vote
1answer
64 views

AWK, Sum of category

I have tons of CSV files with similar content. The values are usually comma separated and they look like this. product_a, domestic, 500 product_a, abroad, 15 product_b, domestic, 313 ...
1
vote
2answers
73 views

How can I use Awk to convert a unix timestamp to human readable when doing a Tail -f logfile?

I'm currently doing a tail -f mysql.log and also have it color coded. But how would I use awk to convert the unix timestamps to human readable format? tail -f /var/lib/mysql/mysql-slow.log | awk '/...
-1
votes
3answers
45 views

Insert array of years into a list of URLS

I have a list of URLS like the example below. /test/test I also have an array of years [1999, 2000, 2001]. I am trying to find a way to automate the task of printing out a list of URLS with ...
0
votes
1answer
51 views

How to Tail -f output with colors using only Awk and show the rest of the output?

I was able to do a tail -f output with colors using the Awk command. But I'm unsure how to output the rest of the log as it seems to be filtering only those 2 lines. I also want to see the other lines ...
2
votes
4answers
401 views

How can I split a line into two lines if the length is greater than 7 using awk? [closed]

For example I just wanted to print something like this in the command line. Let say I have a file called file.txt. What is life? how are you? hi whatup this is more than And I want to print it ...
0
votes
3answers
97 views

insert variable text after another variable text on a specific line

I am trying to insert variable text after another variable text on a specific line. Below is my attempt so far, as far as I can tell, if I can escape the quotations in awk so that the variable can be ...
7
votes
6answers
8k views

How to remove all comments from a file?

I have a file with a few lines commented out: foo bar stuff #Do not show this... morestuff evenmorestuff#Or this I just want to print all the uncommented code: foo bar stuff morestuff ...
1
vote
2answers
49 views

sed + add last word on each string only if word not appears

I have the following file: linuxA.mtt.corp linux3V linux4B linux2A linux5v.mtt.corp how to add the .mtt.corp on each string that not have this word?
6
votes
5answers
6k views

Print line after nth occurrence of a match

I am looking to display the line 4598 in the following file. Effectively I want to display the line AFTER the nth occurrence of a match. In this case, the line after the 3rd occurrence of <Car>. ...
2
votes
2answers
64 views

find duplicate in first, second columns and sum numbers in third column

I have this input: Fred apples 20 Susy oranges 5 Mark watermellons 12 Robert pears 4 Robert oranges 17 Terry oranges 9 Lisa peaches 7 Susy oranges 12 Mark grapes 39 Anne mangoes 7 Greg pineapples 3 ...
1
vote
2answers
45 views

awk + how to append list in file

I have the following list in file more file.ex efifc1a hygg4a hdy5d nhdw4s wesf3a fjfhyr jfhg gsfar how to append any second list after the first list and any third list after the second list ...
1
vote
1answer
49 views

extract time range from log file using AWK

I am trying to extract time range from a logfile by passing shell variables in awk. This is how my log file is. 2014-11-24 12:58:59.290 1.1.1.1 etc.. 2014-11-24 13:58:59.290 2.2.2.2 etc.. 2014-11-...
4
votes
4answers
516 views

Print only the Nth line before each line that matches a pattern

I'm trying to print only the <N>th line before a search pattern. grep -B<N> prints all the <N> lines before the search pattern. I saw the awk code here that can print only the <N&...
6
votes
2answers
12k views

Remove lines based on duplicates within one column without sort

I have large 3-column files (~10,000 lines) and I would like to remove lines when the contents of the third column of that line appear in the third column of another line. The files' sizes make sort a ...
4
votes
4answers
149 views

Replace strings in a file based on a list of strings and a list of corresponding replacements

I am trying to replace strings in a file A: Hello Peter, how is your dad? where is mom? where the strings to be replaced are in file B: Peter dad mom and their corresponding replacements are in ...
0
votes
1answer
39 views

subtract previous line based on another column

I'm trying to figure out how to subtract previous line based on another column using AWK. Here is the input: site year count 040130013 1997 34 040130013 1998 55 040130019 2006 79 040130019 2007 135 ...
1
vote
4answers
84 views

Impose character between repeated characters

Given a string of characters ,,a,,,b,,,, I want to impose the character - at the beginning, if no character is preceded comma, and between every comma that has no character. Meaning the result is -,-,...
1
vote
2answers
61 views

Process several files and print filename without extension and the value of a specific key

I have ASCII file and I want to output the file name and a specific field from this file beside each other. For example: The contents of the ASCII file "data1.txt" is as follow: Name: John Age: 90 ...
-3
votes
2answers
51 views

linux + convert file format columns to one column

I have the following file (Linux.txt) linux-test01 linuxER LinuxCV LinuxWE LinuxPL LinuxDF LinuxXC LinuxSD LinuxAQ how to print using awk or perl one-liner to produce the following ...
0
votes
1answer
56 views

Compare strings with specialized charcters

If I have a lookup table cat tmp1 [//rtwttwtr*fgg]\\\erw``~ 1 ^774574574565665f[[[//]\] 2 ()42543^[[D^[[D^[[D^[[D^[[D353535345****@3242- 3 and a number of strings to be looked up from the first ...
21
votes
9answers
28k views

How to print certain columns by name?

I have the following file: id name age 1 ed 50 2 joe 70 I want to print just the id and age columns. Right now I just use awk: cat file.tsv | awk '{ print $1, $3 }' However, this ...
3
votes
1answer
36 views

Awk Search - Combined Character Classes in Same String

Good Evening, I am attempting to utilize multiple character classes at the same time without success. Given a column that contains the URI of HTTP traffic, I want the REGEX to identify lines that ...
0
votes
2answers
60 views

How to determine what IP addresses are available in a hosts table?

I'm working on a script that will read from the /etc/hosts file and determine what IPs are available in the hosts file. The way the hosts file will look is as followed: 172.17.164.2 System1 172.17....
3
votes
1answer
45 views

Splitting a large file by column with values in the header as file names

I have a large tab-delimited text file which contains a header and data. I want to split the file into smaller files by column, with the header excluded, and the files named as the column name. pos ...
0
votes
2answers
52 views

Awk: Print last N columns, where N is passed through a variable

My input variable $dirPath contains directory path. The action is to be able to retrieve the last N values in the directory path, where N is passed through a variable $depth. For a fixed N value (say ...
0
votes
1answer
73 views

Redirect output of: yum repolist all

If you run the command: yum repolist all you will see there are basically four columns of data. repo id, repo name, status, install count. If however your run something like: yum repolist all > ...
3
votes
2answers
56 views

Simple awk wrapper function or alias

I had seen someone use a bash alias or function based on awk, which I liked for its simplicity, but can't remember how to implement it (in bash). Lets say you have a large formatted file, from which ...
0
votes
3answers
81 views

Right syntax for awk usage in combination with other command inside xargs sh –c

How to make this command work: ls * | xargs -I {} sh -c 'echo {}; awk '{print $1}' {} | uniq' It should do the simple thing: print for each file in the folder its name and uniq values in the first ...
1
vote
1answer
55 views

Filter out duplicate lines of tail -f [duplicate]

I often need to tail -f apache access logs for websites to troubleshoot issues- one thing that makes it annoying is that anyone loading a page once may cause 12+ lines to get written to the log, and ...
2
votes
2answers
80 views

Justifying text using awk

I am trying to format my output file so that text in lines is spreaded evenly, to align it both left and right. To do left and right justifying is trivial, but how can one do both left and right with ...
0
votes
1answer
18 views

Log from a specific line to another specific line [duplicate]

I am doing a data monitoring and I am doing a cron job to get the data everyday. But I am receiving many junk data with my actual data that makes the file huge. For example my data is, eg.hh.jk.ll.mm ...
0
votes
0answers
47 views

find awk log print error

I'm trying to find one year old directories named in yyyymmdd format and print it in the log file placed at different location. find ${LOG} -maxdepth 1 -type d -mtime +${DAY} -name '20[0-9][0-9][0-9][...
1
vote
0answers
30 views

Paging with follow less +F and colorize with awk

I would like to paginate a file with follow, i.e. $less +F and also colorize the output on the screen. I want to use less because I can leave the follow mode, browse the file and then re-enter the ...
281
votes
6answers
552k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
0
votes
1answer
32 views

sed - match on IP octet and remove line

I am in need of a solution, preferably sed, that can match on an IP octet that contains any number combination and remove the line. For example: My file: 192.168.100.12 192.168.200.12 10.10.20....
0
votes
1answer
26 views

Linux red hat + replace PATH or word on scripts

Under folder: /var/userTN/applo folder I have around ~100 shell scripts that include the following PATH: GLOBAL_PATH=/export/pop/PLW5 I want to replace the PATH on all the scripts under folder: ...
2
votes
2answers
145 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
0
votes
1answer
23 views

Using awk to change an entire column of both positive and negative values, to just negative? [duplicate]

I use awk alot to manipulate xyz text files (lat,long,elevation). I have a specific file that has both positive and negative values in the z column, I would like all of these to be negative. So {$3*-1}...
2
votes
4answers
6k views

Split a file into two

I have a big file and need to split into two files. Suppose in the first file the 1000 lines should be selected and put into another file and delete those lines in the first file. I tried using split ...
1
vote
4answers
141 views

How to add a new text line at the first line of a file? [duplicate]

File: TABLE1 1234 9555 87676 2344 Expected output: Description of the following table: TABLE1 1234 9555 87676 2344
6
votes
8answers
7k views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
-1
votes
2answers
35 views

errors using alias for an awk command

I'm attempting to add this alias to my .zshrc but run into errors. I've tried escaping the ==, ", and && values and still no luck. What am I missing? alias broke="ssh -t server@5.555.555.555 ...
2
votes
1answer
71 views

Can I determine type of an awk variable?

I have the gawk version of awk. In this part of gawk manual, it is stated that awk variables have "attributes", which are used to determine how to treat them in various operations. For example, a ...
0
votes
0answers
39 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...