awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
53 views

Compare strings with specialized charcters

If I have a lookup table cat tmp1 [//rtwttwtr*fgg]\\\erw``~ 1 ^774574574565665f[[[//]\] 2 ()42543^[[D^[[D^[[D^[[D^[[D353535345****@3242- 3 and a number of strings to be looked up from the first ...
20
votes
9answers
26k views

How to print certain columns by name?

I have the following file: id name age 1 ed 50 2 joe 70 I want to print just the id and age columns. Right now I just use awk: cat file.tsv | awk '{ print $1, $3 }' However, this ...
3
votes
1answer
30 views

Awk Search - Combined Character Classes in Same String

Good Evening, I am attempting to utilize multiple character classes at the same time without success. Given a column that contains the URI of HTTP traffic, I want the REGEX to identify lines that ...
0
votes
2answers
56 views

How to determine what IP addresses are available in a hosts table?

I'm working on a script that will read from the /etc/hosts file and determine what IPs are available in the hosts file. The way the hosts file will look is as followed: 172.17.164.2 System1 ...
-3
votes
0answers
34 views

Awk command I want to compare two rows in two files and update the second file if the row matches [duplicate]

File1.txt Zuma|ZUMA File2.txt Series|2|Zuma|2| Series|2|Zuma|2| If row 1 in File1 exist in File2.txt then replace it with row 2 of file 1.txt Expected output in File2.txt should be ...
0
votes
2answers
916 views

Bash to join column from multiple files

I have around 20 files in different folders for which I have created a separate file containing path information called pathtofiles.in; /home/users/gray_wolf/unix/File_1.tsv ...
3
votes
1answer
37 views

Splitting a large file by column with values in the header as file names

I have a large tab-delimited text file which contains a header and data. I want to split the file into smaller files by column, with the header excluded, and the files named as the column name. pos ...
0
votes
2answers
37 views

Awk: Print last N columns, where N is passed through a variable

My input variable $dirPath contains directory path. The action is to be able to retrieve the last N values in the directory path, where N is passed through a variable $depth. For a fixed N value (say ...
0
votes
1answer
32 views

Redirect output of: yum repolist all

If you run the command: yum repolist all you will see there are basically four columns of data. repo id, repo name, status, install count. If however your run something like: yum repolist all > ...
3
votes
2answers
48 views

Simple awk wrapper function or alias

I had seen someone use a bash alias or function based on awk, which I liked for its simplicity, but can't remember how to implement it (in bash). Lets say you have a large formatted file, from which ...
0
votes
3answers
64 views

Right syntax for awk usage in combination with other command inside xargs sh –c

How to make this command work: ls * | xargs -I {} sh -c 'echo {}; awk '{print $1}' {} | uniq' It should do the simple thing: print for each file in the folder its name and uniq values in the first ...
0
votes
2answers
2k views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
1
vote
1answer
43 views

Filter out duplicate lines of tail -f [duplicate]

I often need to tail -f apache access logs for websites to troubleshoot issues- one thing that makes it annoying is that anyone loading a page once may cause 12+ lines to get written to the log, and ...
2
votes
2answers
75 views

Justifying text using awk

I am trying to format my output file so that text in lines is spreaded evenly, to align it both left and right. To do left and right justifying is trivial, but how can one do both left and right with ...
0
votes
1answer
18 views

Log from a specific line to another specific line [duplicate]

I am doing a data monitoring and I am doing a cron job to get the data everyday. But I am receiving many junk data with my actual data that makes the file huge. For example my data is, eg.hh.jk.ll.mm ...
0
votes
0answers
35 views

find awk log print error

I'm trying to find one year old directories named in yyyymmdd format and print it in the log file placed at different location. find ${LOG} -maxdepth 1 -type d -mtime +${DAY} -name ...
1
vote
0answers
27 views

Paging with follow less +F and colorize with awk

I would like to paginate a file with follow, i.e. $less +F and also colorize the output on the screen. I want to use less because I can leave the follow mode, browse the file and then re-enter the ...
253
votes
6answers
486k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
-4
votes
0answers
41 views

Join column with columns of next two line in shell script

I want to join value of the first line with the value of next two line. If some value includes number (G, G1, G2) in input data, It should be considered for all combinations. For example, input ...
0
votes
1answer
20 views

sed - match on IP octet and remove line

I am in need of a solution, preferably sed, that can match on an IP octet that contains any number combination and remove the line. For example: My file: 192.168.100.12 192.168.200.12 ...
0
votes
1answer
22 views

Linux red hat + replace PATH or word on scripts

Under folder: /var/userTN/applo folder I have around ~100 shell scripts that include the following PATH: GLOBAL_PATH=/export/pop/PLW5 I want to replace the PATH on all the scripts under folder: ...
1
vote
4answers
794 views

How to extract column name (header) from a CSV file which contains the max value in a row?

I am trying to extract the column name with the maximum value in each row using bash script i.e., the column header value or the value from the same column in the first row. I am using the following ...
2
votes
2answers
137 views

How do I double each line of piped output? [duplicate]

How do I double each line of input piped in? Example: echo "foobar" | MYSTERY_COMMAND foobar foobar
0
votes
1answer
18 views

Using awk to change an entire column of both positive and negative values, to just negative? [duplicate]

I use awk alot to manipulate xyz text files (lat,long,elevation). I have a specific file that has both positive and negative values in the z column, I would like all of these to be negative. So ...
1
vote
4answers
5k views

Split a file into two

I have a big file and need to split into two files. Suppose in the first file the 1000 lines should be selected and put into another file and delete those lines in the first file. I tried using split ...
1
vote
4answers
111 views

How to add a new text line at the first line of a file? [duplicate]

File: TABLE1 1234 9555 87676 2344 Expected output: Description of the following table: TABLE1 1234 9555 87676 2344
6
votes
8answers
6k views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
-1
votes
2answers
32 views

errors using alias for an awk command

I'm attempting to add this alias to my .zshrc but run into errors. I've tried escaping the ==, ", and && values and still no luck. What am I missing? alias broke="ssh -t server@5.555.555.555 ...
2
votes
1answer
51 views

Can I determine type of an awk variable?

I have the gawk version of awk. In this part of gawk manual, it is stated that awk variables have "attributes", which are used to determine how to treat them in various operations. For example, a ...
0
votes
0answers
33 views

awk + how to manipulate awk output

The following awk command prints the results from services file # awk '{print $2}' /etc/services 48002/tcp 48002/udp 48003/tcp How can I manipulate the output so it prints the "service file" ...
2
votes
5answers
399 views

How to find what device a file is on (and use that in a script)?

I want to find out what device my file is on so that I can use it in a script. I can get this far: $ df . Filesystem 512-blocks Used Available Capacity Mounted on /dev/disk0s2 498438976 ...
-4
votes
2answers
28 views

How to process only certain columns and ignore header and footer in delimited text file using awk?

I have an input text file called text.txt: Header Column1|Column2|Column3 C11|C12|C13 C21|C22|C23 C31|C32|C33 Footer My output should be: Column1|Column3 C11|C13 C21|C23 C31|C33 My command that ...
-1
votes
1answer
36 views

Unix/Linux - File Formating

We have a source file which needs to be formatted into a fixed width file. Source File Format: Hdr1 Dtl1.1 Dtl1.2 Dtl1.3 Dtl1.4 Trailer0004 Hdr2 Dtl2.1 Dtl2.2 Dtl2.3 Dtl2.4 Dtl2.5 Trailer0005 . . . ...
9
votes
7answers
15k views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
3
votes
1answer
32 views

Replace a word after matched word with incrementing values

I have a file like this - ! interface csa2549abA xxxx yyyy zzzz ! interface aso23apwoeAe aaa bbb ccc ! interface FastEthernet0/1 ppp qqq rrr ! and so ...
1
vote
1answer
17 views

How do I interpolate a text file of 3D coordinates (X,Y,Z) onto a regular grid?

I have a 3D surface in a text file which I need to plot on a regular X/Y grid. However, the values for X and Y are not regularly spaced and are not necessarily in ascending order. I need to regularly ...
2
votes
3answers
43 views

linux + ssh + perform two tasks in one ssh and print to CSV

the following command print the two values in one ssh shut ssh test01 "hostname && cat /etc/sysconfig/rhn/rhnsd " test01 INTERVAL=240 but how to change the syntax in order to print the ...
0
votes
2answers
44 views

return first instance of characters from a list

Lets say I have the output of this command saved to a file. cat /dev/urandom | tr -dc '[:graph:]' | fold -w 1000 | perl -pe 's/(.)(?=.*?\1)//g' | head -n 50 I would like to compare only the first n ...
4
votes
1answer
381 views

How do I sort words by syllable count?

How do I perform an ascending sort of a word list, based upon how many syllables each word contains? Example Input: something important dog cat bookshelf Example Output: dog cat something ...
4
votes
1answer
77 views

How Do I Remove Duplicate Words With Suffixes?

How do I compare a list of unsorted words of variable length and remove duplicate words that have a suffix? Example word list: iron curl curled railroad curler ...
1
vote
3answers
63 views

Delete the nth line from a matched string (which only occurs once in the file)

I have a question related to deletion of strings that appear at the n+2 position after the matched string which is at position n using awk for multiple files. I am able to print it using the command: ...
8
votes
6answers
12k views

How to grep for text in a file and display the paragraph that has the text?

Below is the text in the file: Pseudo name=Apple Code=42B state=fault Pseudo name=Prance Code=43B state=good I need to grep for "42B" and get the output from the above text like: Pseudo ...
2
votes
3answers
84 views

Replace values in file

I have a files: $cat file1.txt 1234|W 1345|S 8427|D 2132|C 3243|V and my sql file is : $cat select.sql SELECT * FROM CUSTOMERS WHERE ID IN (FLAG); and I have a shell with the next ...
0
votes
3answers
31 views

split lines and print it to new line for certain character

I need to split lines and print it to next line when pattern match. Like I have: ABC123xxx:: 2345 ABC345yyy:: 5678 ABC986zzz:: 7955 And I want it to print to new line when ABC pattern will come: ...
-1
votes
2answers
31 views

How to convert rows to columns based on cloumn value in awk in unix

I have data like below dn: uid=032096,OU=Customers,OU=People,O=B2C objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user objectClass: inetOrgPerson objectClass: ...
2
votes
3answers
388 views

How to count the number of words and print the lines that match exactly a given pattern?

I want to count the number of words and print the matched pattern lines which matches exactly with following pattern: abc-ERROR: The input File contains: # abc-ERROR: xyxxkkfgfr # def-Error: ...
-4
votes
3answers
52 views

I need a command to filter the statements from a .txt file

I have a file with the following contents: select * from emp; select * from dept; selection end; I want to obtain this output: select * from emp; select * from dept; I tried by ...
25
votes
13answers
10k views

Remove duplicate $PATH entries with awk command

I am trying to write a bash shell function that will allow me to remove duplicate copies of directories from my path environment. I was told that it is possible to achieve this with a one line ...
3
votes
7answers
314 views

grep with continuation lines

How can I grep/awk/sed a file looking for some pattern, and print the entire line (including continuation lines if the matched line ends with \? File foo.txt contains: something whatever thisXXX ...
1
vote
4answers
65 views

Replace the specified string to its appearance time using sed or awk

I want to replace oooo to 1, 2, 3, ..., like a example below: From > cat demo.snippet snippet anova.cca "anova.cca fun" anova.cca( ,${oooo:object} ${oooo:,...} ,permutations = ...