awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

3
votes
2answers
263 views

Bash Script To Find Debian Release Number from /etc/debian_version

The following is what I'm currently using for testing the version of Debian. Instead of doing it this way, can I make it more succinct? #!/bin/bash VERSION=NONE if [ $(cat /etc/debian_version) == ...
2
votes
3answers
105 views

To assign variable from a file separated by delimiter

I have a text file with datas as below separated by delimiter: 1000:Tom:user1 1002:Jerry:user2 1003:Victor:user3 1004:Jason:user4 I need to run a while loop (any loop) to fetch details from first ...
5
votes
3answers
163 views

The ERE regex to split() string between a delimiter and end-of-word

I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem... Sample ASCII input... [[link|label label]] [[link]] ...
-1
votes
1answer
155 views

extracting ip address from a file [closed]

I'm trying to extract all the IP address from a file. FILE *>i85.95.2.160/29 172.16.24.4 0 200 0 64515 4445 15412 i *>i85.112.80.0/28 172.16.24.4 0 200 ...
1
vote
1answer
100 views

Processing FreeBSD's PF logs with tcpdump/sed/awk

I'm running FreeBSD and have PF firewall set up and this is command i use to display daily number of firewall hits (blocked traffic) sudo tcpdump -n -e -tttt -r /var/log/pflog | grep $(date ...
1
vote
1answer
204 views

how to subtract a constant value from a data file?

I want to subtract a constant number from my data to get new data file. For example, I want to subtract 10 from my data. data.txt Id a b c d e 1 1 2 3 4 5 2 2 3 4 5 6 ...
1
vote
3answers
191 views

How to calculate the sum of the data that have the same ID in the first column?

I want to sum the values of the rows of my data that have the same ID in the first column. My data looks like data.txt Id a b c d e 1 1 2 3 4 5 1 2 3 4 5 ...
0
votes
2answers
86 views

Merging two different files with conditions

I have two different files as shown below. Content of a.txt: HDR|1|||||||||| DTL|@||||||||||| TLR||||||||||||| HDR|1|||||||||||| ...
0
votes
2answers
102 views

How to Lookup the values of an index values from a big data file?

I have a data file (data_array.txt in the example) and index file, index.txt, at which I want to extract the data from the data file that have the same ID in the index file and store in to a new file, ...
1
vote
5answers
611 views

Cut and transpose some lines of a file

I have 1344 rows of data column. I want to form 32 row by 42 columns from my input data file. I want to cut the first 42 row and paste to one raw of 42 column data and continue down likewise. My data ...
2
votes
4answers
163 views

sed remove end of line for specific lines

I'm trying to find the sed command to remove the end of the line (meaning, get the next line to the current), but only if line starts with the string NOTOK. example input: NOTOK something ...
-1
votes
1answer
61 views

Using AWK to monitor changing values every 60 seconds [closed]

Does any one have an easy way of doing this using awk, maybe associative arrays? One other thing is there is a delay of 60 seconds between these two input streams and any time a value on the right ...
0
votes
3answers
85 views

Copying specific text from one file into a schema

I have two files. One being a schema, and the other a list of data. The schema looks like: line_name: shot_edit: channel edit: cable_edit and repeats regularly for the number of lines that I ...
0
votes
2answers
656 views

Substring only the first column in awk

I want to take first 500 characters of the first column and leave the rest of the columns as it is, using awk. This is what I am trying to do: awk '{print ${$1:1:500} $2,$NF;}' doc.txt But I get ...
1
vote
2answers
293 views

How can I make awk on Busybox treat the entire input with NULL character in between as a single record?

I wrote a base64 encode script for my Busybox system. #!/bin/sh base64encode() { /usr/bin/awk -- ' function asc(char,l_found) { l_found=0; for (i=0;i<=255;i++){ if ...
0
votes
3answers
61 views

linux + awk + file manipulation

I have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 until the line END_OF_LINE , and from ...
0
votes
3answers
102 views

Print certain fields of each line until a marker is encountered, then print whole lines till the end of file

Hi have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 from each line until the line ...
2
votes
2answers
419 views

Processing two files using awk

I read Comparing two files using Unix and Awk. It is really interesting. I read and tested it, but I can't understand it completely and use it in other cases. I have two files. file1 has one field ...
0
votes
1answer
36 views

rearranging from colum to row in a csv [duplicate]

I have a CSV file that presently has an output like this 1 2 3 4 5 what I need to do is reconfigure the output as follows 1 2 3 4 5 all the entry data on one line, any help would be greatly ...
1
vote
2answers
76 views

awk split parameter by char

file is: BASH.NIR SH.ABI I want the awk will show: User is NIR, SHELL is BASH User is ABI, SHELL is SH I dont know how to split a parameter by char. The Idea is: cat file.txt | awk '{print ...
2
votes
3answers
395 views

awk print ' char

I want to print: cat file.log | awk '{print nir's $1}' output should be: nir's aaa nir's bbb nir's abc nir's dbc The problem is with the ' in nir's. I also tried: cat file.log | awk '{print ...
1
vote
3answers
75 views

How to add zero (0) in blanks in a big file (example file is below)

Here is a example file where I need to replace blanks with zero A 2 2 2 2 2 B 2 2 2 2 2 C 2 2 2 2 2 D A E B F D R
2
votes
1answer
174 views

Extract multiple lines with secondary keyword search

I have a text file with multiple log data, I would like to search for 'Keyword-1' in a line, and when it finds the line, to search within that line for another keyword such as 'engine-1'. Then ...
4
votes
5answers
4k views

extract the 4th column from a csv file using unix command

I have a csv file formatted as below. "col1","col2","col3","col4" "1","text1","<p>big html text</p> ","4th column" "2","text2","<p>big2 html2 text2</p> ...
1
vote
2answers
129 views

What does the following script do? [closed]

This was a question on my study guide, and I believe that the script is pinging the ip addresses, so the 2nd choice. Can somebody confirm this for me? The 5th column of NMAP output below is the IP ...
4
votes
2answers
52 views

Find holes in an enumeration in awk

I have a file that looks like this 1400a_comp 49472 2868636.1026583919660 GOOD 1400a_comp 49473 2868647.1026579919650 GOOD ...
1
vote
1answer
99 views

First line of awk output isn't what I expect

I am trying to figure out why a part of a file name disappears. In my case, I have created three files $ ls some-part-of-file.1 some-part-of-file.2 some-part-of-file.3 and when I use awk with FS ...
4
votes
3answers
186 views

How to only output lines that are younger than 3 months?

cat a.txt 1387111124 ./asfjlasdf.txt 1348681215 ./akdfyxcv.txt So I need this: cat a.txt | SOMEMAGIC 1387111124 ./asfjlasdf.txt The output should only have the lines that are younger than 3 ...
1
vote
2answers
839 views

Type URL and show HTTP status

I want to make a shell script where I type an url and it returns me the status code.So far I've tested wget is there any other cause it has the tendency to save whatever you run. Then I tried to some ...
5
votes
3answers
426 views

Extracting fixed width records with no delimiter from a single line

I need to extract strings of text from a single file containing one very long line of text with no delimiters. Using the sample line below, these are the following known facts: ...
0
votes
2answers
159 views

Format of SMTP headers for scanning inbox email

I am looking for a simple procedure to extract email from my inbox into a file for further processing, using e.g. awk. With "extract" I mean use e.g. regex to get emails with a particular subject. My ...
2
votes
1answer
65 views

awk - streamline this script

I have a text file. On odd number lines, it gives new dimensions of key elements of a building that I'm constructing. On even number lines, the old dimensions are given. My aim is to process this ...
0
votes
3answers
3k views

Shell script read separately columns from file

When we have file which is a kind of list like : Mary 34 George 45 John 56 Josh 29 using the awk command $1 refers to the first column and $2 to the second column.But once we write a shell script ...
-1
votes
2answers
263 views

How to manipulate output from commands with shell scripts [closed]

Simple question, having a ls -l command it makes a list on the command prompt and if we want to manipulate it we pipeline it with awk command, for instance if we want to to show specific files. Is ...
2
votes
1answer
54 views

Matching two different lines that are at least N lines distant from each other

I would like to match files that contain try (in one line) and consecutive catch (in another line) at least N lines later (to find very long try-catches and try to get rid of them). I mean, the first ...
3
votes
3answers
182 views

Print unique of who command

I want to show the online users using the who command but I want to have a unique output and not show any duplicates. I piped the output into awk but I am not very familiar with that, is this the ...
0
votes
2answers
227 views

bash script, printing multiple lines that matching to a specific patern

I have a log file and I'm making a script to do some actions. An action is to print a specific area of the log. Every block at the log starts with a specific time stamp and inside the block may have ...
2
votes
2answers
108 views

Convert numeric string from file to exit code in shell

I'd like to grep /etc/password for a username and return the last two digits of a user-id number as the return code. This is because I have an application whose main way of interacting with other ...
0
votes
0answers
651 views

gsub command in awk [duplicate]

I am trying to replace the field separator from "," to "|" except for the strings which are under double quotes " " . The following command works fine : awk -F\" '{for(i=1;i<=NF;i+=2) ...
2
votes
1answer
116 views

gawk with gawkextlib

In my system (Slackware64-14.1) gawk is installed in version 4.1.0. I'd like to install gawkextlib to have access to some of its features, namely xml. From the README file of gawkextlib-0.4.9: ...
3
votes
1answer
70 views

Automatically load a user function at startup with awk?

I guess this answer from awk separator preferences already gives a hint, but I thought I'd ask explicitly: Is there an equivalent to a .bashrc file for awk, where I could define my own library ...
-2
votes
1answer
47 views

delete first two line in awk script [closed]

How can I delete the first two line in awk script? From blabla blala 1 2 3 4 To 1 2 3 4
4
votes
2answers
63 views

Concatenating fields from lines with different numbers of fields

Given input like this: x y a b c t p q w w t a b c d p q r I'd like to concatenate fields from field 3 up to but not including the last one. If field 3 is the last field, I'd like to insert a ...
1
vote
2answers
175 views

using awk to create a LaTex table from my data in a text file

So I have used awk a few times to add or remove text from each line of a text file. What I would like to learn how to do now is to add some characters to each line in multiple places. What this is ...
5
votes
3answers
262 views

removing redundancy from output columns

What is a *NIX way of removing redundancy in a case where I have pairwise comparisons like these in two columns A B B A A C A D C A D A B C C B A B and B A ...
3
votes
1answer
31 views

How to calculate (weighted) majority over columns?

I have two files: the first one contains lines where the first column is an ID, and the remaining columns are votes, e.g. 3242 -1 1 -1 1 1 1 In a first step I want to calculate the majority for ...
5
votes
5answers
490 views

Pattern matching across multiple lines

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output? I was using something like the following: awk ...
1
vote
2answers
54 views

Replace “-from” with “this” in the begining of line

I want to replace "-from" with "this" in the begining of line. This should happen when that line has 'R' at the end and line above it has "D" at the end. For example for the block shown below: ...
5
votes
2answers
97 views

Adding corresponding Numbers in a Range

I have a dataset with 30000 lines. I want to add all the corresponding values (i.e. Column 2) between the range 0-50 and then 50.1-100 and then 100.1-150 etc. (presented in column 1), and this ...
4
votes
3answers
97 views

awk separator preferences

I write ad hoc awk commands in cygwin to process tab delimited files. Pretty much every command starts with: BEGIN {FS="\t";OFS="\t";ORS="\r\n";} How can I make these separators the default to ...