awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
1answer
44 views

Why does AWK not like dots in one of two substitutions?

I'm trying to use AWK in a bash script to look for a placeholder string in a template file and replace it with the contents of a variable that (can) contain various special characters. Example: ...
2
votes
2answers
388 views

Use awk to split line into array and use that array's values in calling shell

First of all, my apologies for the somewhat clumsily worded title. If you can think of a better one, please feel free to edit. [EDIT: done] What I am trying to do is use awk inside a bash script and ...
3
votes
1answer
45 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
-2
votes
3answers
71 views

Deleting Lines, which match a particular Identifier from another file

I have 2 files. File 1, has an identifier (eg. D7MHBF:11:1449:1988) and every new entry starts with @. It has few more fields, which are not important in our analysis. File 2 consists of a column of ...
4
votes
2answers
51 views

awk pairwise differences

I run into this problem a lot with data, if I have some integers, I frequently want to look at differences between adjacent integers, I usually solve this in Ruby or Python, but I think such a thing ...
6
votes
1answer
166 views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
0
votes
3answers
146 views

Awk/grep/sed get comma separated list of numbers from lines of text

I have a log file that has lines that look like this: blah blah blah Photo for (702049679 - blah blah blah Now I know I could get all the lines like that from the file by doing: grep "Photo for" ...
1
vote
2answers
101 views

what is awk '{print $1+0.45 “ ” $2 “ ” $3 }' file1> file2?

What does awk '{print $1+0.45 " " $2 " " $3 }' positionXYZ > positionX0.45YZ mean? Does it mean changing a row in the first file and put the new data into the second one?
0
votes
2answers
66 views

Using AWK to delete line and output result

I have a text file and manage to find the line through output comparison. With the book title and author name. awk -F ':' '/KungFu Feet|Chuck Norris/' test.txt Kungfu Feet:Chuck Norris:12:1:1 Is ...
1
vote
7answers
239 views

Sed / Awk - Find string exactly n characters long

Looking through the man pages for sed, awk and grep, I'm unable to find a way to search for a string that is exactly n characters long. Given the following text file, I want to extract just ...
5
votes
2answers
105 views

how to add command line calculator to bash that uses comma as decimal mark?

How can I add a command line calculator to my bash? I have found some, but all of them use the full stop as decimal mark, but I want to have it to use the comma as decimal mark as most of the world ...
0
votes
1answer
59 views

AWK command failing for large file

I have a .csv file of 6.5 GB. I need to sort the files with 1st, 2nd, 3rd, 4th, 5th and 13th column and need to get the unique records. Also I need to get the duplicate records and redirect it to some ...
1
vote
5answers
166 views

Use awk to find all Ports for each IP that have https open

I'd like to print all ips (IP space Port) which have open https ports given a gnmap file. An example output for a line that has only https running on port 443: 123.123.123.123 443 A more ...
3
votes
2answers
128 views

Bash Script To Find Debian Release Number from /etc/debian_version

The following is what I'm currently using for testing the version of Debian. Instead of doing it this way, can I make it more succinct? #!/bin/bash VERSION=NONE if [ $(cat /etc/debian_version) == ...
2
votes
3answers
42 views

To assign variable from a file separated by delimiter

I have a text file with datas as below separated by delimiter: 1000:Tom:user1 1002:Jerry:user2 1003:Victor:user3 1004:Jason:user4 I need to run a while loop (any loop) to fetch details from first ...
5
votes
3answers
98 views

The ERE regex to split() string between a delimiter and end-of-word

I'm using a long gawk 3.1.6 script to do a complex conversion of Zim markdown text into GtkDialog code and am stuck on the following problem... Sample ASCII input... [[link|label label]] [[link]] ...
-1
votes
1answer
87 views

extracting ip address from a file [closed]

I'm trying to extract all the IP address from a file. FILE *>i85.95.2.160/29 172.16.24.4 0 200 0 64515 4445 15412 i *>i85.112.80.0/28 172.16.24.4 0 200 ...
1
vote
1answer
53 views

Processing FreeBSD's PF logs with tcpdump/sed/awk

I'm running FreeBSD and have PF firewall set up and this is command i use to display daily number of firewall hits (blocked traffic) sudo tcpdump -n -e -tttt -r /var/log/pflog | grep $(date ...
1
vote
1answer
135 views

how to subtract a constant value from a data file?

I want to subtract a constant number from my data to get new data file. For example, I want to subtract 10 from my data. data.txt Id a b c d e 1 1 2 3 4 5 2 2 3 4 5 6 ...
1
vote
3answers
140 views

How to calculate the sum of the data that have the same ID in the first column?

I want to sum the values of the rows of my data that have the same ID in the first column. My data looks like data.txt Id a b c d e 1 1 2 3 4 5 1 2 3 4 5 ...
0
votes
2answers
64 views

Merging two different files with conditions

I have two different files as shown below. Content of a.txt: HDR|1|||||||||| DTL|@||||||||||| TLR||||||||||||| HDR|1|||||||||||| ...
0
votes
2answers
72 views

How to Lookup the values of an index values from a big data file?

I have a data file (data_array.txt in the example) and index file, index.txt, at which I want to extract the data from the data file that have the same ID in the index file and store in to a new file, ...
1
vote
5answers
268 views

Cut and transpose some lines of a file

I have 1344 rows of data column. I want to form 32 row by 42 columns from my input data file. I want to cut the first 42 row and paste to one raw of 42 column data and continue down likewise. My data ...
1
vote
4answers
90 views

sed remove end of line for specific lines

I'm trying to find the sed command to remove the end of the line (meaning, get the next line to the current), but only if line starts with the string NOTOK. example input: NOTOK something ...
-1
votes
1answer
50 views

Using AWK to monitor changing values every 60 seconds [closed]

Does any one have an easy way of doing this using awk, maybe associative arrays? One other thing is there is a delay of 60 seconds between these two input streams and any time a value on the right ...
0
votes
1answer
26 views

Copying specific text from one file into a schema

I have two files. One being a schema, and the other a list of data. The schema looks like: line_name: shot_edit: channel edit: cable_edit and repeats regularly for the number of lines that I ...
0
votes
2answers
115 views

Substring only the first column in awk

I want to take first 500 characters of the first column and leave the rest of the columns as it is, using awk. This is what I am trying to do: awk '{print ${$1:1:500} $2,$NF;}' doc.txt But I get ...
1
vote
2answers
150 views

How can I make awk on Busybox treat the entire input with NULL character in between as a single record?

I wrote a base64 encode script for my Busybox system. #!/bin/sh base64encode() { /usr/bin/awk -- ' function asc(char,l_found) { l_found=0; for (i=0;i<=255;i++){ if ...
0
votes
3answers
42 views

linux + awk + file manipulation

I have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 until the line END_OF_LINE , and from ...
0
votes
3answers
53 views

Print certain fields of each line until a marker is encountered, then print whole lines till the end of file

Hi have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 from each line until the line ...
2
votes
2answers
96 views

Processing two files using awk

I read Comparing two files using Unix and Awk. It is really interesting. I read and tested it, but I can't understand it completely and use it in other cases. I have two files. file1 has one field ...
0
votes
1answer
33 views

rearranging from colum to row in a csv [duplicate]

I have a CSV file that presently has an output like this 1 2 3 4 5 what I need to do is reconfigure the output as follows 1 2 3 4 5 all the entry data on one line, any help would be greatly ...
1
vote
2answers
73 views

awk split parameter by char

file is: BASH.NIR SH.ABI I want the awk will show: User is NIR, SHELL is BASH User is ABI, SHELL is SH I dont know how to split a parameter by char. The Idea is: cat file.txt | awk '{print ...
2
votes
3answers
242 views

awk print ' char

I want to print: cat file.log | awk '{print nir's $1}' output should be: nir's aaa nir's bbb nir's abc nir's dbc The problem is with the ' in nir's. I also tried: cat file.log | awk '{print ...
1
vote
3answers
67 views

How to add zero (0) in blanks in a big file (example file is below)

Here is a example file where I need to replace blanks with zero A 2 2 2 2 2 B 2 2 2 2 2 C 2 2 2 2 2 D A E B F D R
2
votes
1answer
98 views

Extract multiple lines with secondary keyword search

I have a text file with multiple log data, I would like to search for 'Keyword-1' in a line, and when it finds the line, to search within that line for another keyword such as 'engine-1'. Then ...
4
votes
5answers
1k views

extract the 4th column from a csv file using unix command

I have a csv file formatted as below. "col1","col2","col3","col4" "1","text1","<p>big html text</p> ","4th column" "2","text2","<p>big2 html2 text2</p> ...
1
vote
2answers
103 views

What does the following script do? [closed]

This was a question on my study guide, and I believe that the script is pinging the ip addresses, so the 2nd choice. Can somebody confirm this for me? The 5th column of NMAP output below is the IP ...
4
votes
2answers
48 views

Find holes in an enumeration in awk

I have a file that looks like this 1400a_comp 49472 2868636.1026583919660 GOOD 1400a_comp 49473 2868647.1026579919650 GOOD ...
1
vote
1answer
83 views

First line of awk output isn't what I expect

I am trying to figure out why a part of a file name disappears. In my case, I have created three files $ ls some-part-of-file.1 some-part-of-file.2 some-part-of-file.3 and when I use awk with FS ...
4
votes
3answers
174 views

How to only output lines that are younger than 3 months?

cat a.txt 1387111124 ./asfjlasdf.txt 1348681215 ./akdfyxcv.txt So I need this: cat a.txt | SOMEMAGIC 1387111124 ./asfjlasdf.txt The output should only have the lines that are younger than 3 ...
1
vote
2answers
218 views

Type URL and show HTTP status

I want to make a shell script where I type an url and it returns me the status code.So far I've tested wget is there any other cause it has the tendency to save whatever you run. Then I tried to some ...
5
votes
3answers
285 views

Extracting fixed width records with no delimiter from a single line

I need to extract strings of text from a single file containing one very long line of text with no delimiters. Using the sample line below, these are the following known facts: ...
0
votes
2answers
98 views

Format of SMTP headers for scanning inbox email

I am looking for a simple procedure to extract email from my inbox into a file for further processing, using e.g. awk. With "extract" I mean use e.g. regex to get emails with a particular subject. My ...
1
vote
4answers
181 views

How do I search for all the folders that begin with a dot ( . )?

My server had been infected and the attacker spread a bunch of .5randomletters folders around my file system, with malicious code inside of them. I want to search for all the folders that begin with a ...
2
votes
1answer
52 views

awk - streamline this script

I have a text file. On odd number lines, it gives new dimensions of key elements of a building that I'm constructing. On even number lines, the old dimensions are given. My aim is to process this ...
0
votes
3answers
906 views

Shell script read separately columns from file

When we have file which is a kind of list like : Mary 34 George 45 John 56 Josh 29 using the awk command $1 refers to the first column and $2 to the second column.But once we write a shell script ...
-1
votes
2answers
134 views

How to manipulate output from commands with shell scripts [closed]

Simple question, having a ls -l command it makes a list on the command prompt and if we want to manipulate it we pipeline it with awk command, for instance if we want to to show specific files. Is ...
2
votes
1answer
46 views

Matching two different lines that are at least N lines distant from each other

I would like to match files that contain try (in one line) and consecutive catch (in another line) at least N lines later (to find very long try-catches and try to get rid of them). I mean, the first ...
3
votes
3answers
111 views

Print unique of who command

I want to show the online users using the who command but I want to have a unique output and not show any duplicates. I piped the output into awk but I am not very familiar with that, is this the ...