awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
2answers
20 views

Deleting lines by matching 3rd and 4th character only

I am running a script that fetches me the data as given below, 1400d2b8 1f4d1e2507 1200781e As you can see the 3rd and 4th character is 0 in line 1 and line 3. I am trying to write a single line ...
0
votes
1answer
40 views

Using Perl for Even/Odd numbers

Currently, I'm using a process to create a very specific set of four files. The Process I'm using is shown in the attached Process Image, below: The first line is part of a solution, given to a ...
1
vote
1answer
33 views

How to output multiple lines using a single condition?

I have a file with the pattern: n0 n1 n2 ... ni ------------------------------- N0 N1 <empty line> if N0 is less than a specific number, I want: 2 lines above the N1's line N1's line Empty ...
1
vote
2answers
63 views

How to apply an awk script to multiple files in one folder using a loop

I am trying to apply one awk script to all the files in one folder, I am using: for %%i in (*.txt) do (awk -f NormalizeFiles.awk "%%i" > new\"%%i") but then I get the message: Syntactic error ...
2
votes
1answer
39 views

grep a range of unix timestamps from a logfile

I have a logfile where every log event has a unix timestamp: 1460332800 event a 1460332999 event b 1460342800 event c 1460417039 event d Is there a clever unix-style way to grep all log events ...
6
votes
3answers
82 views

sed/awk subscript numbers in chemical formulas using markup tags

I have hundreds of text files that include chemical formulas together with narrative including numerical values. The formulas are always preceded by white spaces but can be followed by white spaces, ...
1
vote
3answers
45 views

Need help defining a variable in awk

I have an input file thus: 3.59717487E+05 3.40210880E+06 4075.32 7066.00 4075.32 7066 4075.322 2 a_final_psdm_LY1-1250_20160307 3.59725248E+05 3.40211860E+06 ...
0
votes
1answer
34 views

Printing all lines around a string until matching another string [duplicate]

I have an example dataset where a section has fields beginning with "something". I would like to obtain all of the lines in each "something" section when matching a specific string of "1234". I was ...
1
vote
2answers
52 views

I want search for a string and to pull all the lines between two patterns [duplicate]

I want search for a string and to pull all the lines between two patterns(including the pattern). START#1 Name S 101 Frankfurt_1_1 19 A+ Street END START#2 Name B 104 Berlin 20 A+ ...
2
votes
1answer
37 views

grep/awk specific lines based on specific fields; using ksh variable with awk

I have this input file: file_in.txt (delimited by pipe) 3345:tyg|rty|27|0|0|ty6|{89|io|}62|0 3346:tyg|rtyuio|63|0|1|ty6|{89|gh|}45|0 3347:tyu|ray|24|0|0|ty6|{89|uh|}27|0 3348:tyg|rtoy|93|0|1|ty6|{89|...
1
vote
1answer
25 views

Appending a value to all columns

I have a "simple" question yet I have not found an answer to it nor have I gotten my code to work. I have a bunch of columns, and I want to append the average of the last column to ALL columns. For ...
3
votes
2answers
92 views

Search, insert, move and insert

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's AGE line and move that line to the top ...
1
vote
1answer
38 views

How to add an word for a line where transpose has done

I have a file fruits: apple grapes banana orange By using cat < fruits | tr "\n" "," It will give output as apple,grapes,banana,orange, Here I want to print the file name also along with ...
2
votes
2answers
42 views

Using awk to combine two csv files on three fields too create a new csv file

I would like to compare fields in 2 csv file and create new csv file for output. I have 2 csv files; this first file is PCs in my network: 192.68.205.76,189,FC3F.DB02.ED78,2ndfloor-c001a-asw1,...
1
vote
1answer
49 views

I want to search and replace a pattern

I want to search each record (records are defined by blank lines) in a file for the pattern NAME#AAAA. If it matches, then insert an # in front of the record's age and insert a line age NIL at the end ...
0
votes
0answers
16 views

I want to search for a pattern in a file. If the pattern matches, it should check for second pattern and replace it with another pattern file [duplicate]

I want to search for a pattern in a file. If the pattern matches, it should check for second pattern and replace it with another pattern INPUT FILE Name#AAAAA std 1 sec c age 5 Name#BBBB std 2 ...
0
votes
2answers
54 views

calculate ratio using awk

awk 'NR==1{$4="ratio"}NR>1{$4 = ($3)/($2)} {print $1 "\t" $2 "\t" $3 "\t" $4}' A B awk: cmd. line:1: (FILENAME=A FNR=18) fatal: division by zero attempted Can anybody help with this error? ...
2
votes
2answers
137 views

bash source script.sh with exit 0

I use a lot of external files in my scripts. But I have a problem, that if I source a file with exit in the contents of the file, the script exits from loading and the strings that I want to use are ...
1
vote
3answers
64 views

Bash script to replace some words present in one file from another file

I have two files, one, file1 is 1000 lines with router configurations in it. A portion of it is like this - ! ipv6 prefix-list CCAV6 seq 5 permit 2217:AB23::/40 route-map DENY-ALL deny 11 ! ip ...
0
votes
0answers
20 views

Printing specific column in bash script and avoiding duplicates [duplicate]

I am writing a Bash script to print a specific column from a file. But that column contains many same strings and I want to just print every content just once. I tried this - awk '!seen{print $9}' $...
2
votes
4answers
299 views

print specific line from multiple files

I have a collection of 5000+ files and I just want to create a output.txt file which contain 27th line of all file along with filename. well I am noob at linux level so what i got from internet is to ...
0
votes
3answers
25 views

How to get second directory name in the perforce file path?

I need to get the directory in the second column after depot. For example given the below file(s) list, I need "Directory2" as the output. //depot/Directory2/Dir3/Dir4/Dir5/Dir6/Filename1 //depot/...
-3
votes
1answer
27 views

AWK interval expressions to clean IP [closed]

Credit to a great many entries from you online and manuals. It works, but, is it efficient enough? Do you approve? {Reading logs +- 5k items fast as I can see} Source data from file "myAwk3.txt": ...
2
votes
7answers
2k views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
1
vote
1answer
44 views

Sed, Awk Multiline log file, a number {0-999] identify relation

Short version of issue: Need to extract from logs: {Using sed and awk on Linux} Time, PlayerName{Can have many special characters}, GUID, IP from log example at end. Longer version: How to combine 2 ...
1
vote
4answers
247 views

Add line to configuration file from a bash script?

I am trying to figure out to locate a line in a configuration file, then to drop down two lines then to insert a line of code. I was attempting to do this in awk/sed but got stuck on the carriage ...
2
votes
2answers
29 views

AWK - how to get contents of a file pointed by first column of a file

I have a file f1.txt which has contents like below file1 file2 file3 file4 and file1 has contents like below student1 10 student2 20 Now I want to get the first column from f1.txt and print out ...
0
votes
1answer
29 views

Delete Non numeric row from a csv

I have used sed to allow only numeric and csv format ,I have used sed '/[^0-9, -, .]/d' as regex. It has still "_","1-0","1-1" character as o/p which worries me. sed 's/'"$(printf '\015')"'//g' ./...
-2
votes
2answers
48 views

Can someome explain me the below awk substring command val=$( echo $text $len|awk '{print substr($0,0,$2)}')

command: val=$( echo $text $len|awk '{print substr($0,0,$2)}') The above command works fine. I just want to understand the values taken by $0 and $2. I understand $0 takes the value of $text and $2 ...
0
votes
1answer
65 views

replace long text string (script with MANY special characters). sed, awk, grep

My site was hacked / infected. I replaced the url of the malicious link, but other elements in the malicious script are still making my site get blocked. Without inserting a hundred or so "escapes", ...
0
votes
2answers
46 views

How to take the values from two columns in a txt file and match them to values in another

I have two text files formatted as follows: e.g. File1: 900 480 10060.62 2740 -266864.19 3843493.50 2740.00 2740 176.07 900 479 10046.50 2741 -266874....
1
vote
5answers
11k views

Printing a new line using awk

I have a load of files in which I need to take out specific rows and then put the data I've taken out into a spreadsheet. An example would be my file shows: Name: w Age: x Height: y Weight: z I ...
4
votes
3answers
71 views

Split file by number of lines including header in each one

I need to split a .txt file into smaller ones containing 100 lines each, including the header. I don't know if this is relevant, but the original file is delimited like this: COLUMN1 | COLUMN2 | ...
3
votes
3answers
55 views

Grep that works with overlapping patterns

I want to count the number of repeating patterns aa in a very big file. So I use the following command: grep -o "aa" ./bwt/dblp.txt | wc -l This works, except when I have a string such as aaa, ...
0
votes
2answers
70 views

Select rows based on the values of a specific column (not the first column)

I would like to read a file.csv file and keep the rows (i.e., all the columns) of the file if and only if the sixth column of that file matches values in user_ids_to_keep.csv. The file.csv file I ...
3
votes
3answers
68 views

Get lines between two timestamps from a file

I have this large file that I need to go through using date and time. Every line in this file has a date and time in it. I need to search through the lines and get the lines that lie between two ...
16
votes
4answers
14k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
10
votes
12answers
3k views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
-1
votes
2answers
36 views

awk print all matching lines including duplicates [duplicate]

How can I print all matching lines between two files including the duplicates sourcefile: SNP value RS1 0.55 RS2 0.52 RS3 0.25 RS4 0.22 queryfile: SNP RS1 RS2 RS2 RS3 RS4 RS4 Desired output: ...
0
votes
2answers
39 views

Remove lines in a file where pattern only occurs once

I have a CSV file. It is ordered by an identifier the first field. The identifier in the first field can appear in either 1 or in 2 lines in the field. I want to remove all lines in the file where ...
0
votes
1answer
19 views

Check if stdout is attached to a terminal from an AWK script

Is there any way I can check if an AWK script's stdout is attached to a terminal, just like test -t 1 in a shell?
0
votes
1answer
39 views

Awk to get same column with maximum filed

I have two files. I have to get matching name from file1, print the entire content of file1, and add a fourth column (exp) in file 1 using a second file with max value file1 name value1 ...
1
vote
1answer
46 views

Awk field printing within the ternary operator

If you're familiar with DISA STIGs, the test for RHEL-06-000518 is /bin/rpm -Va 2>/dev/null \ | /bin/grep '^.M' What I want to do is pull just the file name out of that test, and make pretty ...
1
vote
1answer
18 views

Awk problem: an 'if column conditional' replaces the column entry!

I'm trying to get a grip on awk. Suppose I have an input: this header fred blah 1 blank fred blah 1 blank huh blah 2 blank fred blah 3 blank fred blah 3 blank and after I use the following awk ...
0
votes
2answers
54 views

get day from date with append 0 in 1 to 9 digit

i am using awk command to get creation date of file and from that i m fetching day,month and year but for 1-9 digit i am getting output like 2016041 but i want output 20160401. D=$(ls -ltr $line | ...
1
vote
1answer
66 views

Break up large log files

I am trying to break a large log file into smaller files based on date. The file is of the following form, where some lines may not have a date. Those lines should be included with the previous ...
2
votes
1answer
66 views

POSIX Awk sort array

I would like to sort an array with POSIX Awk, however I have discovered that: asort() asorti() PROCINFO["sorted_in"] are all GNU extensions. Can this be done with without GNU Awk?
10
votes
6answers
50k views

How to use a shell command to only show the first column and last column in a text file?

I need some help to figure out how to use the sed command to only show the first column and last column in a text file. Here is what I have so far for column 1: cat logfile | sed 's/\|/ /'|awk '{...
1
vote
2answers
171 views

Printing all lines of a file with multiple columns in a shell script

I have a file with 4 columns with the below format: name;user_id;dob;date_of_joining I want to print all the variables in below output via shell script The employee "$name" is having "$user_id" ...
0
votes
1answer
74 views

Ternary operator and print

What is wrong with below command awk 'BEGIN {a = 5;b=6; (a==b)? print "a==b" : print "a!=b"}' Output : awk: BEGIN {a = 5;b=6; (a==b)? print "a==b" : print "a!=b"} awk: ^...