awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

0
votes
1answer
26 views

Removing “,” for calculation and adding it back using awk

I have a file as January 124,567.89 Feburary 234,567.78 March 349,123.77 April 112,248.00 May 107,345.22 June 180,670.23 July 122,457.03 August 345,789.99 September ...
0
votes
2answers
35 views

Use a shell variable with Awk

I have this: if [[ $1 = "-c" ]] && [[ $2 =~ ^[a-z A-Z]+$ ]] then awk -v var="$2" '{print $2}' something.txt fi What I want is with a comand like -c abc to show the lines in something.txt ...
1
vote
2answers
44 views

Print smallest integer from file using awk custom function?

awk function looks like this in a file name fun.awk: { print small() } function small() { a[NR]=$0 smal=0 for(i=1;i<=3;i++) { if( a[i]<a[i+1]) smal=a[i] else smal=a[i+1] } return smal } ...
2
votes
4answers
1k views

How to subtract two times in shell

I nee to subtract two lines which is in the format of time in shell. The time format is hh:mm:ss I used the code below to get the time. cat /var/log/kern.log |grep usb |tail -2| awk '{print $3}' ...
1
vote
4answers
189 views

Move lines 2 3 after line 5 using awk

I have file content as: this line 1 no un1x this lines 22 0 butbutbut this 33 22 has unix but not 1 THIS is not butbutbut ffff second line awk 'NR==2 && NR==3 {f=$0;next} NR ==5 ...
0
votes
1answer
37 views

Consecutively copy two lines and skip the third using awk

With fairly simply awk: awk '(NR%3)' awk.write for this file: this line 1 no un1x this lines 22 0 butbutbut this 33 22 has unix but not 1 THIS is not butbutbut ffff second line I have output as: ...
0
votes
1answer
17 views

Awk print next after matched condition?

using awk 'f&&!--f;{if(length($1)>8){f=1}}' awk.write I can print line after the matched line. I know use of flags but have no idea what &&!--f really do? this line 1 no un1x this ...
2
votes
1answer
96 views

Why POSIX left `$(expression)` in awk unspecified?

Answering this question, I found that in: echo 1 2 | awk '{print $(expression)}' the return value of expression is unspecified. It can lead to confusion, as if expression is an uninitialized ...
1
vote
3answers
76 views

Add line to configuration file from a bash script?

I am trying to figure out to locate a line in a configuration file, then to drop down two lines then to insert a line of code. I was attempting to do this in awk/sed but got stuck on the carriage ...
0
votes
2answers
73 views

Unable to understand printing array results for awk?

For a simple file which contains: 1 a 2 b 3 c 4 d 5 e 6 f 7 g 8 h 9 i 10 j I use that command: awk 'lines[NR]=$0 { print ...
2
votes
2answers
65 views

Extracting subset from fasta file

I have a fasta file which looks like this: >chr1 ACGGTGTAGTCG >chr2 ACGTGTATAGCT >chrUn ACGTGGATATTT >chr21 ACGTTGATGAAA >chrX GTACGGGGGTGG >chrUn5 TGATAGCTGTTG I just want to ...
2
votes
1answer
53 views

How can I refer to an awk field defined with an arbitrary expression?

In shell scripting we have the ${name} construct to explicitly specify a variable name and separate it from surrounding text. In awk, I can refer to the fifth field with $5 or the Nth field as ...
3
votes
2answers
81 views

Merge and Append columns for more than 2 files

The files are tab separated. I have "N" number of files whose contents are as shown in the screenshot above. I want to merge them and append the 5th column. The first 4 columns are the same. I ...
2
votes
2answers
32 views

gawk: How i can print output genrated by gawk command in multiple line on terminal in single line in file

I am using gawk command to print column content in file like: gawk '{print $2205764, $2205763}' /home/mohasin/fil1.txt It generates output on terminal with new line: a a b b c c d d . . . . I ...
6
votes
1answer
85 views

How to extract first base64 code out of file?

On Linux I'm having text files and each contains 1 or 2 base64 blocks of code. Following is part of the file and 1 base64 block. Content-Transfer-Encoding: base64 ...
2
votes
2answers
158 views

How to keep the field separator when printing $0 with awk [duplicate]

I try to modify one column of my file, then print the result. awk -F"|" '{ if(NR!=1){$5 = $5+0.1} print $0}' myfile It does what I want, but when printing, only the first line keeps its field ...
1
vote
1answer
47 views

Simulate sed read command with awk

I want to stimulate use of sed: sed '3r awk.scr' awk.script with awk.scr: a b c d e f and awk.script as: hello there is hello i'am there is hello sdfdf dfdfdf aads 23213 3 434 Using awk ...
5
votes
2answers
181 views

Command to remove the first N number of lines in input

Background I am running an SSH server and have this user that I want to delete. I cannot delete this user because he is currently running a few processes that I need to kill first. This is the ...
4
votes
2answers
5k views

Remove lines based on duplicates within one column without sort

I have large 3-column files (~10,000 lines) and I would like to remove lines when the contents of the third column of that line appear in the third column of another line. The files' sizes make sort a ...
1
vote
2answers
35 views

Return line conditonally on next line

I am trying to return a list of IP Addresses for computers that have a windows OS (for my security course). The output is in this format Nmap scan report for 192.168.xx.xxx Host is up (0.066s ...
2
votes
2answers
58 views

How do I list all lines after two subsequent patterns are matched up to when they don't

I have files that are used to install files with specific owner:group and permissions. I'll ignore permissions for clarity and the details of installing files. Once an owner & group are set to a ...
3
votes
5answers
112 views

split column after nth character

I'm trying to split my second column in the file below after every 4 characters. file.txt >1A THISISATEST >1B THATISATEST desired output: >1A THIS ISATEST >1B THAT ...
6
votes
4answers
42k views

How to use regex with AWK for string replacement?

Suppose there is some text from a file: (bookmarks ("Chapter 1 Introduction 1" "#1" ("1.1 Problem Statement and Basic Definitions 23" "#2") ("Exercises 31" "#30") ("Notes and References 42" "#34")) ) ...
1
vote
0answers
33 views

How to extract weather data from a html doc using linux string manipulation

I need to extract these info from this weather forecast site, but I don't know how much about string manipulation in linux: acm_RecentLocationsCarousel.push({name:"Irvine, CA", daypart:'day', ...
2
votes
2answers
59 views

How to duplicate a file a number of times while embedding an index in each file

I have a file named, for example, ascdrgi.txt, with the following contents: tiger lion cat I want to duplicate this file a (variable) number of times by changing the last character of the filename ...
7
votes
4answers
4k views

remove blank lines in awk display

I was trying to figure out a solution for this question. I wanted to use awk for the solution. My input file is something like below. -bash-3.2$ cat file ramesh ramesh_venkat ramesh3_venkat3 ...
-1
votes
1answer
39 views

awk getline to display first three lines in file

I'm using awk 'BEGIN {while((getline gf < "awk.script") > 0) {print gf}}' to read me first 3 three lines of the pattern. I can do with head -3 . But I don't want to since, I have to switch ...
3
votes
2answers
181 views

merge duplicate rows in columns

Given a file like this: x y y z x x x y z z y x x x z y y z z y x x x x x x x x I would like the output to be: x y+ z x x+ y z+ y x+ z y y z+ y x+ x+ Is that possible to do with awk or perl in a ...
2
votes
6answers
160 views

Use sed to find and keep only the lines with 2 alpha characters in them

I have a file with many lines like these 33B87401 33B87402 33B87403 33B8EE44 33B87405 33B87406 33B87407 33B87408 33B87409 33B8740A 33B8740B 33B8740C 33B87D0D 33B8740E 33B8740F 33B87410 33B87411 ...
4
votes
4answers
111 views

Extracting strings from between special characters

I have a large text file created from combining lots of html files. cat *.html > all_html_files.txt Inside the text file are specfic strings that I want to extract to another text file. For ...
3
votes
4answers
63 views

Bulk data generation

I need to generate nearly 1 Billion records of unique integers. I tried with awk but it is not generating more than 5million records. Please help. Below is what I had tried so far - awk -v ...
1
vote
1answer
76 views

Want to print NULL if value is not present as awk output

I am trying to print a tshark command output using awk below is my command: tshark -r "test.pcap" -odiameter.tcp.ports:"1234" -R 'diameter.cmd.code == 272 and diameter.flags.request==0 and ...
-2
votes
1answer
24 views

insert patterns into text file after the last line [closed]

I use below codes to insert the lines after specific line into text file w.r.t. the user's variable. echo -n "WHICH STATIONS?" read station echo ${station[${i}]} awk -v input="$station" ' BEGIN { ...
1
vote
1answer
42 views

Awk split and conditional printing

I have an ugly data file that I need to clean up - Date User Name Item Id Title Quantity Price ...
1
vote
3answers
34 views

awk to replace columns and print if condition statisfy

I have a file with below output Sending showtrans string ... Oldest redo log files necessary to restart Extract are: Redo Thread 1, Redo Log Sequence Number 29334, SCN 3364.4078507030 ...
5
votes
5answers
433 views

Remove lines from file up to a pattern, unless the pattern doesn't exist

I am attempting to write a filter using something like sed or awk to do the following: If a given pattern does not exist in the input, copy the entire input to the output If the pattern exists in ...
11
votes
5answers
15k views

Delete range of lines above pattern with sed (or awk)

I have the following code that will remove lines with the pattern banana and 2 lines after it: sed '/banana/I,+2 d' file So far, so good! But I need it to remove 2 lines before banana, but I can't ...
1
vote
1answer
40 views

Unix awk decimal export in pipe

I'm struggling to adapt a script, which loops through a large number of individual files (subj_id), and prints out the value of $4 in a specified line. Text file contains lines # Measure Brain, ICV, ...
1
vote
3answers
89 views

Compare two files with awk

I'm trying to compare two files and print the output if they match on some fields. File1.txt: bart:29482164591748 apu:29482164591748 smithers:68468468468464 lisa:68468468468464 ...
0
votes
2answers
63 views

Substituting \ for \\ in column

I exported a column of thousands of salts from mysql for hashcat. Every instance of the character \ was replaced with a double character \\ in the exported column. Is there a way to either sed or awk ...
3
votes
3answers
281 views

awk manipulation of file

I have a big file in this format, whose first few lines I am showing you including the header. gene c1 c2 c3 c4 c5 c6 c7 c8 G1*1 0 0 0 0 0 0 0 0 G2*2 0 0 0 0 1 ...
3
votes
2answers
43 views

using printf function for repeated patterns

I have codes which append the patterns into text file w.r.t. the user's input as follows; echo -n "WHICH STATIONS?" read station awk -v input="$station" ' BEGIN { n = split(tolower(input), ...
0
votes
2answers
42 views

Remove both lines if column A values are repeated in next row

My input is like this: pak_c35079_g1_i1|m.14890 Pfam pak_c20797_g1_i1|m.3458 ProDom pak_c20797_g1_i1|m.3458 Pfam pak_c28265_g1_i3|m.9595 TIGRFAM pak_c28265_g1_i3|m.9595 ...
2
votes
2answers
58 views

Exchanging One Column From File1 with files from File2

I have two files. The first file has 11 columns, for example: 1 2 3 4 5 6 7 8 9 10 11 The second has 10 columns and might look like this: 11 22 33 44 55 66 77 88 99 100 What I want to do is look ...
1
vote
1answer
41 views

Unable to run awk program

I have created a simple awk program file in /media/ravan/data/program The code: #! /usr/bin/awk -f BEGIN { print "Don’t Panic bro!" } While executing in terminal: ...
3
votes
2answers
81 views

different results from awk and grep

This gives result 124: awk 'BEGIN {FS = ","; count = 0}; { if ($7 ~ /Nature Life/) { count++ }} END {print count}' file.csv This gives result 123: grep -cE '^([^,]*,){6}[^,]*Nature Life' ...
5
votes
5answers
1k views

Why doesn't awk make the sum zero but instead a very small number?

I have this file and I want to sum all the number in first column. Easy: awk '{s+=$1;print $1,s}' file 0.1048 -1.2705 0.4196 -0.8509 0.4196 -0.4313 0.2719 -0.1594 0.0797 -0.0797 0.0797 -5.55112e-17 ...
5
votes
4answers
354 views

How can I blank the nth to mth field using the awk command?

Can someone please give me a solution for the below using awk command? if any other command would help, that is also much appreciated. Below is the input U,N,UNIX,000,A,5 N,P,SHELL,111,B,6 ...
-1
votes
1answer
35 views

Trying to sum a decimal column from a csv file for each different vendor

I am new to Unix/Linux and am trying to sum a column for each vendor in a loop. I apologize for the vagueness. I will try to explain. It is suppose to read from a file and based on the vendor name ...
1
vote
1answer
48 views

Remove leading 0 from specific field

I have a file with multiple fields but only want to remove the leading 0s from the 2nd column (col B) rather than all of the columns. I have tried: sed 's/^0//g' filename > new_filename but this ...