awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
1answer
55 views

How can I refer to an awk field defined with an arbitrary expression?

In shell scripting we have the ${name} construct to explicitly specify a variable name and separate it from surrounding text. In awk, I can refer to the fifth field with $5 or the Nth field as ...
2
votes
2answers
355 views

How to keep the field separator when printing $0 with awk [duplicate]

I try to modify one column of my file, then print the result. awk -F"|" '{ if(NR!=1){$5 = $5+0.1} print $0}' myfile It does what I want, but when printing, only the first line keeps its field ...
5
votes
2answers
267 views

Command to remove the first N number of lines in input

Background I am running an SSH server and have this user that I want to delete. I cannot delete this user because he is currently running a few processes that I need to kill first. This is the ...
6
votes
1answer
158 views

How to extract first base64 code out of file?

On Linux I'm having text files and each contains 1 or 2 base64 blocks of code. Following is part of the file and 1 base64 block. Content-Transfer-Encoding: base64 ...
3
votes
2answers
121 views

Merge and Append columns for more than 2 files

The files are tab separated. I have "N" number of files whose contents are as shown in the screenshot above. I want to merge them and append the 5th column. The first 4 columns are the same. I ...
1
vote
2answers
38 views

Return line conditonally on next line

I am trying to return a list of IP Addresses for computers that have a windows OS (for my security course). The output is in this format Nmap scan report for 192.168.xx.xxx Host is up (0.066s ...
3
votes
5answers
289 views

split column after nth character

I'm trying to split my second column in the file below after every 4 characters. file.txt >1A THISISATEST >1B THATISATEST desired output: >1A THIS ISATEST >1B THAT ...
1
vote
1answer
62 views

Simulate sed read command with awk

I want to stimulate use of sed: sed '3r awk.scr' awk.script with awk.scr: a b c d e f and awk.script as: hello there is hello i'am there is hello sdfdf dfdfdf aads 23213 3 434 Using awk ...
1
vote
0answers
44 views

How to extract weather data from a html doc using linux string manipulation

I need to extract these info from this weather forecast site, but I don't know how much about string manipulation in linux: acm_RecentLocationsCarousel.push({name:"Irvine, CA", daypart:'day', ...
-1
votes
1answer
65 views

awk getline to display first three lines in file

I'm using awk 'BEGIN {while((getline gf < "awk.script") > 0) {print gf}}' to read me first 3 three lines of the pattern. I can do with head -3 . But I don't want to since, I have to switch ...
2
votes
2answers
68 views

How to duplicate a file a number of times while embedding an index in each file

I have a file named, for example, ascdrgi.txt, with the following contents: tiger lion cat I want to duplicate this file a (variable) number of times by changing the last character of the filename ...
3
votes
2answers
207 views

merge duplicate rows in columns

Given a file like this: x y y z x x x y z z y x x x z y y z z y x x x x x x x x I would like the output to be: x y+ z x x+ y z+ y x+ z y y z+ y x+ x+ Is that possible to do with awk or perl in a ...
2
votes
2answers
60 views

How do I list all lines after two subsequent patterns are matched up to when they don't

I have files that are used to install files with specific owner:group and permissions. I'll ignore permissions for clarity and the details of installing files. Once an owner & group are set to a ...
2
votes
6answers
228 views

Use sed to find and keep only the lines with 2 alpha characters in them

I have a file with many lines like these 33B87401 33B87402 33B87403 33B8EE44 33B87405 33B87406 33B87407 33B87408 33B87409 33B8740A 33B8740B 33B8740C 33B87D0D 33B8740E 33B8740F 33B87410 33B87411 ...
-2
votes
1answer
31 views

insert patterns into text file after the last line [closed]

I use below codes to insert the lines after specific line into text file w.r.t. the user's variable. echo -n "WHICH STATIONS?" read station echo ${station[${i}]} awk -v input="$station" ' BEGIN { ...
3
votes
4answers
68 views

Bulk data generation

I need to generate nearly 1 Billion records of unique integers. I tried with awk but it is not generating more than 5million records. Please help. Below is what I had tried so far - awk -v ...
3
votes
4answers
127 views

Extracting strings from between special characters

I have a large text file created from combining lots of html files. cat *.html > all_html_files.txt Inside the text file are specfic strings that I want to extract to another text file. For ...
1
vote
1answer
65 views

Awk split and conditional printing

I have an ugly data file that I need to clean up - Date User Name Item Id Title Quantity Price ...
1
vote
3answers
41 views

awk to replace columns and print if condition statisfy

I have a file with below output Sending showtrans string ... Oldest redo log files necessary to restart Extract are: Redo Thread 1, Redo Log Sequence Number 29334, SCN 3364.4078507030 ...
1
vote
1answer
55 views

Unix awk decimal export in pipe

I'm struggling to adapt a script, which loops through a large number of individual files (subj_id), and prints out the value of $4 in a specified line. Text file contains lines # Measure Brain, ICV, ...
1
vote
3answers
96 views

Compare two files with awk

I'm trying to compare two files and print the output if they match on some fields. File1.txt: bart:29482164591748 apu:29482164591748 smithers:68468468468464 lisa:68468468468464 ...
3
votes
2answers
50 views

using printf function for repeated patterns

I have codes which append the patterns into text file w.r.t. the user's input as follows; echo -n "WHICH STATIONS?" read station awk -v input="$station" ' BEGIN { n = split(tolower(input), ...
0
votes
2answers
46 views

Remove both lines if column A values are repeated in next row

My input is like this: pak_c35079_g1_i1|m.14890 Pfam pak_c20797_g1_i1|m.3458 ProDom pak_c20797_g1_i1|m.3458 Pfam pak_c28265_g1_i3|m.9595 TIGRFAM pak_c28265_g1_i3|m.9595 ...
5
votes
4answers
382 views

How can I blank the nth to mth field using the awk command?

Can someone please give me a solution for the below using awk command? if any other command would help, that is also much appreciated. Below is the input U,N,UNIX,000,A,5 N,P,SHELL,111,B,6 ...
1
vote
1answer
133 views

Remove leading 0 from specific field

I have a file with multiple fields but only want to remove the leading 0s from the 2nd column (col B) rather than all of the columns. I have tried: sed 's/^0//g' filename > new_filename but this ...
-1
votes
1answer
59 views

Trying to sum a decimal column from a csv file for each different vendor

I am new to Unix/Linux and am trying to sum a column for each vendor in a loop. I apologize for the vagueness. I will try to explain. It is suppose to read from a file and based on the vendor name ...
0
votes
3answers
95 views

Matching lines in two files [duplicate]

Let's say I have two files, file1 and file2. file1: passd:xxx hopla:alli gnar:gungg araf:utap file2: alli utap How can i check what lines/words from file2 match file? Indeed I could probably do ...
1
vote
1answer
37 views

Matching 2 main columns between files; and paste other columns into the output file when those main columns match. Keep row size of 1st file intact

This is a follow-up question to my previous question asked about 24 hours ago. Link to previous question: Previous Question Link G-Man solved that problem with a useful code, but I have a follow-up ...
2
votes
1answer
87 views

Matching two main columns at the same time between files, and paste supplementary columns into the output file when those main columns match

I have 3 files, each with a unique number of columns, all tab-separated, but some columns are shared between the 3 files. It's the shared columns between the 3 files that I want to use to create some ...
0
votes
1answer
45 views

awk script to extract the argument of an option

Can some one explain what the awk '{for(i=1;i<=NF;i++) if($i~/-f/) print $(i+1)}') part does in the following snippet line='/wwws/apache/apache2.4.16w-r01/instroot/bin/httpd -f ...
4
votes
4answers
151 views

Use awk interactively through a pipe

Looking at this question, I've noticed that awk can't read user input if the file is being piped into standard input, but it does behave as expected if reading input from a file given as a command ...
-4
votes
1answer
35 views

I'm using the following snippet for Log rotation

Can some one explain how this worksps -ef | grep mongod | awk '$0!~/grep/ && $2~/[0-9]/{print "kill -SIGUSR1 "$2}' | bash; find /tmp/logs/* -type f -mtime +7 -exec rm {} \;
0
votes
4answers
47 views

How to remove a first character of that particular sentences from a list of output [duplicate]

grep /var/log /etc/rsyslog.conf | awk '{print $2}' Output: /var/log/messages /var/log/secure -/var/log/maillog /var/log/cron -/var/log/spooler /var/log/boot.log I want the result to be: ...
1
vote
0answers
46 views

Unix: C-shell:Disk space checking in Hp-UX

i have 10 devices that using hp-ux and i want to check the disk space in each devices. my requirement is if the space more than 90%, the info of device ans space will be save to a log. this is list of ...
0
votes
2answers
61 views

awk print to file

I have a big c file that gives a lot of output that I don't care about right now. I wrote this awk script to do mostly what I want. Not sure how to put the finishing touches on it. I would also like ...
4
votes
1answer
71 views

How to use Awk to format numbers with a thousands separator

Can anyone tell me how to use the awk command for this statement "the number will contain commas after every three digits. For example, 100000000 will become 100,000,000." I know how to use sed ...
0
votes
1answer
41 views

writing of user's input with loop into text file

echo -n "##### STATIONS? #######" read station awk -v input="$station" ' BEGIN { n = split(tolower(input), user) fmt = "%-4s\n" } {print} /stab_site clear/ { for (i=1; ...
0
votes
1answer
31 views

Make new line above last line and add something there

Like a reverse of awk 'NR==2 {print ""} 1 which adds a new line under the first and prints whatever there. I can't replace the number after NR== with a fixed value because I want to be able to do this ...
0
votes
0answers
28 views

Canvas(blackboard)-like program

I have to make a program that has most of the functions those websites have and I am stuck at the point where I have to get the grades from a file and use a formula to calculate the students final ...
3
votes
3answers
83 views

Print all matching lines and the previous unindented line

For example, lets say I want to find all lines with 'matching' and the previous unindented lines in this text. Container 1 some text some text matching text some text Container 2 some ...
1
vote
2answers
52 views

Parsing with awk by modifying the field and record separators

I have a file which is as follows: ------------------------------------------------------------------------------------------------------------------------------------------------------------------ ...
1
vote
1answer
83 views

Inserting variables in a command within a shell script

I'm attempting to write an automated script which will update the IP address for my Raspberry Pi within my /etc/hosts file. I'm able to execute this line just fine, IP=`sudo nmap -sP 10.61.4.0/24 | ...
2
votes
1answer
183 views

GREP exclude first 11 values

I have a massive log file that I need to filter. In this log, I want to display all logs that contain the string dns and I want see them just once. i.e from this: Dec 9 07:24:02 94.15.218.140 ...
0
votes
1answer
24 views

How to see the lines whose date column is less than x day from today

My file ID=29524 02/12/2015,23:59:34,'PAt, 0, ID=29628 02/12/2015,23:59:37,'PAb, 1.9, ID=30816 03/12/2015,00:00:20,'PAc, 4.15, I would like to print files with X days less than from today. My try ...
0
votes
2answers
304 views

Search for a specific word in each line and print rest of the line

I have text file with server URL lines like: request get https://abc.net/search?q=hello/world/hello/word/search=5&size=10 request get https://abc.net/search?q=hello/world/hello/world/hello/word=5 ...
1
vote
3answers
117 views

How to sort blocks of lines with a specific line from the block?

I have a file with the data like this BEGIN hello2 5 world1 END BEGIN hello4 2 world5 END BEGIN hello6 4 END I want to sort the lines in the following manner based on the number inside the block. ...
-1
votes
2answers
55 views

Sorting multiple line columns

In the text: 35 EAST 23rd Street SOUTH AFRICA 5 600 5000000 6 83 NORTH YELLOWLIGHT AVENUE SOUTH AFRICA 4 700 7000000 5 777 NEW AVENUE SAUDIA ...
1
vote
1answer
169 views

I am trying to fetch the logs for last five minutes through a script

I have a output log file which will be continuously appended and I am planning to have a script (runs for every 5 min) to retrieve the last 5 min data from of the output log. Sample data from the ...
0
votes
2answers
116 views

Shell script to kill processes except those owned by user

I am Looking for some help in creating a script to kill processes, but return an error message if they are owned by root. Firstly I would like to say that I am a total beginner at this. At the moment ...
0
votes
1answer
79 views

awk print out file with a row number

I want to print out a file that has 1 column listing values like: cat dog house But I want to have the file print like this to another file: 1 cat 2 dog 3 house I think that awk could be used to ...