awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

2
votes
1answer
47 views

Adding two columns together in CSV and outputting to new CSV file

I have a problem I have been trying to figure out: We have a stock file CSV, which contains the stock in multiple locations. The csv looks like this: ...
4
votes
6answers
154 views

Make longer strings from substrings keeping new line between each string

I have a file that contains strings as: GTACGACGGAGTGTTATAAGATGGGAAATCGGATACCAGATGAAATTGTGGATCGGTGCAAAA GTCGGCAGATATCGTTGAAGTCATAGGTGATTATGTTCAATTAAAGAAGCAAGGCCGAAACTAC ...
0
votes
1answer
39 views

Finding Matches between two columns of different files and printing it out a different column

I was trying to figure out how to see if there were any matches between column 2 of file1 and column 6 of file2. If so, I wanted that match printed along with the respective string in column 14 of ...
3
votes
3answers
73 views

Passing variables to the sed command,

We have a file containing 6 columns. FILEDESCRIPTOR . DESCR00001 FILEDESCRIPTOR . DESCRIPTIVENAME FILEDESCRIPTOR . LSTUPDNAM FILEDESCRIPTOR . LASTUPDATENAME FILEDESCRIPTOR . LOCAT00001 ...
4
votes
3answers
76 views

awk ward problem

I'm trying to use awk to pull some different fields from a string. Example: selected="a2.flac|a3.flac|a4.flac" first_file=$(echo "$selected" | awk -F'|' '{print $1}') The above code returns ...
0
votes
1answer
45 views

Search for a pattern in a file and move it to the new files one by one

I am trying to search for a particular pattern in a file and move it to a new files one by one. I am a noob and I tried sed -n 's/CREATE PROCEDURE/,/END ;/p' File It is again printing the same file. ...
0
votes
2answers
23 views

Passing timestamp variable from shell to gawk [duplicate]

I want to take time difference of two time stamp but getting an error while passing timestamp variable from shell toawk. Shell code: FTIMESTAMP="2015-07-01 12:30:50" gawk -v FTIMESTAMP=$FTIMESTAMP ...
-1
votes
2answers
81 views

Printing all lines of a file with multiple columns in shell script

I have a file with 4 columns with the below format name;user_id;dob;date_of_joining I want to print all the variables in below output via shell script The employee "$name" is having "$user_id" ...
1
vote
4answers
55 views

Add a space or tab before the last character in each line

I have a file with a list of values that end with a letter. Is there an easy way to add a space or tab before the last character of each line to separate the number from the letter? 44A 354T ...
1
vote
1answer
108 views

How can I write a bash script that sends a graphical notification to all users' X displays?

This is a request for improvement and update (NOT a duplicate!) on these 2 questions & answers: Can I launch a graphical program on another user's desktop as root? Show a notification across ...
1
vote
2answers
80 views

How can I use the environment variables inside a regular expression in awk?

#!/bin/sh ALTER="1153" NAME="aio" for i in $(ps -eo pid,etime,comm | \ awk -v alt="$ALTER" -v name="$NAME" '$2~/[^0-alt]-/ && $3~/name/{print$1} '); do echo $i done
12
votes
6answers
2k views

How to delete the last column of a file in Linux

I want to delete the last column of a txt file, while I do not know what the column number is. How could I do this? Example: Input: 1223 1234 1323 ... 2222 123 1233 1234 1233 ... 3444 125 0000 5553 ...
-1
votes
2answers
52 views

how to run a command in many files by running the command only one time?

I would like to run following command on 31 separated file name chr1.txt ... chr31.txt , and get 31 outputs only by one run. this is command : cat input.txt | sed 's/.\{5\}/& /g' > ...
2
votes
1answer
42 views

How to limit printed output based on number of occurences (AWK)

So I am trying to track down individuals on our network surfing porn via https. The pattern is pretty easy to see, 15+ hits in a row for images without a referrer in the HTTP header on our Bro logs. ...
3
votes
2answers
78 views

Using find -exec how to print name of file in front of each line?

Similar to, but beyond: find . -iname "somepatter*txt" -print -exec cat {} \; How might the file name from each file be printed in front of each line with the results from find?
3
votes
3answers
86 views

How do I create uniform columns from text containing both tabs and spaces?

In the simplest form, let's say I have an auto-generated file called file.txt. The contents of file.txt appears as follows: Source Destination Maximum To Maximum ...
1
vote
2answers
49 views

How do I stream edit progress-like output?

I'm having trouble editing some progress output. In this case it's Postgres' pg_basebackup, but it's similar to other progress output like curl and wget. Take for example the output of this: ...
0
votes
4answers
241 views

How to sum up values of each two rows across their line in linux

I have a data like this: input.txt 1 0000100101000000 1 0000010100000000 2 1110000001000000 2 1111000000001000 3 0000000111111111 3 1111111100000000 4 8888345500000000 4 0000000000000000 and I ...
-1
votes
0answers
15 views

How to accumulate values of each two rows across a line in linux? [duplicate]

I have a data like this: 1 0000100101000000 1 0000010100000000 2 1110000001000000 2 1111000000001000 3 0000000111111111 3 1111111100000000 and I want to sum up the values within eachtwo rows with ...
4
votes
5answers
403 views

Sum of filesize of a list of files

I have a text-file that contains a list of filenames (one filename per line). Now I would like to calculate the size of all these files. I think I will have to do a ls -la on every line of the file ...
1
vote
3answers
185 views

Help with awk / sed shell script

I have to make a script using the info in the following table (fake info) AnimalNumber,DOB,Gender,Breed,Date-moved-in IE161289240602,04/02/2010,M,AAX,20/07/2011, ...
6
votes
5answers
299 views

Find 2nd Occurrence of string from the end of file

I have a log file that looks like this, Another thousand lines above this I 10/03/15 12:04AM 42 [Important] 4th to last I 10/03/15 04:31AM 42 (534642712) [1974,2106,258605080,0,0,32817,30711] I ...
2
votes
1answer
112 views

How can I append an incremental count to every line of a text file?

I have a large document with over 500000 lines like this car milk dog And I want to count every line and add the number in the end of line and the final document to look like this car1 milk1 dog1 ...
1
vote
3answers
48 views

awk script to delete blocks in json

I have a newline delimited JSON file with entries like this: {"id":"eprints.ulster.ac.uk/view/year/2015.html","title":"Items where Year is 2015 - Ulster Institutional ...
1
vote
4answers
89 views

Find the smallest numbers in the second column corresponding to index values in first column

I have some very large tables and I need to extract specific rows. I am illustrating the task using a simple example. Say, I have weighed a number of apples, bananas and oranges. I need to extract the ...
0
votes
2answers
71 views

print field that contain a certain value

is it possible to print the field that contain a certain value? Consider below file: foo,boo,123,1234312,ABC foo,boo,ABC,bla,bla There is a field that contain ABC but it is not a fixed with all ...
4
votes
1answer
71 views

Printing more than one field

Is there a way to print multiple fields with a single line command? I want to print $3 to $NF, I can do it like below: awk -F[:] '{print $3,$4,$5 .... $NF}' Yet it is not doable with large ...
1
vote
2answers
76 views

Compare two dates with different format [closed]

I have two files, each file contain a different date's format as below: File1: 2014-NOV-0620:59:59 File2: 2014-12-31 How can I compare those two dates in awk?
1
vote
1answer
142 views

awk - how to ignore wildcard matches?

I want to process a csv file based on 3rd column value and save the matching lines to a separate file. #!/bin/bash awk -F, '$3 ~ /0/ {print}' ./inputfile.csv > ./out/output.csv What I want is ...
0
votes
3answers
52 views

Reformatting a table with awk

I have to reformat a number of very long tables as follows Original format: John Smith,Jones,Taylor Janet Williams,Brown,Wilson Desired format: John Smith John Jones John Taylor Janet Williams ...
1
vote
2answers
109 views

Awk condition along with header

I have a dummy employee.txt file as follows to play around with awk: 100 Thomas Manager Sales $5,000 200 Jason Developer Technology $5,500 300 Sanjay Sysadmin Technology $7,000 ...
0
votes
0answers
20 views

awk - how to reorder the contents of a file based on another pattern input file

I have a huge file with a list of names/characters in each line. The good thing is each line is unique, not sorted, but the order makes some sense and same needs to be retained. This is the file to be ...
1
vote
1answer
28 views

Read out File during awk

My script continuously reads a file with awk and evaluates input data. Now I need to read and write the first line of the output file. There is the information, what in which row is (°C of first ...
0
votes
1answer
38 views

how to extract some rows in a data file by comparing with another data file? (merging) [duplicate]

I have 2 different file : first file: id1 19501112 id2 19831207 1d4 20001212 id6 20000101 id10 20061213 id11 20081212 id12 20100109 id13 20101218 id14 20101212 id15 20111111 id16 20150919 id20 ...
4
votes
5answers
440 views

How to split a file based on information within a column in linux

I would like to separate those rows where the value in second column is smaller than a specific value. example input: id1 19501112 id2 19831207 id3 20001212 id6 20000101 id10 20061213 id11 20081212 ...
5
votes
4answers
241 views

how to split a file into separate files based on the column headers in original file?

I would like to split a file to different files based on the information in the first line. For example, I have: Input: 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 4 30 30 30 30 0 2 2 0 2 ...
4
votes
3answers
271 views

How to convert a single column file into a single row?

I would like to convert my one colum of many rows into one line: example : 1 1 1 1 1 2 2 2 3 3 4 and I want : 11111222334 any suggestion please?
1
vote
2answers
30 views

extract part of the text from all matching lines

I am running a for loop against set of servers to find the cert expiry. Output of the cert command will be below for each server: line 1 line 2 line 3 Valid From: Tuesday, 29 September 2015 10:40:13 ...
1
vote
2answers
22 views

Duplication of each line while changing some value within each copy of a line differently in linux

I Would like to have each line in a file repeated a two number of times while in in first copy of line any value of 3 be changed into 2 and any value of 4 be changed into 1, and in second copy of each ...
0
votes
2answers
56 views

select and move the contents to another file

I have a file with the below contents. All I need to do is filter the ALTER script that has UNIQUE keyword in it and move it to a different file. I tried using sed -n '/ALTER TABLE/,/UNIQUE("ID");/p' ...
0
votes
0answers
24 views

How to restore sbatch array job output file names to their original names

In order to submit array jobs using Slurm, I've renamed hundreds of files with incremental numbers using the following command: COUNT=1 for file in ./*; do mv "$file" "${COUNT}" let COUNT++ ...
0
votes
0answers
34 views

Issue with netstat script

I am trying to run the following script to capture the netstat data and put it in to log file. But this is giving error while running the netstat command, however if i directly run the netstat ...
1
vote
2answers
104 views

Find and replace a string if particular pattern is found in a line

All I need to do is, search for CCSID in this file, wherever it finds CCSID, the CHAR in that line should be replaced with NCHAR and VARCHAR in that line should be replaced with NVARCHAR2. I tried ...
2
votes
2answers
58 views

Exchanging One Column From File1 with files from File2

I have two files. The first file has 11 columns, for example: 1 2 3 4 5 6 7 8 9 10 11 The second has 10 columns and might look like this: 11 22 33 44 55 66 77 88 99 100 What I want to do is look ...
0
votes
1answer
106 views

How to delete the lines that starts with a specific word and end with a specific word

Eg: CREATE TABLE COMM.ACL_ENTRY ( ACL_OID CHAR(18) FOR BIT DATA NOT NULL , USER_DESC_TYPE FOR COLUMN USER_00001 BIGINT NOT NULL , USER_DESC_OID FOR COLUMN USER_00002 ...
1
vote
1answer
44 views

Create new text file with sum of third column of several others

I have several different text files of the format: x y z x y z x y z I want to create a new file which sums the z values in each, and then has the summed value in place of z i.e. x y ...
0
votes
1answer
56 views

Need help in writing awk to extract cpu utilization from sar command

I am trying to extract the values of the %user,%nice etc from the output of the sar command. Command : sar -P ALL 1 1 Output of this : Linux 2.6.32-358.el6.x86_64 (ftizsldapp009.ftiz.cummins.com) ...
0
votes
1answer
69 views

sed html parsing

I have to parse HTML to change text between two brackets that are not HTML code back to < or >. Here an example of my HTML code that I have to replace: ...
0
votes
1answer
57 views

Check if two lines start with the same character, if so the output average, if not, print actual value [closed]

I'd like to check if two rows start with the same number in the 1st column, if that happens, then the average of the 2nd column should be displayed. File example: 01 21 6 10% 93.3333% 01 ...
0
votes
1answer
59 views

Compare first column of 2 files based on the second column using the delimiter ;

I need to compare below 2 files. The head_file: 2345;int builder( 1000;char type::edit::display <test>( 250;void unamed_string_tree* The ref_file: 2000;int builder( ...