awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
2answers
166 views

awk - Group by and sum column values

I have command to list system process by memory usage: ps -A --sort -rss -o comm,pmem Which list a table like COMMAND %MEM firefox 28.2 chrome 5.4 compiz 4.8 ...
1
vote
1answer
43 views

How to print odd numbered fields from a text file using awk in Unix? [closed]

I have posted the same question before but as I an just a beginner in Unix. I am unable to understand the answer given to it though certainly that helps to fulfil my requirement. awk -F\| '{s="";for ...
0
votes
0answers
25 views

jumbled character set

In my Linux env, for some unknown! reason when I process a raw unformatted file which I copied from Win7prof (was encoded in utf-16), the resulting file is all jumbled. It will print OK in Linux ...
1
vote
2answers
114 views

Search 2nd line of files and print filenames to stdout

I have a few directories of ~10,000 files. What's the quickest way to search each file and return the filename if the second line contains a specific string? Edited for clarity
1
vote
4answers
172 views

How to subtract two times in shell

I nee to subtract two lines which is in the format of time in shell. Th time format is hh:mm:ss I used the code below to get the time. cat /var/log/kern.log |grep usb |tail -2| awk '{print $3}' The ...
3
votes
3answers
336 views

How can I delete all text between curly brackets in a multiline text file?

Example: This is { the multiline text file } that wants { to be changed } anyway. Should become: This is that wants anyway. I have found some similar threads in the forum, but they don't seem ...
2
votes
1answer
55 views

Calculate median value for a column and say if this value is “OK” or “ERROR” if falls within acceptable range

I would like to calculate the median value of a particular column in a .txt file, output that value and also say if that output value is ok. I can do the initial steps and output the median value, ...
2
votes
2answers
63 views

Parse several thousand lines of txt into lines and columns

The file at hand lists names, credentials, roles, and privileges for thousands of users. It was passed onto me as .xls but the lines and columns are not arranged in proper form. I have reformated ...
0
votes
1answer
42 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
0
votes
1answer
38 views

Need to compare the two files and print all the columns then missing display as NA, then join another file [duplicate]

File 1: 123, 234 345, 789 678, 900 File 2 123, abc 345, dnc output (all columns in file1 and missing value in file two as NA): 123,234,abc 345,789,dnc 678,900,NA Output file add with another ...
1
vote
3answers
120 views

using awk command to compare the two files and print all the columns then missing display as NA

File 1: 123, 234 345, 789 678, 900 File 2 123, abc 345, dnc Desired output (all columns in file1 and missing value in file two as NA): 123,234,abc 345,789,dnc 678,900,NA
2
votes
3answers
48 views

Removing lines from one file based on non-existence in another

I need to remove lines from fileA which has two columns if fileB (one column only) does not contain the string from column1 in fileA. Any linux command would help. Thanks. Example. Here is fileA: a ...
0
votes
1answer
16 views

add fields from different lines in one line

I'm trying to add the .data and .bss sections of the size -A output size -A mylib.so | grep "\.data" | awk '{print $2} this will print the value of the size of the .data section. I guess I could ...
8
votes
4answers
367 views

Remove duplicate lines from a file that contains a timestamp

This question/answer has some good solutions for deleting identical lines in a file, but won't work in my case since the otherwise duplicate lines have a timestamp. Is it possible to tell awk to ...
2
votes
2answers
108 views

How to capture the port number from this `lsof -F` output using awk (or something better)?

I've got output coming from lsof -F that looks like this: p7646 g7646 R8300 csocat u1000 Labe f3 au l tIPv4 G0x80002;0x0 d640391 o0t0 PTCP n*:51352 TST=LISTEN TQR=0 TQS=0 I am trying to capture the ...
0
votes
2answers
51 views

Delete consecutive lines if patterns match in each line

Good day, I was wondering how to delete to consecutive lines when this pattern is found: /^pt [a-z]\npy [0-9][0-9][0-9][0-9]\n// Expected Input pt a py 01234 pt b cd abc py 5678 Expected Output ...
0
votes
2answers
58 views

Grouping data and assigning alphabet and numeric values for new categories

I have 4 columns in my tab delimited data, I need to perform 2 steps. Group the data according to combinations of first 2 columns. Every time a new combination of values appear in the concatenated ...
1
vote
2answers
58 views

Print only the second field of the second line

I have to extract from the command hcitool dev only the mac address of the BT dongle. Output of hcitool dev is Devices: hci0 xx:xx:xx:xx:xx:xx I write this output on a file and try to get the ...
0
votes
5answers
62 views

Subset data with awk

I want to subset data with awk. Lets say I have this file called test: IP MAC Bandwidth etc etc 192.1.1.1 ff:ff:ff:ff 5.421M 192.1.2.3 ff:ff:ff:f3 5.120M 192.1.2.5 ff:ff:ff:f1 5.100M stuff I dont ...
1
vote
1answer
58 views

Multiply rows of a column in different chunk of data, add them together and get the average?

I have a chunk of data that looks like this: 924 777 777 777 8445.65349 P 9.44657 -0.840283 -0.394467 C 4.0555 -0.683104 -1.70917 P 7.5169 3.54172 2.77681 C 3.21536 2.67146 2.20242 P 4.33641 7.62012 ...
0
votes
2answers
63 views

How to list only filenames removing directories and subdirectories

How can I get the list of only filenames and remove directory and subdirectories using find, sed, awk, cut or any. For e.g : I get list of all media files using find as such: find /var/warehouse/* ...
2
votes
4answers
62 views

Awk remove field only if numeric

ls /var/log/packages/*SBo|cut -d / -f 5 return webcore-fonts-3.0-noarch-3_SBo winetricks-20120923-noarch-1_SBo wireshark-1.12.1-x86_64-1_SBo wxGTK-2.8.12-x86_64-3_SBo I want to substitute from ...
4
votes
4answers
353 views

How do I remove every number that's surrounded by <>

I have tried this sed -i '' 's/[0-9]*<>/g' But it didn't work. Example file: <Number1> </Number8> output: <Number> </Number>
4
votes
2answers
373 views

How do I populate end tags using sed, awk or any other command?

I have a txt file that looks like this: <sss>ss< or <firstword>anotherword< I want it to look like this <sss>ss</sss> or ...
0
votes
1answer
88 views

How to update files based on another file

I have 4 csv files and I want to update set of three based on values in the other one. The first 2 files (file_1 and file_2) contains names and their numerical ids. The third file (file_3) contains ...
1
vote
2answers
60 views

Extract and format data with `cut` and `awk`

I have the below file in txt: "Firstname","Lastname","Company" "Alex","Johnson","Alex Corp." "Baran","Atasoy","Gülerler Ltd." I would like to extract and format data by using awk and/or cut and ...
1
vote
1answer
39 views

Insert missing string in multiple ordered columns

I have a multi-column file that looks like this: 6 93129B 177 306.98 350.20 8.84 0.23 -0.79 0.22 0.46 m 7 8 192 269.08 413.35 9.29 0.15 -0.81 ...
2
votes
3answers
285 views

Using grep/awk/sed to sort and combine 2 files

Ok I am running into a problem. I am taking a wifi log that has MAC addresses listed in it. I want to find out what MAC vendors/manufactors for the devices that are connected to my router. I have 2 ...
2
votes
2answers
64 views

How to wrap the comma delimited values to the next line without disturbing the other fields using awk or ksh

I have the following input, in which some of the values are comma delimited. I would like decouple each value and print it in the new line. I tried many ways in awk, but no luck. If I have three ...
3
votes
2answers
218 views

Shell script to assign values from a lookup table is too slow

I'm trying to assign some variables from a lookup file with a shell script. I have something working but it seems unnecessary slow. Script: while read line do ...
0
votes
1answer
54 views

How to copy lines in awk command? (ksh scripting is preferred)

I have the below index file key1|1|1001 key1|1|2001 key2|2|3001 key2|2|4001 using this index file, I have to update my main file key1|1000|2000|3000|4000 key2|1000|2000|3000|4000 The ...
0
votes
1answer
29 views

Using find to search and report files that have one but not two keywords inside

I'd like to use find to report files under the following conditions: If the file contains the word "Serializable" Then report the file if it does NOT contain the word "serialVersionUID" Something ...
1
vote
1answer
50 views

Linux log file structure change for usage with kst

My file (tcp dump from networked sensors) has this shape: 20:06:57.049686 IP (tos 0x0, ttl 64, id 26871, offset 0, flags [DF], proto UDP (17), length 68) e108-193.eduroam.tugraz.ac.at.34225 > ...
3
votes
2answers
43 views

Help with iterating fieldA in CSV file, where fieldB has specified value

I've been trying to setup a script to run on gitbash, to process a CSV file so each record that has a specific value, the last field is changed from empty ("") to a value that iterates from 1 to 16. ...
1
vote
4answers
218 views

How to convert fixed length fields in a file to space delimited

I have a fixed length matrix type file where each field is 20 characters long. How can I convert this to a space delimited file? Also there are some missing values, which I would like to ...
0
votes
3answers
59 views

How to print loop index to a file using awk?

I'm a new in bash scripting. I need a help in how to print the loop index to a file using awk? Here is the code I used. echo "Write the start and end file number:" read sta end echo "$sta" "$end" for ...
2
votes
2answers
60 views

How to exclude lines that has given columns?

There is an input file that has TAB delimited columns. We need to remove the lines which has "NA" for the fourth AND the eleventh column. Question: how can we do this in awk?
0
votes
1answer
51 views

Perl/awk/bash: multiply and sum values putting values inline instead original text [closed]

I have a text like this: ;--------------------------------------- JOURNEY_ARRAY_01 .BYTE <JOURNEY_00 .BYTE <JOURNEY_01 ;--------------------------------------- JOURNEY_ARRAY_03 ...
3
votes
3answers
958 views

Count records matching pattern with Awk

So I was told to do last > lastloggedin which creates a file that shows the classes last login since the last system reboot, and now I am asked to write an Awk script which is named myawk that ...
1
vote
2answers
85 views

Print a line in stdout that matches an expression if the output contains another expression

This might be a common/easy task but I couldn't figure out from examples on the web or manuals of awk/sed/grep. So, here is the scenario: There is an internal command line tool that prints out a ...
0
votes
2answers
57 views

Why awk does not show the correct record length?

I'm concerned regarding what awk shows as the record length. I'm checking some files for a specific record length - awk shows the result I wanted, but the file size shows that each record in the file ...
0
votes
0answers
25 views

to check fields in configuration file

i have configuration file in which first field is filename, second field is the column if we find the value of that column is null in the filename that is given in first column. if any of value of ...
1
vote
3answers
94 views

How to List files which contain white space in filename? [duplicate]

I am trying to read files one by one by using the for loop like below, by passing argument to the functions. Argument 3 makes lot of problem due to space in file names. Example: ...
0
votes
1answer
70 views

Using ARGV to get user input in Awk script [closed]

I know ARGV[i] can be used for storing user input. However, I want to use it in awk script and get the ARGV[i] to compare the field from another text file. If the ARGV[i] matches the field or the ...
1
vote
2answers
37 views

Adding entries to the end of a row via awk

I was wondering if anyone would be able to point out what is going wrong here. I have a txt file of data, delimited by a space, of 7 columns by 10000 rows. I want to add two extra columns one with the ...
1
vote
3answers
102 views

How to align a table of display nicely

I am trying to create a script that act as a ps command with it's own properties that I want to display. Say this is how processes would look like in ps command: sas 24431 1 0 Oct10 ? ...
2
votes
1answer
58 views

How to replace the first line using sed?

Hi i am trying to replace the header line from my file using the sed command as mentioned below, sed "1s/.*/$new_header/" Test_file.csv The above line replaces the header and prints in stdout , ...
1
vote
4answers
174 views

Sed/awk/perl: reversing order of comma separated values, preserving other text

I have this text: LABEL1 .BYTE 01, 02, 03, 04, 05 .BYTE 01, 02, 03 I need to reverse only comma separated values order: LABEL1 .BYTE 05, 04, 03, 02, 01 .BYTE 03, 02, 01 I need ...
3
votes
2answers
111 views

How can I partition a set of words given pairs that must end up together?

I have a space or comma separated table with two columns, each row representing equivalence of the two words. A B B C B D C E F G What I want is a table with each row listing all mutually ...
2
votes
2answers
89 views

Environment variable not expanded inside the command line argument

I have a file user-pid.out2 which has "usernumber" and "process id" as two columns. based on usernumber I want to find corresponding process id. the first two lines below does not show the output ...