awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

-1
votes
0answers
8 views

Uniqe values in a table [duplicate]

I have asked this question before but later on I realized that the answer provided by @Anthon is not working properly for me How can I find unique values in two a column table? Let me again define ...
0
votes
0answers
16 views

to check fields in configuration file

i have configuration file in which first field is filename, second field is the column if we find the value of that column is null in the filename that is given in first column. if any of value of ...
1
vote
3answers
36 views

How to List files which contain white space in filename? [duplicate]

I am trying to read files one by one by using the for loop like below, by passing argument to the functions. Argument 3 makes lot of problem due to space in file names. Example: ...
0
votes
1answer
32 views

Using ARGV to get user input in Awk script [on hold]

I know ARGV[i] can be used for storing user input. However, I want to use it in awk script and get the ARGV[i] to compare the field from another text file. If the ARGV[i] matches the field or the ...
0
votes
2answers
23 views

Adding entries to the end of a row via awk

I was wondering if anyone would be able to point out what is going wrong here. I have a txt file of data, delimited by a space, of 7 columns by 10000 rows. I want to add two extra columns one with the ...
1
vote
3answers
55 views

How to align a table of display nicely

I am trying to create a script that act as a ps command with it's own properties that I want to display. Say this is how processes would look like in ps command: sas 24431 1 0 Oct10 ? ...
2
votes
1answer
34 views

How to replace the first line using sed?

Hi i am trying to replace the header line from my file using the sed command as mentioned below, sed "1s/.*/$new_header/" Test_file.csv The above line replaces the header and prints in stdout , ...
1
vote
4answers
103 views

Sed/awk/perl: reversing order of comma separated values, preserving other text

I have this text: LABEL1 .BYTE 01, 02, 03, 04, 05 .BYTE 01, 02, 03 I need to reverse only comma separated values order: LABEL1 .BYTE 05, 04, 03, 02, 01 .BYTE 03, 02, 01 I need ...
2
votes
2answers
67 views

How can I find unique values in two a column table?

I have a space or comma separated table with two columns. A B B C B D C E F G What I want is a table with unique values like following A B C D E F G This means that if any value of ...
2
votes
2answers
43 views

Environment variable not expanded inside the command line argument

I have a file user-pid.out2 which has "usernumber" and "process id" as two columns. based on usernumber I want to find corresponding process id. the first two lines below does not show the output ...
0
votes
2answers
14 views

Replicate lines in text fine according to a column value

I have a text file with a column containing text/data followed by an abundance column, e.g. Car 3 Apple 2 In order to subsample these columns I would like to replicate each text column by ...
0
votes
1answer
48 views

How to update specific row and column using awk? [on hold]

I have the index file as below balsum|3|8|13|24|07:29:29|1111989 smaval|6|11|16|16|20:21:19|2565 paycan|3|8|13|3149|07:24:36|2930 format: alrt_type|index1|index2|index3|value1|value2|value3 ...
1
vote
1answer
33 views

count matches and mismatches group by

Please help with a shell script of the following. I need count number of consistent variables in each lane (col1) across samples (col2). For example, since all the values (col4) of lane1 variable 1 ...
2
votes
2answers
35 views

Sed/awk/perl: modify text preserving parts and align to a column

I have a text like this: A1JOURNEY0TO1 .BYTE 00, 00, 00 A2JOURNEY0TO2 .BYTE 00, 01, 00 A3JOURNEY1TO0 .BYTE 00, 01, 01 I need to have: JOURNEY_01 ; 00 TO ...
1
vote
1answer
14 views

redirect output of awk gsub command to file

I found an answer on how to remove commas between quotes of a comma delimited csv file over here on stackexchange (cannot comment there as I don't have the needed rep). awk -F'"' -v OFS='' '{ for ...
0
votes
2answers
20 views

When interpret awk as a command or a programming language?

I know that awk is a script programming language, but sometimes I get confused about when I interpret as command or as a program. Eg. 1 - Here I interpret as a command: awk '{print $2}' file.txt ...
0
votes
1answer
78 views

Extract values from filename and add it into a text file

I have a gridded model data (separated by different depth) saved in text files. The structure is like this: Format of every column in a row in the text file: x_coordinate y_coordinate density ...
0
votes
3answers
43 views

Filtering a .CSV file based on consecutive numbers in a line

I have a CSV file as follows: 1st 2nd 3rd 4th ID ... Res Res Res Res (other columns) ... RZ_AUTO_1, 1cx0, C118, B, C119, B, A165, ...
2
votes
2answers
49 views

Compare 2 delimited files and output differences

I have asked similar questions here a couple times in the past with great success, but now my needs have slightly changed and I am struggling to get the exact output I am looking for. I would like to ...
6
votes
5answers
148 views

Using awk to identify the number identical columns

I am new to all this, and this is probably a rather simple question, but I am stuck: I have a large number of individual files that contain six columns each (number of rows can vary). As a simple ...
1
vote
1answer
74 views

Summing by common strings in different files

I have a file file1 with the amount of times a user shows up in the files, something like this: 4 userC 2 userA 1 userB and I have another file file2 with users and other info like: userC, degree2 ...
0
votes
1answer
55 views

Cut a specific part of a column from a csv file [closed]

I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
1
vote
1answer
65 views

splitting up a MySQL dump using awk

I have a large-ish (250G) MySQL dump file from which I only want / need one database. I was hoping to use awk to split the file up so I did some searching and found this expression: awk ...
2
votes
4answers
71 views

replace several comma out side one or more sets of curly braces and exception that in one or more sets of curly braces

In the text file I have multi records. Every record has multi column with comma separated, some column have one sets of curly brace and other have more than one curly brace. I need: If a comma is ...
4
votes
2answers
191 views

Remove entire row in a file if first column is repeated

I have a file containing two columns and 10 million rows. The first column contains many repeated values, but there is a distinct value in column 2. I want to remove the repeated rows and want to keep ...
1
vote
2answers
63 views

Merging 2 files with based on field match

I want to create a file that contains columns from two input files. File1 is like: aa 32 bb 15 cc 78 File2 is: fa 19 bc 23 cc 50 de 28 aa 45 bb 31 The task is, read through File1, if the 1st field ...
2
votes
1answer
74 views

Sed to Modfiy Hosts File

I have a text file, modifyhostslist.txt, which contains entries that correspond to entries found in my hosts file. Not every entry in my hosts file needs to be modified, only entries also found in ...
1
vote
1answer
43 views

Sum values in the other columns for the 1st column with the same ID

The input txt file looks like (there are more columns in the actual txt file): target_id length eff_length tot_counts uniq_counts est_counts eff_counts mthl7 61 0 0 0 0 0 loqs 72 ...
1
vote
3answers
128 views

keep only certain part of a string in a certain column

I have a file like this: id target_id length eff_length 1 intron_FBgn0000721:20_FBgn0000721:18 1136 243.944268 1 intron_FBgn0000721:19_FBgn0000721:18 1122 ...
1
vote
1answer
57 views

Replace pattern in file with pattern in second file

I want to replace part of a file's data, with data from another file. Suppose file1 has data as is written below and file2 has some data I want to store and replace file1 data from file2 from ...
12
votes
2answers
503 views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
4
votes
2answers
142 views

Comma separated words to next line in new file

I have a input file like this (input.txt): Main1 one,two,three,four Main2 five,six,seven,eight I want the output like this in output file (output.txt): Main1 one Main1 two Main1 three Main1 four ...
1
vote
2answers
55 views

How to match a number in one file to a name in another file?

Okay, I hope this is the right place to ask. I have 2 files. The first file has 4 fields: class CRN #, Instructor ID #, Class Section #, Number of Students. Example: 0002 T00005 006 15 0004 ...
2
votes
4answers
60 views

how to print a new line in awk

I am trying to print collected data from a file where each set gets a new line (\n) Code: awk '/mail:|fullName:|uid:/{s=s", "$0} END{print substr(s,3)}' dump2.txt Output now: mail: ...
1
vote
2answers
78 views

grepping for multiple items

I have about 6 items I need extracted from and LDAP base dump file fullName: uid: email: ... the colon is required for the search, mainly because uid appears in a number of places in the file but ...
1
vote
3answers
115 views

sed + find value before word in line

How can I find the value before the word "free"? top -n1 | grep Mem Mem: 2054968k av, 2034120k used, 20848k free, 0k shrd, 186768k buff First I tried top -n1 | grep ...
-2
votes
1answer
48 views

Extract/print every 3rd occurence of pattern

I need help with extracting/printing lines until a stop pattern. This needs to happen after every 3rd occurrence of start pattern, i.e ERROR_FILE_NOT_FOUND and continue until the end of the file. ...
-2
votes
1answer
585 views

convert CSV to XLS file on linux [closed]

The following Perl script can convert CSV file to XLS file the problem is that I need to install on customer Linux machine many Perl modules in order to run this Perl script , and actually I cant ...
0
votes
3answers
104 views

How does the following shell script work in finding all files larger than 20MB?

I came across a post advising that the following will find all files greater than 20Mb in size. find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | ...
0
votes
5answers
43 views

df -k + how to match the root volume line

please advise how to match the "/" volume from df -k command? I tried the following: [root@machine /]# df -k | grep "/" /dev/vx/dsk/bootdg/rootvol 130926685 71378710 58238709 56% / /devices ...
1
vote
2answers
40 views

Extracting file name and string from multiple files

I have a lot of files (>100,000) from which I would like to extract a specific string and the filename itself. Files are organized like this: Filename: sequence.php?ID=gi|90022703|ref|YP_528530.1| ...
0
votes
1answer
47 views

Finding words with one vowel group

I am trying to use a command to find all words in a list of words with one group vowels. Like four, for and far, but also words like ore and or. What I got so far: grep -i ...
1
vote
3answers
46 views

Find a string in one section of a file with mutiple sections

How to sed/awk for a string contained in another string. TESTVAR=' { icon : "icons/773_l.png", id : 80, initialState : true, isPng : false, label : "Imagery", opacity : 1, requestType : ...
2
votes
4answers
142 views

Remove all comments from a file?

I have a file with a few lines commented out: foo bar stuff #Do not show this... morestuff evenmorestuff#Or this I just want to print all the uncommented code: foo bar stuff morestuff ...
1
vote
0answers
91 views

extracting data from a log file and plotting it using Gnuplot

I have a logfile with the following data in it: Sat Sep 20 11:25:01 BST 2014 -- temp=53.0'C -- message from script /usr/src/scripts/wifi_test_2.sh Sat Sep 20 11:26:01 BST 2014 -- temp=53.0'C -- ...
1
vote
4answers
91 views

splitting a column using awk

I have a file like shown below. The 9th column has values joined by **. chrXV 234346 234546 snR81 + SNR81 chrXV 234357 0.0003015891774815342**0.131826816475 + chrXV 234346 ...
1
vote
2answers
97 views

Extract text from one iteration of top

I want to execute the top command from my bash script the problem is that usually when I type the top command on the console , I exit by Ctrl+C. but how to exit from top when I run the top command ...
-1
votes
1answer
33 views

search for a character in one column and replace value in another column of matched records

I have a requirement to find value 2 in column 4 and wherever it matches replace 6th column with '-'sign in the beginning suppose if i have csv file - file1 like this abc,0,def,1234,ijk ...
2
votes
1answer
47 views

Getting matched fasta file

list.txt 58759__len__2903 58759__len__2903 673957__len__1655 673957__len__1655 3566454__len__1744 seq.fasta >58759__len__2903 TTTTCCGTAGAGGAGATCCCTATTTTTAGGTTTGTAAGAGATCATTTT ...
-1
votes
1answer
57 views

Match and combine records in Bash [duplicate]

Consider two csv files: $ cat given.csv 123,John 246,Paul 369,George 987,Ringo $ cat family.csv 246,McCartney 123,Lennon 987,Starr 369,Harrison Note that the lines are not ordered! How might I use ...