awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
1answer
35 views

how to move a header to the last column, using awk or sed

how to move a header to the last column, using awk or sed input file look like this: Line 1.000N x y z 23.88 44.66 56.6 23.81 41.66 53.6 Line 81.000N x y z 13.88 34.66 56.6 13.81 ...
8
votes
6answers
480 views

How to append Line to previous Line?

I have a Log file which need to be parsed and analysed. File contains something similar like below: File: 20141101 server contain dump 20141101 server contain nothing {uekdmsam ikdas jwdjamc ...
1
vote
2answers
113 views

how to compare values in two columns in two different files, echoing full lines where absolute value of difference is < a small maximum value?

I want to do a line-by-line comparison of values in, say, column 6 of two files, and then echo the lines and their full contents, piping them into a combined file, say the line from one file and then ...
2
votes
1answer
85 views

Specify lines in awk?

Im going to use this awk script to delete duplicate lines from my text file: awk '!x[$0]++' However, I want to specify which lines that they are to be run on. There could be 10000 duplicates, but ...
0
votes
2answers
78 views

delete multiple users

I am the root user and I am setting up a menu for another user to use. This other user will only get this menu. There are two options that are interlinked: the first option is to search users. The ...
2
votes
2answers
39 views

Create small record files from a big group of record file

I have a file like this : Mainfile: Record|1111|ABC text in between for record 1 text in between for record 1 Record|2222|XYZ text in between for record 2 Record|3333|XYZ ...
0
votes
1answer
53 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
2
votes
7answers
624 views

How to print lines with blank 5th field in CSV

I have to check if the fifth field is empty in a CSV file. This is my file: 1,abc,543,87,DATA,fsg; 1,abc,543,87,,fsg; 1,abc,543,87,DATA,fsg; 1,abc,543,88,,fsg; 1,abc,543,,DATA,fsg; As you can ...
4
votes
2answers
217 views

grep for repeated values from pattern list

I have file A.txt which is a lists of unique ids (column 1) and its corresponding values(column 2): A.txt ABC1D_T1_B1 123.4 ABC2D_T1_B2 146.7 ABC3D_T1_B1 567.8 ABC4D_T0_B2 96.8 ABC1D_T0_B1 ...
1
vote
1answer
36 views

awk how to set parameter $2 become $b?

In a Bash script: #!/usr/bin/bash #grep() #{ #grep -q #} if [[ -z $1 ]]; then echo "usage: pkill -signal process name " fi if [[ -z $2 ]]; then echo "error: not enough parameters " exit fi ...
2
votes
2answers
51 views

Combine output from df -h /dev/xvda1 into a single line

I want to convert the 2 line output from df -h into a single, meaningful line i.e. Take [root@ip-10-42-1-246 tmp]# df -h /dev/xvda1 Filesystem Size Used Avail Use% Mounted on /dev/xvda1 ...
0
votes
4answers
65 views

Print last character in the given path

I am trying to concat "comnp.dat_CY*" files into one file. File name, source path and destination path are the parameters. I am trying to find the last character in the path to check if it is "/" or ...
4
votes
2answers
2k views

Remove entire row in a file if first column is repeated

I have a file containing two columns and 10 million rows. The first column contains many repeated values, but there is a distinct value in column 2. I want to remove the repeated rows and want to keep ...
-6
votes
3answers
203 views

How do you print the current month, day and year using awk and date?

I can't seem to grab the month, day and year using awk '{print $2, $3, $6}' date Does anyone know how to?
3
votes
1answer
38 views

create a realtime machine by assigning CPU cores to critical processes

I'm using an Udoo board for running two critical processes. It can run Ubuntu ARM edition and it has a Quadcore processor. I need to set the affinity of all the processes afinity running and the ones ...
-1
votes
2answers
92 views

Can sed and awk take in multiple files?

grep can search in multiple files, including those files recursively under a specified directory. This makes grep able to find which file contains a pattern. Can awk and sed also? It seems that they ...
2
votes
2answers
159 views

How to base awk selection on contents of 1 field?

How can I select all the records that have '2' as the second field? My data is: $ cat numbers.txt 1 2 3 4 5 6 7 8 2 4 6 8 10 12 14 16 3 6 9 12 15 18 21 24 My awk is: awk '$2 - /^2$/ {print}' ...
3
votes
3answers
513 views

awk + print lines from the first line until match word

I want to print all lines from file until the match word please advice how to do that with awk for example I want to print all lines until word PPP remark the first line chuld be diff from AAA ( ...
3
votes
2answers
92 views

Read a two-character column as two separate columns

I have a file with genotype data. The second column has both alleles for a particular genetic variant concatenated, as below. rs969931 CA 1.000 2.000 2.000 2.000 2.000 2.000 1.000 ...
0
votes
0answers
40 views

Extracting trace information using gawk

I am new to awk programming and am using gawk to extract trace information from a mrt file in order to use it further for analysis. I have successfully been able to extract trace information from a ...
0
votes
2answers
50 views

Substituting \ for \\ in column

I exported a column of thousands of salts from Mysql for hashcat. Every instance of the character \ was replaced with a double character \\ in the exported column. Is there a way to either sed or awk ...
0
votes
4answers
80 views

sed remove last space only on end of line,combined with awk

I have those line Siouxsie Sioux Siouxsie Sioux ... Sylvia Tobias Sylvia Tobias ... Sydney Urshan Sydney Urshan ... Club Drummer (uncredited) Club Drummer (uncredited) ... ...
0
votes
1answer
52 views

Issue finding lines using grep in UNIX [closed]

I am trying to use a .txt file with around 5000 patterns (spaced with a line) to search through another file of 18000 lines for any matches. So far I've tried every form of grep and awk I can find on ...
6
votes
7answers
621 views

Command line tool to “cat” pairwise expansion of all rows in a file

Suppose I have a file (call it sample.txt) that looks like this: Row1,10 Row2,20 Row3,30 Row4,40 I want to be able to work on a stream from this file that is essentially the pairwise combination of ...
1
vote
2answers
82 views

Compare and print data from different text files to one in Shell

How can I compare and print data from different text files to one in Shell. I have captures NAS details of three different boxes using SSH, now I need to combine all the three text files to one file ...
1
vote
3answers
232 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
0
votes
2answers
65 views

Read file and find all occurances and generate hash for the content in between the occurance

I have a file like this : 1 Record|1111|ABC 2 text in between for record 1 3 text in between for record 1 4 Record|2222|XYZ 5 text in between for record 2 6 Record|3333|XYZ 7 text ...
3
votes
2answers
255 views

Why does $var give the value of $0?

I have written a very simple awk script that reverses every word of a file-: awk ' { for (i=1;i<=NF;i++) { x="" for(j=length($i);j>0;j--) x=x substr($0,j,1); print x }}' file1 The contents of ...
0
votes
0answers
39 views

awk - how does the for loop work when counting words? [duplicate]

I have got the following awk script from here that counts the frequency of the words in a file-: # Print list of word frequencies { for (i = 1; i <= NF; i++) freq[$i]++ } END { ...
3
votes
1answer
85 views

How do I use awk to create an index of words in file?

I have a problem with my homework I need to create a shell script using awk. The script will output only the words in the file in an alphabetical order, and after the word is : after that a space then ...
1
vote
3answers
106 views

Merging multiple rows into a single one using awk

I'm working with a very large data set in the format: chr1 45162 . . C T chr1 45162 . . C T chr1 45162 A . . T chr1 45162 . . C T chr1 45257 A . . T chr1 45257 A . . T chr1 45413 A . . T chr1 46844 ...
3
votes
3answers
216 views

Split in awk not not printing array values

I have a string tstArr2 which has the following content '3 5 8' Now in awk I want to parse a flat file test my array which array is better array INDIA USA SA NZ AUS ARG GER BRA US AUS INDIA ENG ...
1
vote
3answers
50 views

making 4 out of 1 file with awk

i have large list - text file with 1 column of numbers (length=14). Numbers have 4 difrent prefixes 00001-00004. how do i split 1 list to 4 diffrend files. 1 file shoud contain list of numbers with ...
1
vote
2answers
69 views

Save AWK FNR as a Bash variable to continue search from the line

I'm trying to get to run this on a sorted file. This returns the last line which is less than or equal to the given value comparing the first 9 characters. This would be done multiple times and VALUE ...
3
votes
5answers
228 views

removing extensions in a column

I have a file like this ILMN_1343291 TGTGTTGAGAGCTTCTCAGACTATCCACCTTTGGGTCGCTTTGCTGTTCG NM_001402.5 ILMN_1343295 CTTCAACAGCGACACCCACTCCTCCACCTTTGACGCTGGGGCTGGCATTG NM_002046.3 ILMN_1651209 ...
1
vote
1answer
452 views

Match first fields of two tab separated files and print matching values

I have two files. Match the 1st column of first file to 1st column of second file and print entire line of the second file. Second file is a <tab> separated file. I have tried many awk one ...
0
votes
3answers
133 views

How can I convert timestamps in a column to a date?

I have a file containing this: 1415602803,LOGIN SUCCESS,AUTH,user2,192.168.203.63,10.146.124.73,59996,22 1415602807,LOGIN SUCCESS,AUTH,user1,172.24.31.10,172.32.1.1,48191,22 1415602811,LOGIN ...
1
vote
0answers
84 views

Unable to run awk script [closed]

I'm running a tcl script which is going to perform an network performance analysis(running it in NS2) I'm unable to run the awk script and it is throwing me an error "improper conversion" if {$argc ...
3
votes
2answers
354 views

Delete every Nth line in shell

I'm trying to delete a line after N lines using awk and I can't seem to get it right. The file format is like this YYYYYY XXXXXX XXXXXX YYYYYY XXXXXX XXXXXX The real example would be office3 3 1 ...
2
votes
2answers
10k views

Use awk to split line into array and use that array's values in calling shell

I am trying to use awk inside a bash script and do a rather common task: iterate well structured file lines and split them by a delimiter into an array. Here is a sample from the file: ...
1
vote
2answers
30 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
1
vote
1answer
76 views

How to print odd numbered fields from a text file using awk in Unix? [closed]

I have posted the same question before but as I an just a beginner in Unix. I am unable to understand the answer given to it though certainly that helps to fulfil my requirement. awk -F\| '{s="";for ...
3
votes
3answers
2k views

Extracting part of lines with specific pattern using awk,sed

I have a question regarding the awk/sed operators. I have a big file which has following set of lines repeated Expression loweWallrhoPhi : sum=-6.97168e-09 Expression leftWallrhoPhi : ...
1
vote
2answers
79 views

How to add a column from a file to the last column of another file using awk?

I want to add the second column of file2 to file1, that contains a number an unknown number of columns, so becoming the last column of file1. In this aim, I tried to create a file3 composed by pasting ...
1
vote
4answers
322 views

How to subtract two times in shell

I nee to subtract two lines which is in the format of time in shell. Th time format is hh:mm:ss I used the code below to get the time. cat /var/log/kern.log |grep usb |tail -2| awk '{print $3}' The ...
1
vote
2answers
532 views

awk - Group by and sum column values

I have command to list system process by memory usage: ps -A --sort -rss -o comm,pmem Which list a table like COMMAND %MEM firefox 28.2 chrome 5.4 compiz 4.8 ...
0
votes
0answers
34 views

jumbled character set

In my Linux env, for some unknown! reason when I process a raw unformatted file which I copied from Win7prof (was encoded in utf-16), the resulting file is all jumbled. It will print OK in Linux ...
1
vote
2answers
148 views

Search 2nd line of files and print filenames to stdout

I have a few directories of ~10,000 files. What's the quickest way to search each file and return the filename if the second line contains a specific string? Edited for clarity
2
votes
6answers
1k views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
13
votes
8answers
9k views

How to print the longest line in a file?

I'm looking for the simplest method to print the longest line in a file. I did some googling and surprisingly couldn't seem to find an answer. I frequently print the length of the longest line in a ...