awk - pattern-directed scanning and processing language

learn more… | top users | synonyms

1
vote
7answers
171 views

repeat a line, splitting one field

I have a tab delimited file in which in every line I have this: K00001;K00004;K00008 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 36 0 0 52 0 0 0 6 0 I would ...
2
votes
5answers
13k views

awk-Printing column value without new line and adding comma

input.txt EN1 EN2 EN3 EN4 EN5 output EN1,EN2,EN3,EN4,EN5 I have tried awk.But it is not printing with comma awk 'BEGIN { OFS = ","} { printf $1}' input.txt I have GNU Awk 4.0.0 version
-1
votes
2answers
46 views

Extract 2 fields from 2 different lines

I have the below lines in my file # Employee Id : ABC123, # Employee Name: Vivek # Employee Type: Temp Join Date: 31-jan-2014 Buz Code: 2098 # Employee Comments: # ...
1
vote
1answer
40 views

how to move a header to the last column, using awk or sed

how to move a header to the last column, using awk or sed input file look like this: Line 1.000N x y z 23.88 44.66 56.6 23.81 41.66 53.6 Line 81.000N x y z 13.88 34.66 56.6 13.81 ...
8
votes
6answers
619 views

How to append Line to previous Line?

I have a Log file which need to be parsed and analysed. File contains something similar like below: File: 20141101 server contain dump 20141101 server contain nothing {uekdmsam ikdas jwdjamc ...
1
vote
2answers
185 views

how to compare values in two columns in two different files, echoing full lines where absolute value of difference is < a small maximum value?

I want to do a line-by-line comparison of values in, say, column 6 of two files, and then echo the lines and their full contents, piping them into a combined file, say the line from one file and then ...
2
votes
1answer
86 views

Specify lines in awk?

Im going to use this awk script to delete duplicate lines from my text file: awk '!x[$0]++' However, I want to specify which lines that they are to be run on. There could be 10000 duplicates, but ...
0
votes
2answers
147 views

delete multiple users

I am the root user and I am setting up a menu for another user to use. This other user will only get this menu. There are two options that are interlinked: the first option is to search users. The ...
2
votes
2answers
42 views

Create small record files from a big group of record file

I have a file like this : Mainfile: Record|1111|ABC text in between for record 1 text in between for record 1 Record|2222|XYZ text in between for record 2 Record|3333|XYZ ...
0
votes
1answer
72 views

Replacing code between two JavaScript comments

I need to set a value of JavaScript variable from bash. Variable lives in index.html file and I'd like to use unix SED command to do that. Inside index.html I have it lake this: <script> ...
2
votes
7answers
2k views

How to print lines with blank 5th field in CSV

I have to check if the fifth field is empty in a CSV file. This is my file: 1,abc,543,87,DATA,fsg; 1,abc,543,87,,fsg; 1,abc,543,87,DATA,fsg; 1,abc,543,88,,fsg; 1,abc,543,,DATA,fsg; As you can ...
4
votes
2answers
289 views

grep for repeated values from pattern list

I have file A.txt which is a lists of unique ids (column 1) and its corresponding values(column 2): A.txt ABC1D_T1_B1 123.4 ABC2D_T1_B2 146.7 ABC3D_T1_B1 567.8 ABC4D_T0_B2 96.8 ABC1D_T0_B1 ...
1
vote
1answer
49 views

awk how to set parameter $2 become $b?

In a Bash script: #!/usr/bin/bash #grep() #{ #grep -q #} if [[ -z $1 ]]; then echo "usage: pkill -signal process name " fi if [[ -z $2 ]]; then echo "error: not enough parameters " exit fi ...
2
votes
2answers
62 views

Combine output from df -h /dev/xvda1 into a single line

I want to convert the 2 line output from df -h into a single, meaningful line i.e. Take [root@ip-10-42-1-246 tmp]# df -h /dev/xvda1 Filesystem Size Used Avail Use% Mounted on /dev/xvda1 ...
0
votes
4answers
83 views

Print last character in the given path

I am trying to concat "comnp.dat_CY*" files into one file. File name, source path and destination path are the parameters. I am trying to find the last character in the path to check if it is "/" or ...
-5
votes
3answers
583 views

How do you print the current month, day and year using awk and date?

I can't seem to grab the month, day and year using awk '{print $2, $3, $6}' date Does anyone know how to?
3
votes
1answer
48 views

create a realtime machine by assigning CPU cores to critical processes

I'm using an Udoo board for running two critical processes. It can run Ubuntu ARM edition and it has a Quadcore processor. I need to set the affinity of all the processes afinity running and the ones ...
-1
votes
2answers
142 views

Can sed and awk take in multiple files?

grep can search in multiple files, including those files recursively under a specified directory. This makes grep able to find which file contains a pattern. Can awk and sed also? It seems that they ...
2
votes
2answers
176 views

How to base awk selection on contents of 1 field?

How can I select all the records that have '2' as the second field? My data is: $ cat numbers.txt 1 2 3 4 5 6 7 8 2 4 6 8 10 12 14 16 3 6 9 12 15 18 21 24 My awk is: awk '$2 - /^2$/ {print}' ...
3
votes
2answers
138 views

Read a two-character column as two separate columns

I have a file with genotype data. The second column has both alleles for a particular genetic variant concatenated, as below. rs969931 CA 1.000 2.000 2.000 2.000 2.000 2.000 1.000 ...
0
votes
2answers
58 views

Substituting \ for \\ in column

I exported a column of thousands of salts from Mysql for hashcat. Every instance of the character \ was replaced with a double character \\ in the exported column. Is there a way to either sed or awk ...
0
votes
4answers
91 views

sed remove last space only on end of line,combined with awk

I have those line Siouxsie Sioux Siouxsie Sioux ... Sylvia Tobias Sylvia Tobias ... Sydney Urshan Sydney Urshan ... Club Drummer (uncredited) Club Drummer (uncredited) ... ...
0
votes
1answer
67 views

Issue finding lines using grep in UNIX [closed]

I am trying to use a .txt file with around 5000 patterns (spaced with a line) to search through another file of 18000 lines for any matches. So far I've tried every form of grep and awk I can find on ...
6
votes
7answers
713 views

Command line tool to “cat” pairwise expansion of all rows in a file

Suppose I have a file (call it sample.txt) that looks like this: Row1,10 Row2,20 Row3,30 Row4,40 I want to be able to work on a stream from this file that is essentially the pairwise combination of ...
1
vote
2answers
91 views

Compare and print data from different text files to one in Shell

How can I compare and print data from different text files to one in Shell. I have captures NAS details of three different boxes using SSH, now I need to combine all the three text files to one file ...
1
vote
3answers
282 views

sort CSV by number of column in rows?

I have multiple CSV files with many records. The total number of rows should be 134. I have many files, and each rows has its own number of columns (from 15 to 200). I need to sort them according to ...
0
votes
2answers
98 views

Read file and find all occurances and generate hash for the content in between the occurance

I have a file like this : 1 Record|1111|ABC 2 text in between for record 1 3 text in between for record 1 4 Record|2222|XYZ 5 text in between for record 2 6 Record|3333|XYZ 7 text ...
3
votes
2answers
329 views

Why does $var give the value of $0?

I have written a very simple awk script that reverses every word of a file-: awk ' { for (i=1;i<=NF;i++) { x="" for(j=length($i);j>0;j--) x=x substr($0,j,1); print x }}' file1 The contents of ...
0
votes
0answers
40 views

awk - how does the for loop work when counting words? [duplicate]

I have got the following awk script from here that counts the frequency of the words in a file-: # Print list of word frequencies { for (i = 1; i <= NF; i++) freq[$i]++ } END { ...
3
votes
1answer
94 views

How do I use awk to create an index of words in file?

I have a problem with my homework I need to create a shell script using awk. The script will output only the words in the file in an alphabetical order, and after the word is : after that a space then ...
1
vote
3answers
129 views

Merging multiple rows into a single one using awk

I'm working with a very large data set in the format: chr1 45162 . . C T chr1 45162 . . C T chr1 45162 A . . T chr1 45162 . . C T chr1 45257 A . . T chr1 45257 A . . T chr1 45413 A . . T chr1 46844 ...
3
votes
3answers
283 views

Split in awk not not printing array values

I have a string tstArr2 which has the following content '3 5 8' Now in awk I want to parse a flat file test my array which array is better array INDIA USA SA NZ AUS ARG GER BRA US AUS INDIA ENG ...
1
vote
3answers
51 views

making 4 out of 1 file with awk

i have large list - text file with 1 column of numbers (length=14). Numbers have 4 difrent prefixes 00001-00004. how do i split 1 list to 4 diffrend files. 1 file shoud contain list of numbers with ...
1
vote
2answers
81 views

Save AWK FNR as a Bash variable to continue search from the line

I'm trying to get to run this on a sorted file. This returns the last line which is less than or equal to the given value comparing the first 9 characters. This would be done multiple times and VALUE ...
3
votes
5answers
270 views

removing extensions in a column

I have a file like this ILMN_1343291 TGTGTTGAGAGCTTCTCAGACTATCCACCTTTGGGTCGCTTTGCTGTTCG NM_001402.5 ILMN_1343295 CTTCAACAGCGACACCCACTCCTCCACCTTTGACGCTGGGGCTGGCATTG NM_002046.3 ILMN_1651209 ...
0
votes
3answers
372 views

How can I convert timestamps in a column to a date?

I have a file containing this: 1415602803,LOGIN SUCCESS,AUTH,user2,192.168.203.63,10.146.124.73,59996,22 1415602807,LOGIN SUCCESS,AUTH,user1,172.24.31.10,172.32.1.1,48191,22 1415602811,LOGIN ...
3
votes
2answers
596 views

Delete every Nth line in shell

I'm trying to delete a line after N lines using awk and I can't seem to get it right. The file format is like this YYYYYY XXXXXX XXXXXX YYYYYY XXXXXX XXXXXX The real example would be office3 3 1 ...
2
votes
2answers
13k views

Use awk to split line into array and use that array's values in calling shell

I am trying to use awk inside a bash script and do a rather common task: iterate well structured file lines and split them by a delimiter into an array. Here is a sample from the file: ...
1
vote
1answer
42 views

Comparing value in same column and directing output to new column

I have my input file as DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 0940 29343414 11 110005 DPortal=ECCN RemoteFile=4004_130122 0256 A02 141111 2336 29843714 11 110006 DPortal=ECCN ...
1
vote
1answer
107 views

How to print odd numbered fields from a text file using awk in Unix? [closed]

I have posted the same question before but as I an just a beginner in Unix. I am unable to understand the answer given to it though certainly that helps to fulfil my requirement. awk -F\| '{s="";for ...
3
votes
3answers
4k views

Extracting part of lines with specific pattern using awk,sed

I have a question regarding the awk/sed operators. I have a big file which has following set of lines repeated Expression loweWallrhoPhi : sum=-6.97168e-09 Expression leftWallrhoPhi : ...
1
vote
2answers
191 views

How to add a column from a file to the last column of another file using awk?

I want to add the second column of file2 to file1, that contains a number an unknown number of columns, so becoming the last column of file1. In this aim, I tried to create a file3 composed by pasting ...
2
votes
4answers
566 views

How to subtract two times in shell

I nee to subtract two lines which is in the format of time in shell. Th time format is hh:mm:ss I used the code below to get the time. cat /var/log/kern.log |grep usb |tail -2| awk '{print $3}' The ...
1
vote
2answers
1k views

awk - Group by and sum column values

I have command to list system process by memory usage: ps -A --sort -rss -o comm,pmem Which list a table like COMMAND %MEM firefox 28.2 chrome 5.4 compiz 4.8 ...
0
votes
0answers
43 views

jumbled character set

In my Linux env, for some unknown! reason when I process a raw unformatted file which I copied from Win7prof (was encoded in utf-16), the resulting file is all jumbled. It will print OK in Linux ...
1
vote
2answers
175 views

Search 2nd line of files and print filenames to stdout

I have a few directories of ~10,000 files. What's the quickest way to search each file and return the filename if the second line contains a specific string? Edited for clarity
2
votes
6answers
2k views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
13
votes
8answers
11k views

How to print the longest line in a file?

I'm looking for the simplest method to print the longest line in a file. I did some googling and surprisingly couldn't seem to find an answer. I frequently print the length of the longest line in a ...
4
votes
2answers
406 views

How do I populate end tags using sed, awk or any other command?

I have a txt file that looks like this: <sss>ss< or <firstword>anotherword< I want it to look like this <sss>ss</sss> or ...
4
votes
3answers
905 views

How can I delete all text between curly brackets in a multiline text file?

Example: This is { the multiline text file } that wants { to be changed } anyway. Should become: This is that wants anyway. I have found some similar threads in the forum, but they don't seem ...