Questions tagged [awk]
A pattern-directed scanning and processing language.
0
votes
0answers
23 views
Linux awk grep from file 1 into file 2
I have regex in file 1
.*8912.*.*.*.*81415444.*
.*8912.*.*.*.*81415444.*
.*8912.*.*.*.*81415444.*
.*8912.*.*.*.*81415444.*
.*8912.*.*.*.*81415444.*
.*8912.*.*.*.*81415444.*
.*8912.*.*.*.*81415444.*
.*...
0
votes
1answer
11 views
AWK - Need to display total value of field and field number
I have data below that needed to count or total the value start with specific field. Sample below the needed field to compute is from Field5.
For sample Field5 total is 42, field6 total is 107, etc..
...
0
votes
3answers
26 views
AWK - sum plus and minus values into separate variables
I have a CSV file like this:
2019.04.15;3.75;
2019.04.29;-5.17;
2019.05.01;7.5;
2019.05.06;0.5;
2019.05.13;0.25;
2019.05.20;-8.5;
I want to get the sum of the plus and the minus values off the ...
0
votes
1answer
40 views
Why is my bash shell script changing spaces to linefeeds?
I started out writing a simple bash shell script to find duplicate files within a given directory using SHA1 to detect matches. Everything worked until I encountered file names with spaces. The ...
0
votes
0answers
30 views
What is a great way to list files found in a directory that add up to a certain file size
This list will then be used to create tar balls for easy movement. This is what I came up with so far. Add up files that are just greater than 10GB list them to file.
find . -type f -exec ls -al {} ...
1
vote
3answers
55 views
AWK - How to count stored or index on array
I have data that need to be count how many occurrences.
1
2
1
2
3
1
And used the awk command below and work properly.
awk -F'","' '{ a[$1]++ } END { for (n in a) print n "=" a[n] } ' text.txt
And ...
2
votes
3answers
31 views
How to delete lines containing string in column n using awk?
I have a tab-separated file that looks like this:
$ cat in_file
NC_013132.1 7260299 7261429 WP_012793281.1
NC_013132.1 7270674 7270862 NC_013132.1 7270674 7270862 ID=cds5678
NC_013132.1 ...
0
votes
2answers
35 views
bash + regular expression + capture java version [on hold]
how to capture the jdk1.8.0_112 under /usr/jdk64
/usr/jdk64/jdk1.8.0_112/
while jdk1.8.0_112 is example
and could be any other version as
jdk1.7.0_112
jdk3.8.2_112
jdk1.8.0_117
jdk1.2.0_112
jdk2....
0
votes
1answer
15 views
Remote carriage return if there is only one column
A proprietary command similar to "df" returns me this :
FOO_vol6 1714236984 1685700840 28536144 98% /root_vdm_10/FOO_vol6
FOO_vol4 253851840 64818424 189033416 26%...
0
votes
1answer
21 views
To subtract a text from the last value
I have a text file like below. I want the check the type of file based on the extension and perform the action.
Example.2019.txt
Example.test.2019.txt
Example.test.2019.txt.gz
Example.test.2019.txt.Z
...
0
votes
1answer
26 views
Print numbers from table
How to get four numbers from this table please? The original input is much longer, so it is necessary to restricted the area.
Input:
chil 2 44843.084886752418 44843.080972494339 ...
0
votes
2answers
19 views
SED or AWK to remove everything before the first dash and after the last
I have a private key file with some extra nonsense in there, and want just the text of the key.
so:
nonsense -----Begin Key-----
keep this1
keep this2
keep this3
-----End Key----- nonsense
should ...
0
votes
1answer
20 views
How to sum each column and print column name and column sum using awk?
I have a file with a table of numbers; the first line is a header line.
I want to sum each column and print out sum and column name for each column using awk.
How can I do it?
2
votes
4answers
57 views
Filter YAML file content using sed/awk
I have a text file with the following content in it.
$ cat hosts.yml
[prod_env]
foo.example.com
bar.example.com
[stage_env]
foo_stage.example.com
bar_stage.example.com
[dev_env]
foo_dev1.example.com
...
1
vote
5answers
54 views
How to grep/awk/sed for a random pattern
I have a log file that I am attempting to pull a string out of. The numerical string is randomly generated and so far all of my attempts to use grep/sed/awk have been unsuccessful.
I have an entry ...
1
vote
1answer
32 views
Printing a variable in a file along with some existing columns
I have a long file with three columns. I want to add a column (as the first column) with '5' for the first 200 lines and then '6' for the next 200 lines and so on...
-1
votes
2answers
36 views
Specifying the range of the arguments to print in every record line
We have a sample data as below
EMP_ID|EMP_NAME|AGE|SALARAy|DepId|DepName
123456|XXXXXXXXX|30|10000000|1234|XYZWE
I have used below code to split the data as below
awk -F\| '
BEGIN { print "EMP_ID|...
1
vote
1answer
31 views
how to print column names as well while reading the values for that variable
I have below sample data
EMP_ID|EMP_NAME|AGE|SALARAy
123456|XXXXXXXXX|30|10000000
Is there a way I can split the record into multiple records and also get header. Example output should be like
...
0
votes
2answers
54 views
Unix command to get 0 count , when no files present for the day along with count
How to do I get the list days were no files been received. Have used the below command to get the file count along with the date:
find . -maxdepth 1 -type f -printf '%TY-%Tm-%Td\n' | awk '{array[$0]+=...
5
votes
8answers
342 views
Separate the element after every 2nd ',' and push into next row in bash [duplicate]
An example of my file looks like this:
201012,720,201011,119,201710,16
Output I want:
201012,720
201011,119
201710,16
2
votes
3answers
75 views
How to remove duplicate values based on multiple columns
I have a file tmp.log with fields like
description ID valueA valueB valueC
xxx x 1 1 1
yyy y 3 100 23
zzz z 0 0 0
aaa a 4 ...
0
votes
1answer
36 views
List lines that are the same in the second part but differ on the first part
Suppose that you have a big file with lines of this format: A, B, C, D (each line has 4 parts comma separated).
I need a list of all the lines which have common fourth part (D is the same across those ...
3
votes
1answer
33 views
Iteratively adding value to column of text file
I have a 3 column text file (XYZ coordinates) and I need to iteratively add constants to the first column while appending to the end of the original file. I have tried several options but this is the ...
1
vote
2answers
33 views
print a sequence in a file
I would like to print in a column in a new file the following
1_number_density_per_atom_1.xvg
1_number_density_per_atom_2.xvg
1_number_density_per_atom_3.xvg
.
.
.
1_number_density_per_atom_653.xvg
...
0
votes
4answers
41 views
Read the data page wise and store in seperate files
I have a huge text file where data in seperated pagewise.for example The Text is in below format i want to read lines page wise and store in seperate files
Page 1
line 1
line 2
line 3
line 4
...
0
votes
4answers
36 views
change format of column
I have in the input file a column with the following format
H6
H7
O4
C4
H8
H9
O5
C5
H10
H11
I want the output to have the following format
"a H6"
"a H7"
"a O4"
"a C4"
"a H8"
"a H9"
"a O5"
"a C5"
"a ...
0
votes
1answer
51 views
Issue while creating csv with awk
I am trying to create a csv file with AWK. I am using output file separator to add commas for columns. But there is one column whose name is separated by space and Its also getting processed in the ...
1
vote
1answer
15 views
awk to add to field based on condition
In the awk below I am trying to add adjust $2 in the ouput by adding +1 if the original value from file that was used in $2 had a - in it. Line 2 of in is an example of this. In my current awk I my ...
0
votes
1answer
32 views
How to group a string that has multiple words with awk? (Prints lines between 2 patterns)
I have an issue in which my string its not a single string; it is a string with multiple words; in which using sed works great; but not with awk;
How can I group all that string with awk? I tried to ...
1
vote
3answers
36 views
read file and based on criterion print desire output
I have input file which looks like this:
H1
O1
C1
H2
H3
O2
C2
Br1
I want to have an output file which to have the following output
H1 = 1
O1 = 8
C1 = 6
H2 = 1
H3 = 1
O2 = 8
C2 = 6
Br1 = 35
So when ...
1
vote
2answers
39 views
Update rows by referring another file
I have two comma separated files as mentioned below,
cat source.txt
A,1000
B,3000
C,2500
D,5000
E,4000
F,3000
cat test.txt
A,1000
C
D,5000
B
E
How can I update file test.txt by referring file ...
0
votes
2answers
47 views
Rearranging columns in UNIX/Linux
I have to move the unique key which is in 3rd column of each record to first column. Now based on this key each record has different total columns in it (means # of fields)
File content is
10,,FH,...
5
votes
4answers
77 views
how to insert new line if a line has more than 60 characters
I need to turn a text file that has a single line per file name and output separated by a single space into specific blocks that have lines that are equal to 60 characters in length.
Like this:
>...
5
votes
5answers
387 views
Extract the characters before last colon
I need to extract the characters before last colon : and also remove the square brackets [] in the last line.
My file structure is
256.XXX.XXX.X:20234
214.XXX.XXX.X:7249
[2200:XXXX:XXXX:XXX:XXXX:XXXX:...
2
votes
4answers
171 views
Grep value of a specific key from a String, concatenated of key : value pairs
I have a string which is concatenation of "key":"value" pairs separated by "," like-
KEY1:VALUE1, KEY2:VALUE2, KEY3:VALUE3
From this string, I have to grep for a specific string -- let's say KEY2 -- ...
1
vote
2answers
32 views
How to add a new column in between existing columns?
File1.txt:
1 Mark USA
2 Amir UK
3 Pant USA
4 Veer IND
I have to add a new column
657890
768908
435678
342343
in 3rd position. So Output must be,
Output.txt
1 Mark 657890 USA
2 Amir 768908 UK
3 ...
-1
votes
0answers
28 views
Compare 2 days and MTM on current day should not be zero when the deal value doesn't exist [closed]
There are two files: one for each day. I need to sum the MTM (column6) value based on the Deal (column2). Like adding the MTM value for the same deal values for each file.
After adding and getting ...
0
votes
0answers
25 views
unix shell scripting :compare 2 days values and on current day the value should not be zero
I have two different files in two different location, like one file is generated on today and another one is generated on previous day.Now, i want to compare the data of column B for both the files ...
0
votes
1answer
33 views
print row if number from file is between two columns
I want to match a list of numeric ids with two columns (each line has a different number/id) in file 1
>cat file1
1 23444
3 422255223
1 35541
and print only those lines from a larger file (file2)...
1
vote
1answer
66 views
Improve performance when using “system” call (shell escape) processing large files in awk
I have an awk script that processes very large files that look something like this:
K1353 SF3987.7PD833391.4 KARE
K1353 SF3987.2KD832231.4 MEAKE
K1332 IF4987.7RP832231.2 LEAOS
K1329 SF2787.7KD362619....
0
votes
2answers
43 views
How to delete string other than pattern?
I have a m3u file as below. I want to delete all strings except the pattern and next first line. or save the pattern and the next first line as another file.
Patterns group-title="TR ULUSAL" , group-...
1
vote
2answers
134 views
Rearrange columns separated by : in LINUX [closed]
The question is how to rearrange the column along with values in desired order.
input
"a":"val1","c":"val2","b":"val3","d":"val4"
"a":"val1","b":[],"c":"val3","d":"val4"
"a":"val1","d":["val2","...
0
votes
2answers
56 views
Which regular expression string ignores single comment lines from a source code file? [closed]
I'm trying to parse a plain text file or a source code file for which I need to build a regular expression to detect single line comments. I'm trying the following one without success:
re = sprintf("...
0
votes
1answer
28 views
Splitting tab-separated file depending on column value
I have a large file that I am trying to split into columns based on the values of the second column. I would like a series of output files that contains the data at the 'switching point'. The ...
0
votes
1answer
28 views
awk save value as variable
I use the command
varioutput=$(awk '{print $j}' OFS=, data/damper.test_temp1.csv)
because I want to extract the j th value of a line of many values seperated by , . But when I want to use $...
1
vote
2answers
55 views
how can I merge multiple column in one column and separated by '-'?
I have a table as below:
1 10 15
2 2 25
1 10 26
I like to merge them and make a new column in linux, like below:
1 10 15 1:10-15
2 2 25 2:2-25
1 10 26 1:10-26
0
votes
0answers
23 views
Awk save the output to the same file [duplicate]
I am using the command
awk -F, '{$12='$varioutput'; print}' OFS="," damper.test.csv
to replace the 12th column with a value. The output is correct, but it does not save the output in damper.test....
0
votes
2answers
31 views
Replace pattern in all columns except in specific column
I would like to replace "colon" from each columns except from first column as it is.
]chr9:67290502]N SUPP_VEC=011 SVLEN=0 SVTYPE=TRA 0 0 0 0/1:NA:58745598:23,0:--:.:TRA:BND00068094:NA:NA:...
4
votes
3answers
154 views
Add Columns to .csv from Multiple Files
I'm trying to add columns to a csv file from multiple files. This is what I have tried so far, but keep ending up with either an empty file or just the column from the last file.
while read i; do
...
0
votes
2answers
40 views
compare two column of two different CSV file & merge the output
TABLE1.csv
DATE, TIMESTAMP, ID, START TIME, END TIME, DURATION
2019-04-05, 13:57:19, 1607,13:06:42, 13:07:12, 00:00:30
2019-04-05, 13:58:00, 2327,13:57:26, 13:57:43, 00:00:17
2019-04-24, ...
