Questions tagged [awk]
A pattern-directed scanning and processing language.
7,113
questions
0
votes
2answers
20 views
How do I clean up a log file to show consecutive repeated lines only once, with a number indicating how many times that line repeated?
Current logs:
18:56:54 Info: Starting
18:56:55 Error: timed out
18:56:56 Error: timed out
18:56:57 Error: timed out
18:56:58 Info: reconnected
18:56:59 Error: timed out
Desired output:
18:56:54 Info: ...
0
votes
1answer
16 views
Need advice using awk to create a single line from a file with multiple lines of info
Using only awk, how would you convert the following text from this:
$ cat latest.csv
psga,alp1acscf0001vm001oam001p1n004v001,192.7.129.7,UVP
apsga,alp1acscf0001vm001oam001p1n004v002,192.7.129.8,UVP
...
0
votes
2answers
38 views
Execute command for all rows with equal field content
I have a student csv file containing fields with student data and the mailadress of the teacher in charge at the end. I try to automatically send only one email to the teacher containing all his ...
1
vote
1answer
39 views
What is the command line to print the output below (use built-in function of toupper from awk)?
I have a text file which is a random file :
price.txt
orange 5 300 Florida
banana 7 500 California
apple 3 600 Maryland
grape 14 50 New Mexico
mango 30 1 Florida
tomato 45 100 ...
2
votes
1answer
85 views
How can I save variables in awk?
How can I print the filename and the result of 1/2*(a+b), where a is the 1st field of the 2nd line and b is the 1st field of the last line please? I do not know how to save the values into variables ...
0
votes
3answers
51 views
syntax error in awk
#!/usr/bin/awk
userinput='Hello World!'
userinput=$userinput awk '
BEGIN {
s = ENVIRON["userinput"] "\n"
n = length(s)
while (1)
printf &...
-1
votes
1answer
32 views
awk in bash cycle
How to apply awk for multiple files, please? The following script does nothing.
FILE=/media/linux/*.txt
for f in $FILES
do
awk '{printf("%.4f %8.2f 1.000 1 1\n", $2, $4)}' $f &...
3
votes
2answers
541 views
Create Numbers 1-6 in random order using AWK
I'm using AWK to generate values between 1 - 6 which need to come out in random order. I have managed to sort out the logic for the creation of the right range of numbers but am struggling with ...
3
votes
2answers
172 views
AWK CUDA version command syntax error when starts from Makefile
I have some issue I just want to catch exact CUDA version using nvidia-smi from command line and it is working in shell:
$ nvidia-smi | awk -F"CUDA Version:" 'NR==3{split($2,a," ")...
0
votes
5answers
104 views
Update delimiter of specific columns
I have a comma-separated file with 7 fields. One of these fields, however, contains , in the value. I want to change the delimiter of the file to | without changing the data of the field with the ,.
...
0
votes
1answer
39 views
Ubuntu command - awk - list files with distinct prefix
I have a directory on an Ubuntu machine with a large number of files, and subgroups of files starting with the same prefix. I need the list of distinct prefixes present in the name of the directory ...
-4
votes
2answers
123 views
random output from input
Related question : how to create a random output from users input
I was thinking to take an input(only one argument) from user just like Hello. And, I want to give output like
olleho
llohe
he
llo
lo
...
0
votes
2answers
45 views
Extract only rows with duplicated strings in tab-delimited table
I have a long list of data with 10 tab delimited columns. First two columns are the IDs. I would like to retrieve rows of selected IDs.
I started with renaming the selected IDs, so that each of them ...
0
votes
1answer
31 views
How to Add and subtract date in existing file?
I am having file with date format YYYYMM. I need to do two operation addition and subtraction.
for Addition in existing input file i need to do add(+14).( For example if i am having date 201610 means ...
4
votes
1answer
279 views
About `NR==/regex/` matching a pattern
When I run an awk command that has syntax NR== /regex/, it gives
unexpected results. The contents of file:
cat file
## This is
## a
## b
## c
## end
## e
## f
## g
Now I add an empty line at start ...
2
votes
2answers
48 views
Add every nth line together
**edited for clarity as per recommendation
I'm looking at averaging a number of data points given from multiple runs of a command. (in this case using the sysbench tool suite)
Example input (this ...
-1
votes
2answers
44 views
Make List with Rows that contain “1” value in particular columns
I need to make 2 files with a list of rows containing "1" in certain columns- one file for column 4 "1"s and one for column 5 "1"s. The original file looks like this:
ID ...
0
votes
4answers
103 views
How to skip every three lines using awk [duplicate]
Could anyone help me on arranging the files? I want to skip every three lines based on the example below using awk.
Current file:
a
a
a
a
b
b
b
b
c
c
c
c
Desired output:
a
b
c
0
votes
4answers
79 views
How to extract fields for specific line numbers that are passed as parameters to a bash script?
I have a file named data.txt with the following content:
1 aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf IT524234
2 bFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf IT524234
3 ...
-1
votes
1answer
27 views
I would like to format my app log in unix
I have a log file like this:
[loggerid:1234-4567] api:api1() 5.91
[loggerid:7899-45671234] api:api12345() 11.00
[loggerid:3333-45671234564] api:api12345789() 11.05
And I need output like this:
First ...
-1
votes
0answers
39 views
How do I access a particular record in the END block of AWK/
I am trying to access a particular record, and a particular field in that record. But I am in the END block of my code, so the FNR is fixed to my last line. Is there a word around?
0
votes
0answers
21 views
Replace a multi-line C# variable with another with sed or awk
I have a C# source code containing an embedded JSON string as a multi-line variable:
source.cs:
class Json {
public static string MultiLineVariable = @"
""aaa"": "&...
0
votes
2answers
51 views
Match a pattern and replace the value of the following key value pair
I'm trying to modify a file that contains sections like this:
[ policy_strict ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName =...
-1
votes
1answer
33 views
syntax error when using the END special pattern in AWK
I am a beginner to shell scripting and I am trying to print out the maximum value of an array from a file that I am reading. The code works fine when I don't use the END statement but gives out an ...
2
votes
3answers
104 views
Adding and subtracting multiple columns into new column
I have copied a partial csv file.
publish_date,headline_text,likes_count,comments_count,shares_count,love_count,wow_count,haha_count,sad_count,thankful_count,angry_count
20030219,aba decides against ...
0
votes
1answer
66 views
awk: line 1: syntax error at or near {
I have been trying to get a maximum and a minimum value in the text file that have a numerical table. In order to do this, I opened up each column in the file using awk function for loop. But I got ...
-1
votes
1answer
37 views
Filter the data to generate the outermost points only
I have the following data, when plotted gives me the following
I would like to save and plot only the outermost points with a line plot. Pls do suugest on how to proceed with this and advise on how ...
0
votes
3answers
56 views
create a summary of object sizes in one comma separated list
I have a large file containing two fields the first one representing an object name and the second field represents the size of that object:
A 1
A 2
B 4
ABC 12
C 5
A 9
B 3
ABC 6
I would like to ...
0
votes
3answers
73 views
Compare row count of a unix file with trailer count for multiple types of record
My file has multiple headers and multiple record types (eg. 0001, 0002, 0003, 0004). Count is given for each record type in the trailer row along with overall detail record count.
Sample File:
XYZH001
...
0
votes
3answers
65 views
print the number of column and “ERROR” if it has an empty field else “CORRECT” if it has no empty fields
I have a csv that looks as follows:
column1, column2, column3, column4, column5,
1,,,,5,
1,2,3,,,
1,2,3,4,5
1,2,3,4,5,
1,2,,,5,
Using awk I want to print the number of all the rows and a comment &...
0
votes
1answer
48 views
awk: add two columns to csv that are the multiplication of a constant by the last column
I am trying to add two new columns to a csv file, that contain the multiplication of the last column to a constant. This last column is formed by the pound symbol and followed by a number.
Input:
id,...
4
votes
3answers
258 views
How to do calculation for each row
I had a csv file of data such like that when read into shell:
name,income,reward,payment
Jackson,10000,2000,1000
Paul,2500,700,200
Louis,5000,100,1800
and I want to find the net earning for each ...
3
votes
6answers
326 views
Bash - remove dashes and new-lines before replacing new-lines with spaces
I have a bunch of text files in the following format:
Lorem ipsum dolor sit amet,
consetetur sadipscing elitr,
sed diam nonumy eirmod tempor
invidunt ut labore et dolore
magna aliquyam erat, sed diam
...
0
votes
2answers
93 views
The awk boundary is the beginning to a pattern
How to have awk to set range or boundary of the beginning to a pattern
tried 1 fails as
$ echo -e 'a\nb\nc\nend\ne\nf\ng\n' |awk '1,/end/ {print $0}'
a
b
c
end
e
f
g
this one for all as pattern works:...
2
votes
3answers
99 views
awk: split datetime column into three separate columns in a csv
I am trying to use substr to split a datetime column, the fifth one (previous_test) into three different ones at the end.
Input:
id,tester,company,chief,previous_test,test,date,result,cost
6582983b-...
0
votes
2answers
52 views
Remove a text from the nth line in a file
I want to remove a string which is coming from a variable in a line.
remove_text=' hi';
File:
kjdkjdkjddkjd
djdk hi sjjsj
jfjf sisj hi
jdkjdk dkjdkdjd hi
hi sksk
I want to to delete hi from Nthline ( ...
0
votes
2answers
47 views
appending a text to the end of nth line without creating a newline
I have a file A.txt and when I try to append a text to end of Nth line, it's printing by creating a new line, is there any command which will print in the same line without creating a newline?
A.txt
...
-1
votes
2answers
54 views
Compare two csv files containing floating-point data and print their difference statistics
I have two csv files: a.csv and b.csv. Both these files contain same set of rows. For example:
a.csv:
Yield
5.68
1.34
1.47
9.01
7.59
b.csv:
Yield
5.68
2.01
0.78
7.98
8.17
I need to compare these two ...
0
votes
3answers
39 views
Combine multiple awk functions
I'm very new to awk, I'm trying to do the following in an environment that doesn't support grep -P and doesn't have Perl installed either.
Given I have a list of env variables starting with a prefix, ...
0
votes
0answers
37 views
BASH: splitting status phrase of the code execution using echo -n
As a part of my bash routine I am printing some message in terminal regarding status of the workflow. The message in splited into two parts (part 1: begining of the task, part 2: status of its ...
0
votes
1answer
30 views
How to change a specific field in multiple files using awk
I am very new to script language. I have multiple output files with the names in a similar pattern as: output1 output2 etc.
In each output file, the records follow the pattern as:
"295534
4750
4 -...
0
votes
1answer
59 views
How to convert a CSV with multiple sub-row items (from a column) to new columns in Awk?
I have a CSV file with the general format shown in the picture below.
In that CSV there are multiple rows that belong to a certain column (desc) and I would like to to extract those items and add them ...
0
votes
1answer
61 views
awk/sed: append counters to multiple different strings
I have a text file, and would like to append a counter to a number of different strings of interest. An example infile:
string_of_interest
abcd
efgh
another_string_of_interest
ijkl
abcd
...
2
votes
1answer
33 views
Rename a JSON key after a specific pattern
I have a JSON object with the following content:
{
"quiz": {
"sport": {
"id": "77",
"q1": {
"...
-1
votes
2answers
50 views
Apache logs: how many different IPs per day with sed or awk?
Given a file /var/log/apache2/other_vhosts_access.log like this:
example.com:443 1.1.1.1 - - [25/Jan/2021:12:00:00 +0000] "GET /abc/def/ghi?token=jklm12 HTTP/1.1" 200 1000 "-" &...
-1
votes
2answers
67 views
Convert row to columns
I have a input like this block of line are delimited by empty line, need to convert row to columns
Input
HEAD1
IF
FI
GH
HEAD2
PU
GT
HEAD3
FG
DF
YT
GU
Need to print like this:
HEAD1 IF FI GH
HEAD2 ...
0
votes
1answer
84 views
Compute and print summary with cumulative amounts by state and gender from csv file
How would I go about something like this?
Purchase Summary Report
State Gender Purchase Amount
TX F 33734.33
CA F 23911.61
TX M 23043.64
FL M 18846.49
I need to sum the total transactions (2 decimal ...
3
votes
2answers
141 views
Collapsing a series of comma separated numbers in a sequence to Beginning-End vs n,n,n,n,x,x,x,x, [duplicate]
Problem I am trying to solve/enhance a BASH script that provides a number sequence:
I am using a topologically aware tool (lstopo-no-graphics) to extract physical processor numbers to use for input to ...
0
votes
3answers
72 views
How would I format filtered data to new file?
Hello all I'm trying to read data from file, and check for each state and increment total transactions counts from data:
REDACTED
The end result should be printed to a new file in this format
This ...
0
votes
1answer
52 views
How to check for matching data and accumulate if match is true?
Hello I currently have this data listed as
CustomerID, FirstName, LastName, gender, amount spent
REDACTED
How would I loop through this datafile in awk, and check for matching customerids $1 then if ...
