Questions tagged [awk]

A pattern-directed scanning and processing language.

Filter by
Sorted by
Tagged with
0
votes
2answers
20 views

How do I clean up a log file to show consecutive repeated lines only once, with a number indicating how many times that line repeated?

Current logs: 18:56:54 Info: Starting 18:56:55 Error: timed out 18:56:56 Error: timed out 18:56:57 Error: timed out 18:56:58 Info: reconnected 18:56:59 Error: timed out Desired output: 18:56:54 Info: ...
0
votes
1answer
16 views

Need advice using awk to create a single line from a file with multiple lines of info

Using only awk, how would you convert the following text from this: $ cat latest.csv psga,alp1acscf0001vm001oam001p1n004v001,192.7.129.7,UVP apsga,alp1acscf0001vm001oam001p1n004v002,192.7.129.8,UVP ...
0
votes
2answers
38 views

Execute command for all rows with equal field content

I have a student csv file containing fields with student data and the mailadress of the teacher in charge at the end. I try to automatically send only one email to the teacher containing all his ...
1
vote
1answer
39 views

What is the command line to print the output below (use built-in function of toupper from awk)?

I have a text file which is a random file : price.txt orange 5 300 Florida banana 7 500 California apple 3 600 Maryland grape 14 50 New Mexico mango 30 1 Florida tomato 45 100 ...
2
votes
1answer
85 views

How can I save variables in awk?

How can I print the filename and the result of 1/2*(a+b), where a is the 1st field of the 2nd line and b is the 1st field of the last line please? I do not know how to save the values into variables ...
0
votes
3answers
51 views

syntax error in awk

#!/usr/bin/awk userinput='Hello World!' userinput=$userinput awk ' BEGIN { s = ENVIRON["userinput"] "\n" n = length(s) while (1) printf &...
-1
votes
1answer
32 views

awk in bash cycle

How to apply awk for multiple files, please? The following script does nothing. FILE=/media/linux/*.txt for f in $FILES do awk '{printf("%.4f %8.2f 1.000 1 1\n", $2, $4)}' $f &...
3
votes
2answers
541 views

Create Numbers 1-6 in random order using AWK

I'm using AWK to generate values between 1 - 6 which need to come out in random order. I have managed to sort out the logic for the creation of the right range of numbers but am struggling with ...
3
votes
2answers
172 views

AWK CUDA version command syntax error when starts from Makefile

I have some issue I just want to catch exact CUDA version using nvidia-smi from command line and it is working in shell: $ nvidia-smi | awk -F"CUDA Version:" 'NR==3{split($2,a," ")...
0
votes
5answers
104 views

Update delimiter of specific columns

I have a comma-separated file with 7 fields. One of these fields, however, contains , in the value. I want to change the delimiter of the file to | without changing the data of the field with the ,. ...
0
votes
1answer
39 views

Ubuntu command - awk - list files with distinct prefix

I have a directory on an Ubuntu machine with a large number of files, and subgroups of files starting with the same prefix. I need the list of distinct prefixes present in the name of the directory ...
-4
votes
2answers
123 views

random output from input

Related question : how to create a random output from users input I was thinking to take an input(only one argument) from user just like Hello. And, I want to give output like olleho llohe he llo lo ...
0
votes
2answers
45 views

Extract only rows with duplicated strings in tab-delimited table

I have a long list of data with 10 tab delimited columns. First two columns are the IDs. I would like to retrieve rows of selected IDs. I started with renaming the selected IDs, so that each of them ...
0
votes
1answer
31 views

How to Add and subtract date in existing file?

I am having file with date format YYYYMM. I need to do two operation addition and subtraction. for Addition in existing input file i need to do add(+14).( For example if i am having date 201610 means ...
4
votes
1answer
279 views

About `NR==/regex/` matching a pattern

When I run an awk command that has syntax NR== /regex/, it gives unexpected results. The contents of file: cat file ## This is ## a ## b ## c ## end ## e ## f ## g Now I add an empty line at start ...
2
votes
2answers
48 views

Add every nth line together

**edited for clarity as per recommendation I'm looking at averaging a number of data points given from multiple runs of a command. (in this case using the sysbench tool suite) Example input (this ...
-1
votes
2answers
44 views

Make List with Rows that contain “1” value in particular columns

I need to make 2 files with a list of rows containing "1" in certain columns- one file for column 4 "1"s and one for column 5 "1"s. The original file looks like this: ID ...
0
votes
4answers
103 views

How to skip every three lines using awk [duplicate]

Could anyone help me on arranging the files? I want to skip every three lines based on the example below using awk. Current file: a a a a b b b b c c c c Desired output: a b c
0
votes
4answers
79 views

How to extract fields for specific line numbers that are passed as parameters to a bash script?

I have a file named data.txt with the following content: 1 aFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf IT524234 2 bFDLKSFD_FDSJFskadfsff_fsadklfj_fdsaf IT524234 3 ...
-1
votes
1answer
27 views

I would like to format my app log in unix

I have a log file like this: [loggerid:1234-4567] api:api1() 5.91 [loggerid:7899-45671234] api:api12345() 11.00 [loggerid:3333-45671234564] api:api12345789() 11.05 And I need output like this: First ...
-1
votes
0answers
39 views

How do I access a particular record in the END block of AWK/

I am trying to access a particular record, and a particular field in that record. But I am in the END block of my code, so the FNR is fixed to my last line. Is there a word around?
0
votes
0answers
21 views

Replace a multi-line C# variable with another with sed or awk

I have a C# source code containing an embedded JSON string as a multi-line variable: source.cs: class Json { public static string MultiLineVariable = @" ""aaa"": "&...
0
votes
2answers
51 views

Match a pattern and replace the value of the following key value pair

I'm trying to modify a file that contains sections like this: [ policy_strict ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName =...
-1
votes
1answer
33 views

syntax error when using the END special pattern in AWK

I am a beginner to shell scripting and I am trying to print out the maximum value of an array from a file that I am reading. The code works fine when I don't use the END statement but gives out an ...
2
votes
3answers
104 views

Adding and subtracting multiple columns into new column

I have copied a partial csv file. publish_date,headline_text,likes_count,comments_count,shares_count,love_count,wow_count,haha_count,sad_count,thankful_count,angry_count 20030219,aba decides against ...
0
votes
1answer
66 views

awk: line 1: syntax error at or near {

I have been trying to get a maximum and a minimum value in the text file that have a numerical table. In order to do this, I opened up each column in the file using awk function for loop. But I got ...
-1
votes
1answer
37 views

Filter the data to generate the outermost points only

I have the following data, when plotted gives me the following I would like to save and plot only the outermost points with a line plot. Pls do suugest on how to proceed with this and advise on how ...
0
votes
3answers
56 views

create a summary of object sizes in one comma separated list

I have a large file containing two fields the first one representing an object name and the second field represents the size of that object: A 1 A 2 B 4 ABC 12 C 5 A 9 B 3 ABC 6 I would like to ...
0
votes
3answers
73 views

Compare row count of a unix file with trailer count for multiple types of record

My file has multiple headers and multiple record types (eg. 0001, 0002, 0003, 0004). Count is given for each record type in the trailer row along with overall detail record count. Sample File: XYZH001 ...
0
votes
3answers
65 views

print the number of column and “ERROR” if it has an empty field else “CORRECT” if it has no empty fields

I have a csv that looks as follows: column1, column2, column3, column4, column5, 1,,,,5, 1,2,3,,, 1,2,3,4,5 1,2,3,4,5, 1,2,,,5, Using awk I want to print the number of all the rows and a comment &...
0
votes
1answer
48 views

awk: add two columns to csv that are the multiplication of a constant by the last column

I am trying to add two new columns to a csv file, that contain the multiplication of the last column to a constant. This last column is formed by the pound symbol and followed by a number. Input: id,...
4
votes
3answers
258 views

How to do calculation for each row

I had a csv file of data such like that when read into shell: name,income,reward,payment Jackson,10000,2000,1000 Paul,2500,700,200 Louis,5000,100,1800 and I want to find the net earning for each ...
3
votes
6answers
326 views

Bash - remove dashes and new-lines before replacing new-lines with spaces

I have a bunch of text files in the following format: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam ...
0
votes
2answers
93 views

The awk boundary is the beginning to a pattern

How to have awk to set range or boundary of the beginning to a pattern tried 1 fails as $ echo -e 'a\nb\nc\nend\ne\nf\ng\n' |awk '1,/end/ {print $0}' a b c end e f g this one for all as pattern works:...
2
votes
3answers
99 views

awk: split datetime column into three separate columns in a csv

I am trying to use substr to split a datetime column, the fifth one (previous_test) into three different ones at the end. Input: id,tester,company,chief,previous_test,test,date,result,cost 6582983b-...
0
votes
2answers
52 views

Remove a text from the nth line in a file

I want to remove a string which is coming from a variable in a line. remove_text=' hi'; File: kjdkjdkjddkjd djdk hi sjjsj jfjf sisj hi jdkjdk dkjdkdjd hi hi sksk I want to to delete hi from Nthline ( ...
0
votes
2answers
47 views

appending a text to the end of nth line without creating a newline

I have a file A.txt and when I try to append a text to end of Nth line, it's printing by creating a new line, is there any command which will print in the same line without creating a newline? A.txt ...
-1
votes
2answers
54 views

Compare two csv files containing floating-point data and print their difference statistics

I have two csv files: a.csv and b.csv. Both these files contain same set of rows. For example: a.csv: Yield 5.68 1.34 1.47 9.01 7.59 b.csv: Yield 5.68 2.01 0.78 7.98 8.17 I need to compare these two ...
0
votes
3answers
39 views

Combine multiple awk functions

I'm very new to awk, I'm trying to do the following in an environment that doesn't support grep -P and doesn't have Perl installed either. Given I have a list of env variables starting with a prefix, ...
0
votes
0answers
37 views

BASH: splitting status phrase of the code execution using echo -n

As a part of my bash routine I am printing some message in terminal regarding status of the workflow. The message in splited into two parts (part 1: begining of the task, part 2: status of its ...
0
votes
1answer
30 views

How to change a specific field in multiple files using awk

I am very new to script language. I have multiple output files with the names in a similar pattern as: output1 output2 etc. In each output file, the records follow the pattern as: "295534 4750 4 -...
0
votes
1answer
59 views

How to convert a CSV with multiple sub-row items (from a column) to new columns in Awk?

I have a CSV file with the general format shown in the picture below. In that CSV there are multiple rows that belong to a certain column (desc) and I would like to to extract those items and add them ...
0
votes
1answer
61 views

awk/sed: append counters to multiple different strings

I have a text file, and would like to append a counter to a number of different strings of interest. An example infile: string_of_interest abcd efgh another_string_of_interest ijkl abcd ...
2
votes
1answer
33 views

Rename a JSON key after a specific pattern

I have a JSON object with the following content: { "quiz": { "sport": { "id": "77", "q1": { "...
-1
votes
2answers
50 views

Apache logs: how many different IPs per day with sed or awk?

Given a file /var/log/apache2/other_vhosts_access.log like this: example.com:443 1.1.1.1 - - [25/Jan/2021:12:00:00 +0000] "GET /abc/def/ghi?token=jklm12 HTTP/1.1" 200 1000 "-" &...
-1
votes
2answers
67 views

Convert row to columns

I have a input like this block of line are delimited by empty line, need to convert row to columns Input HEAD1 IF FI GH HEAD2 PU GT HEAD3 FG DF YT GU Need to print like this: HEAD1 IF FI GH HEAD2 ...
0
votes
1answer
84 views

Compute and print summary with cumulative amounts by state and gender from csv file

How would I go about something like this? Purchase Summary Report State Gender Purchase Amount TX F 33734.33 CA F 23911.61 TX M 23043.64 FL M 18846.49 I need to sum the total transactions (2 decimal ...
3
votes
2answers
141 views

Collapsing a series of comma separated numbers in a sequence to Beginning-End vs n,n,n,n,x,x,x,x, [duplicate]

Problem I am trying to solve/enhance a BASH script that provides a number sequence: I am using a topologically aware tool (lstopo-no-graphics) to extract physical processor numbers to use for input to ...
0
votes
3answers
72 views

How would I format filtered data to new file?

Hello all I'm trying to read data from file, and check for each state and increment total transactions counts from data: REDACTED The end result should be printed to a new file in this format This ...
0
votes
1answer
52 views

How to check for matching data and accumulate if match is true?

Hello I currently have this data listed as CustomerID, FirstName, LastName, gender, amount spent REDACTED How would I loop through this datafile in awk, and check for matching customerids $1 then if ...

1
2 3 4 5
143