Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
1answer
17 views

Using grep to print all of the matching strings not separated by a space

I'm looking through a text file of emails and trying to get a list of ones from a certain domain. I'm running: grep -oh "\w*domain.com\w*" file.txt This finds the lines, but only prints that ...
4
votes
3answers
74 views

How to add a header and/or footer to a sed or awk stream?

I have a bunch of output going through sed and awk. How can I prefix the output with START and suffix the answer with END? For example, if I have All this code on all these lines and all these ...
3
votes
1answer
39 views

How to use awk to indent a source file based on simple rules?

How can I indent source code based on a couple of simple rules? As an example, I've used sed and ask to transform a selenium HTML source table to the following rspec like code. How could I ...
2
votes
1answer
24 views

Filtering and extracting PIDs as space-separated list from ps using sed

I want to filter and format the output of ps -ax -o pid,command, but not sure what pipe to use for it. I want to filter those lines matching a fixed string at a fixed position, just keep the pids for ...
0
votes
5answers
37 views

update the json file

I have a file, actually json file. How I can add a comma to the end of every but the last line ? For example: {"a":1, "b":1, "c":2} {"a":3, "b":3, "c":1} {"a":1, "b":2, "c":3} What I want: {"a":1, ...
2
votes
2answers
27 views

adding column in CSV file using awk

I need to a add a column in CSV file from an array using awk. For example, input.csv a,10 b,11 array = (100 200) output.csv should look like a,10,100 b,11,200 I tried this command but it does ...
2
votes
1answer
35 views

Question about replacing character in text file

So I have some data that is parsed in this format seperated by colons: username:email@email.com:password:salt Sometimes the username/salt can contain a colon (:) in it. I want to replace the ...
2
votes
3answers
88 views

FNR and NR variables problem

I have two files, one of them has 10 records, and the other one with 15 record, the problem is when I cat them together, always FNR=NR. For example, consider below files: File1: 1,boo 2,foo 3,boo ...
0
votes
2answers
38 views

How to search between the 2nd and 3rd delimiters

So, I have a data file and I would like a match to occur only if a match is found between the 2nd and third Vertical Bars (|) So given this data sample, I if I search for 'wilson' I want the 2nd line ...
3
votes
4answers
213 views

Conditional if clause in awk

Please consider below file: foo,1000,boo,A foo,1000,boo,B foo,1001,boo,B foo,1002,boo,D And we have below rules: If $2 equal 1000, $4 should be equal A If $2 equal 1001, $4 should be equal B If ...
0
votes
1answer
34 views

How to add new column, where the value is based on existing columns with awk

I have a file of multiple columns. I would like to make an additional column based on the values of 2 columns from this file. Example input: A B C D E F 1 2 T TACA A 3 2 3 4 I R 8 2 9 3 A C 9 3 ...
3
votes
3answers
47 views

Find a pattern and insert # at the beginning of 2 line befoe that and 1 line after that

I have a big file (more than 2000 lines). Where i have to insert a # at the beginning of 2 above lines and at the beginning of 1 below line after finding a pattern. Also insert # at the beginning of ...
1
vote
1answer
42 views

How to print the line numbers with corresponding line that matches a pattern using shell command? [duplicate]

File, .false alarm is bla no. 11 .no alarm generated is bla .application will be killed is bla wall command .doc file is document file with .doc extension .no authority for this selection bla Now ...
1
vote
2answers
32 views

How to replace string in files except with a line begins with # [duplicate]

I've looked at some other questions related to using sed to replace strings, but I'm having trouble figuring out how to apply it to my particular need. In a case like the snippet below, I want to ...
4
votes
6answers
192 views

Changing matrices into single lines of data

I have multiple matrices in a .txt file and I need each matrix to be in a single line. For example, matrices.txt: 1 2 3 4 2 3 4 5 3 4 5 6 3 4 5 6 2 3 2 5 2 3 4 5 2 3 5 6 2 3 4 5 ... What I would ...
1
vote
5answers
116 views

format file to remove " characters

I have a file with the following data "MG1507XXXXXX|" "|020000XXXXXX" "20261031|" "|3,827.92" "|3,581.41" "|542,729.62" "MBA" "MG1507XXXXXX|" "|020000XXXXXX" "20261130|" ...
0
votes
5answers
68 views

Appending One File to the Second Until End of File

I have two files. The first is just a single number. This number is calculated and changes every time. The second file has a bunch of columns, and I do not know the number of rows (which can change). ...
-1
votes
1answer
28 views

replace numbers with zero while we have the lines number [on hold]

I am trying to write a bash script (containing awk or sed ) to replace some numbers with zero. for example : replace line 3 second and third number, and line 4 first and third number and line 5 first ...
1
vote
3answers
24 views

fetch a column from a file based on another column

I have a file which contains approximately 5 million records, as follows: - 1223423,21,foo,data1,data2,data3,data4,data5,45,267,index1 4234234,34,bar,cat1,cat2,cat3,cat4,cat5,34,2323,index2 ...
0
votes
1answer
82 views

How to replace \1\ with new line

I have a record like: "evSchema"\1\"UAT" "evSN"\1\"uadb" "evDirsep"\1\"/" "evRootPath"\1\"/work_area/APP_UAT/" "evSchema"\1\"RMS13" "evUser"\1\"STAGE" I want to replace \1\ with new line. I tried ...
0
votes
1answer
33 views

Repeatedly extract every second and third line of a file with 3-line data blocks

I have a file where I have data in 3-line sets. I want the 2nd and 3rd line from each set. How do I get that? For an example from: [set no][no of line in set] 11 12 13 21 22 23 31 32 33 I want: ...
4
votes
4answers
129 views

Current date in awk

Is there a constant variable in awk, that store today's date? if no, is there a way that can store today's date for daily use? let's say we have below file: boo,foo,2016-08-30 foo,boo,2016-07-31 ...
12
votes
4answers
347 views

Sort lines by number of words per line

Given input: hello: world foo bar baz bar: baz: bin boop bop fiz bang beep bap: bim bam bop boatkeeper: poughkeepsie I would like to sort it into most words at the top, to least at the end, like ...
2
votes
3answers
61 views

remove lines that contain an specific text in a file

I have a file with a list of emails in it and each line has an email in it. I want to remove lines that contain example.com or test.com. I tried this: sed -i 's/example\.com//ig' file.txt But it ...
2
votes
2answers
63 views

Extract multiple instances of text between two words inclusive of starting word but exclusive of ending word

A PDB file contains numerous paragraphs of conformations of a protein. Each conformation starts with the keyword ATOM and ends with keyword END. I am trying to read the file in bash such that I ...
0
votes
1answer
42 views

Read all the values of a particular column of a file and store it in another file

I have a file.txt which contains data like below: col11 col12 col13 col21 col22 col23 I fetch 2nd column of all the available rows (no. of rows may vary) using awk '{print $2}' file.txt Now i ...
0
votes
3answers
49 views

Append a string to the beginning of second line using unix

I want to append a string to the beginning of second line of my text file, for example: 1 2 3 should become: 1 42 3 Any idea about how to go about with this thing?
0
votes
2answers
41 views

Monitor a logfile and execute different commands depending on different conditions

I have a logfile which I want to monitor and depending on which condition is met, different commands should be executed. I found a solution that comes close to this here. Unfortunately it doesn't ...
5
votes
5answers
221 views

Merging columns in a file using Awk

Input: ABC,SSSD,12345,NSS,12345,xxx,TS11 ,,,,,,TS21 ,,,,,,TS22 ,,,,,,BS26 ,,,,,,GPRS ABC,SSSD,12356,NSS,12356,xxx,TS11 ,,,,,,TS21 ,,,,,,TS22 ,,,,,,GPRS ,,,,,,BS26 Output: ...
1
vote
3answers
54 views

Find lines in (muliple) files that contain the given line in an input file

So, if I have a number of files like the following: file1: 123 456 789 012 file2: line1 922 line2 392 line3 456 line5 291 line6 201 ... file3: line1 111 line2 123 line3 19 line5 542 ...
1
vote
2answers
47 views

How to subtract lines, in reverse, only if greater than previous line

I am working on a shell script that will read the lines of a file, in reverse. I need it to subtract the previous line from the latter, but only if the latter is larger. If it's smaller, then I just ...
0
votes
2answers
30 views

How to add a new text after every matched string using shell command?

Input: Job name: ns, Job ID: 2312, Status: ODB_ACTIAVTION SUCCESSFUL Job name: ps, Job ID: 3353, Status: ODB_ACTIAVTION SUCCESSFUL Job name: pm, Job ID: 1265, Status: ODB_ACTIAVTION SUCCESSFUL Job ...
2
votes
4answers
56 views

Join pattern line and consecutive non-empty lines, separated by commas and enclosed in ()

I have a file that has click css_add_violation_false click css_add_claim_false and select css_driver1_birth_date_month birth_date_month select css_driver1_birth_date_day birth_date_day How ...
4
votes
2answers
92 views

POSIX behavior of sed's ^ and $ regex anchors with multiline pattern spaces

What is the POSIX behavior of the sed ^ and $ anchors when used at the beginning and end of a regex if the pattern space contains embedded newlines, like after you use the N command? Should they ...
3
votes
4answers
58 views

sed - how to remove all lines that do not match

I have a html file. I want to remove all lines that do not start with <tr>. I tried: cat my_file | sed $' s/^[^tr].*// ' | sed '/^$/d' but it deleted all the lines.
8
votes
4answers
730 views

Splitting file for every 10000 numbers ( not lines )

I have a file that looks like the following: chr19 61336212 + 0 0 CG CGT chr19 61336213 - 0 0 CG CGG chr19 61336218 + ...
1
vote
3answers
56 views

Process text file - Extract data starting with number

I have a list of contact information I need to process in .txt format seperated by , . We only want to keep SIC codes that start with 65 which is real estate. The command should only check for the ...
0
votes
4answers
49 views

How to add double quotes to every line and then add a comma at the end of the line?

The commands to ad double quotes: sed -e 's/.*/\"&\"/' some_file The command to add comma at the end of each line: perl -i -pe 's/$/,/' some_file I've tried both of them,if I add first ...
10
votes
6answers
455 views

Remove lines from a file depending on lines found in another file

File file1.txt contains lines like: /api/purchase/<hash>/index.html For example: /api/purchase/12ab09f46/index.html File file2.csv contains lines like: <hash>,timestamp,ip_address ...
5
votes
3answers
46 views

Censor text with regex

I'm currently running this command to censor an indented to-do list. sed -e 's/\(\s\+- \)\(.*\)/\1XXX/g' It's great except that I'd like the number of Xs to match the number of matched characters. ...
0
votes
5answers
63 views

Print all data related to a certain field

Considering below file: foo,5 foo,7 foo,9 boo,5 boo,10 boo,10 What I am looking to, is printing all the data in $2 that is related to $1 in one record. For this example the needed output will be: ...
2
votes
2answers
97 views

Find differences in file and sorting file by key

So I have two language files, english and french. They are in this format: key=translation An example in English would be: ui.title=My Title And in French: ui.title=Mon Titre So I need a ...
2
votes
1answer
53 views

Compare 1st Column in 2 Files and Replace 3rd Column of File 1 with 4th Column of File 2

I have a requirement of comparing the first column between 2 pipe delimited files and if they match, I need to replace the 3rd Column in File 1 with 4th Column in File 2. File 1: 111|xyz|23345 ...
1
vote
1answer
40 views

add pairwise values looking up from a file

I have several variables (col1) which have 3 values (col 3,4,5) for some levels (col2) mango123 L1 1 2 3 mango123 L2 4 5 6 squash378 L1 7 8 9 squash378 L5 10 12 13 squash378 L15 0 1 3 ...
3
votes
3answers
64 views

Multiple column matching and adjusting with awk

I have a problem that's giving me a bit of a headache, but it doesn't tend to take much as a relative beginner. I have a file that looks like this ID A1 A2 A3 1 A G A 2 T G A ...
0
votes
2answers
31 views

sed command to append different text at multiple occurrence of string

I want to use sed command to append text at multiple occurrence of string. For Ex: Hello world types="program" Mario types="Game" Hello world types="program" Mario types="Game" So at first ...
2
votes
2answers
42 views

filter based on a field value in awk

I want to print the countries which which have to after ~:. welcome~:to~:Germany welcome~:no please~:Italy welcome~:to~:Brazil welcome~:not ok~:China Note: I do not know what will be there other ...
1
vote
1answer
22 views

Compare records value with each others.

Please consider below file: foo,boo,900 foo,boo,900 foo,boo,850 I need to compare the a field ($3) with the next record, if the difference is equal or more than 50, then print the record. i.e ...
2
votes
4answers
104 views

Fill empty lines in specific column with values

I have data of the following format: 2342 1 6473 2 7463 2 3647 2734 1 9343 The lines that do not have a value in the 2nd column are empty in that spot (i.e. there is no space, tab etc.). I am ...
2
votes
2answers
32 views

Can I convert fields in XML to tags using xmlstarlet?

I want to convert fields in Tags to tags within that tag for example <book name="Data Structure" price="250" pages="350"/> To <book name="Data Structure"> ...