Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

5
votes
8answers
163 views

awk or sed command to match regex at specific line, exit true if success, false otherwise

I need to determine if a file contains a certain regex at a certain line and to return true (exit 0) if found, and otherwise false. Maybe I'm overthinking this, but my attempts proved a tad unwieldy. ...
1
vote
4answers
52 views

Cost-efficiently pair each line of a file with all others

I have a very huge file containing numbers only. file - 123212 234234 12324 1243223 5453443 And want to pair each line with all others. Output like below 123212,234234 123212,12324 123212,1243223 ...
1
vote
2answers
24 views

Replacing a “time” column with corresponding values

I have a file in the following format: "2004-04-19 12:25:57" 44 44 "2004-04-19 13:39:32" 36 36 "2004-04-19 14:00:53" 34 34 I need 2 new files: a) A file which will replace "time" values of the ...
105
votes
3answers
224k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
24
votes
12answers
12k views

Is there a robust command line tool for processing csv files?

I work with CSV files and sometimes need to quickly check the contents of a row or column from the command line. In many cases cut, head, tail, and friends will do the job; however, cut cannot easily ...
7
votes
7answers
28k views

bash - replace space with new line

How can I replace spaces with new lines on an input like: /path/to/file /path/to/file2 /path/to/file3 /path/to/file4 /path/to/file5 etc... To obtain the following: /path/to/file /path/to/file2 ...
-1
votes
3answers
37 views

How to delete certain line from file?

I have a text file with line containing only comma. How can I delete this particular line? Consider following example: CREATE TABLE ENTERPRISE.TSDAILYACCTSUMMARY ( TSDAILYACCTID FOR COLUMN ...
-1
votes
1answer
46 views

How to add double quotes to a various strings in a same pattern in a file

I have a file like this CREATE TABLE ENTDTA.$$BAPWD ( PERSID NUMBER(10) DEFAULT 0 NOT NULL , "PASSWORD" CHAR(10) DEFAULT '' NOT NULL , UPDATE_IDENT NUMBER(7, 0) DEFAULT 0 NOT NULL ) ...
2
votes
2answers
51 views

How can I compare two files and create another file that aren't in first file?

file1 contains: 100 200 300 400 file2 contains: 500 600 700 200 300 Expected output file will be: 500 600 700
2
votes
1answer
26 views

Merge text files with 1st column and difference in output file

File 1: (text1.txt) Row_Added_Ts Count 01/01/14 2022448 02/01/14 8345155 03/01/14 8663852 04/01/14 6785739 05/01/14 5279913 File 2: ...
1
vote
1answer
36 views

Merge lines with matching first column

I've been trying to use awk to merge two files when they have the same first column. Here are my example files: FileA.txt A2M 1 A4GALT 11 AAAS 35 AAGAB 7 FileB.txt A4GALT 2 AAAS ...
4
votes
1answer
52 views

How to use awk to indent a source file based on simple rules?

How can I indent source code based on a couple of simple rules? As an example, I've used sed and ask to transform a selenium HTML source table to the following rspec like code. How could I ...
32
votes
7answers
3k views

Is there a tool that combines zcat and cat transparently?

When handling log files, some end up as gzipped files thanks to logrotate and others not. So when you try something like this: $ zcat * you end up with a command line like zcat xyz.log xyz.log.1 ...
-1
votes
1answer
22 views

Look for a particular pattern and do a swap on that line

I have a set of files containing VIEW_NAME FOR COLUMN DNAME , VIEW_OWNER FOR COLUMN DCREATOR , OBJECT_NAME FOR COLUMN ONAME , OBJECT_SCHEMA FOR COLUMN OSCHEMA So, ...
1
vote
4answers
93 views

Show ratio for all availiable resolutions

I tried to get the ratio with this script: echo "$(xrandr)"|cut -f 4 -d" "|sed 's/x/00 \\\* /g'|while read i; do echo "$i = "$(expr $i); done but I get errors like 102400 * 768 = expr: ...
0
votes
1answer
21 views

replacing first link of a text “awk equivalent of sed command”

I wanted to replace the first line of a text with sed it should be like sed -i.bak "1 s/^.*$/"CompoundState\tMethod\tApproach\tS^2\tEnergy\tPath"/" awk.xls but I also need to run it on OSX but it ...
4
votes
2answers
217 views

How to change all strings in python file from snake_case to camelCase in sed

I've tried some sed patterns like this from commandlinefu sed -r "s/('[a-z]+)_([a-z])([a-z]+)/\1\U\2\L\3/" But somehow it's not working. For one thing they forgot the digits, which I can fix, but ...
2
votes
2answers
44 views

adding column in CSV file using awk

I need to a add a column in CSV file from an array using awk. For example, input.csv a,10 b,11 array = (100 200) output.csv should look like a,10,100 b,11,200 I tried this command but it does ...
3
votes
2answers
79 views

Using grep to print all of the matching strings not separated by a space

I'm looking through a text file of emails and trying to get a list of ones from a certain domain. I'm running: grep -oh "\w*domain.com\w*" file.txt This finds the lines, but only prints that ...
3
votes
2answers
1k views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
7
votes
4answers
202 views

How to grep -v and also exclude the next line after the match?

How to filter out 2 lines for each line matching the grep regex? this is my minimal test: SomeTestAAAA EndTest SomeTestABCD EndTest SomeTestDEFG EndTest SomeTestAABC EndTest SomeTestACDF EndTest ...
6
votes
4answers
195 views

How to add a header and/or footer to a sed or awk stream?

I have a bunch of output going through sed and awk. How can I prefix the output with START and suffix the answer with END? For example, if I have All this code on all these lines and all these ...
2
votes
1answer
31 views

Filtering and extracting PIDs as space-separated list from ps using sed

I want to filter and format the output of ps -ax -o pid,command, but not sure what pipe to use for it. I want to filter those lines matching a fixed string at a fixed position, just keep the pids for ...
17
votes
9answers
13k views

Text processing - join every two lines with commas

I have more than 1000 lines in a file. The file starts as follows (line numbers added): 1. Station Name 2. Station Code 3. A N DEV NAGAR 4. ACND 5. ABHAIPUR 6. AHA 7. ABOHAR 8. ABS 9. ABU ROAD 10. ...
6
votes
5answers
888 views

Grouped sorting of continuous paragraphs (separated by blank line)?

I think I'm pretty experienced now in sorting by columns; however, I haven't found anything so far how to sort continuous rows. Supposing we have a text file that looks like this: (very simplified, ...
0
votes
5answers
39 views

update the json file

I have a file, actually json file. How I can add a comma to the end of every but the last line ? For example: {"a":1, "b":1, "c":2} {"a":3, "b":3, "c":1} {"a":1, "b":2, "c":3} What I want: {"a":1, ...
2
votes
1answer
37 views

Question about replacing character in text file

So I have some data that is parsed in this format seperated by colons: username:email@email.com:password:salt Sometimes the username/salt can contain a colon (:) in it. I want to replace the ...
2
votes
3answers
93 views

FNR and NR variables problem

I have two files, one of them has 10 records, and the other one with 15 record, the problem is when I cat them together, always FNR=NR. For example, consider below files: File1: 1,boo 2,foo 3,boo ...
0
votes
2answers
38 views

How to search between the 2nd and 3rd delimiters

So, I have a data file and I would like a match to occur only if a match is found between the 2nd and third Vertical Bars (|) So given this data sample, I if I search for 'wilson' I want the 2nd line ...
19
votes
9answers
3k views

Grep lines but let the first line through [duplicate]

Consider the following output from df. Filesystem Size Used Avail Use% Mounted on /dev/sda1 23G 6.1G 16G 29% / udev 10M 0 10M 0% /dev tmpfs 397M 420K ...
4
votes
2answers
202 views

grep string where next line does not contain string

I want to search all files inside a directory and its subdirectories for lines containing a certain string, but I want to exclude those results that contain a different certain string in the line ...
3
votes
4answers
215 views

Conditional if clause in awk

Please consider below file: foo,1000,boo,A foo,1000,boo,B foo,1001,boo,B foo,1002,boo,D And we have below rules: If $2 equal 1000, $4 should be equal A If $2 equal 1001, $4 should be equal B If ...
0
votes
1answer
37 views

How to add new column, where the value is based on existing columns with awk

I have a file of multiple columns. I would like to make an additional column based on the values of 2 columns from this file. Example input: A B C D E F 1 2 T TACA A 3 2 3 4 I R 8 2 9 3 A C 9 3 ...
3
votes
3answers
47 views

Find a pattern and insert # at the beginning of 2 line befoe that and 1 line after that

I have a big file (more than 2000 lines). Where i have to insert a # at the beginning of 2 above lines and at the beginning of 1 below line after finding a pattern. Also insert # at the beginning of ...
4
votes
4answers
133 views

Current date in awk

Is there a constant variable in awk, that store today's date? if no, is there a way that can store today's date for daily use? let's say we have below file: boo,foo,2016-08-30 foo,boo,2016-07-31 ...
0
votes
5answers
68 views

Appending One File to the Second Until End of File

I have two files. The first is just a single number. This number is calculated and changes every time. The second file has a bunch of columns, and I do not know the number of rows (which can change). ...
4
votes
6answers
200 views

Changing matrices into single lines of data

I have multiple matrices in a .txt file and I need each matrix to be in a single line. For example, matrices.txt: 1 2 3 4 2 3 4 5 3 4 5 6 3 4 5 6 2 3 2 5 2 3 4 5 2 3 5 6 2 3 4 5 ... What I would ...
1
vote
1answer
48 views

How to print the line numbers with corresponding line that matches a pattern using shell command? [duplicate]

File, .false alarm is bla no. 11 .no alarm generated is bla .application will be killed is bla wall command .doc file is document file with .doc extension .no authority for this selection bla Now ...
1
vote
2answers
32 views

How to replace string in files except with a line begins with # [duplicate]

I've looked at some other questions related to using sed to replace strings, but I'm having trouble figuring out how to apply it to my particular need. In a case like the snippet below, I want to ...
1
vote
5answers
119 views

format file to remove " characters

I have a file with the following data "MG1507XXXXXX|" "|020000XXXXXX" "20261031|" "|3,827.92" "|3,581.41" "|542,729.62" "MBA" "MG1507XXXXXX|" "|020000XXXXXX" "20261130|" ...
0
votes
1answer
82 views

How to replace \1\ with new line

I have a record like: "evSchema"\1\"UAT" "evSN"\1\"uadb" "evDirsep"\1\"/" "evRootPath"\1\"/work_area/APP_UAT/" "evSchema"\1\"RMS13" "evUser"\1\"STAGE" I want to replace \1\ with new line. I tried ...
3
votes
2answers
441 views

Need shell script to transform a CSV into Apache httpd format

Need some pointing in right direction on script to fetch and regex or sed. Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV ...
-1
votes
1answer
28 views

replace numbers with zero while we have the lines number [closed]

I am trying to write a bash script (containing awk or sed ) to replace some numbers with zero. for example : replace line 3 second and third number, and line 4 first and third number and line 5 first ...
1
vote
3answers
24 views

fetch a column from a file based on another column

I have a file which contains approximately 5 million records, as follows: - 1223423,21,foo,data1,data2,data3,data4,data5,45,267,index1 4234234,34,bar,cat1,cat2,cat3,cat4,cat5,34,2323,index2 ...
0
votes
1answer
33 views

Repeatedly extract every second and third line of a file with 3-line data blocks

I have a file where I have data in 3-line sets. I want the 2nd and 3rd line from each set. How do I get that? For an example from: [set no][no of line in set] 11 12 13 21 22 23 31 32 33 I want: ...
12
votes
4answers
369 views

Sort lines by number of words per line

Given input: hello: world foo bar baz bar: baz: bin boop bop fiz bang beep bap: bim bam bop boatkeeper: poughkeepsie I would like to sort it into most words at the top, to least at the end, like ...
4
votes
2answers
2k views

grep one line before the match plus the match

zzzzzzzzz aaaaaaaaa bbbbbbbbb & ccccccccc & ddddddddd hhhhhhhhh eeeeeeeee fffffffff & ggggggggg & in the above line, what i want is to grep/sed/awk (any method is fine) line that ...
19
votes
4answers
28k views

Grabbing the first [x] characters for a string from a pipe

If I have really long output from a command (single line) but I know I only want the first [x] (let's say 8) characters of the output, what's the easiest way to get that? There aren't any delimiters.
6
votes
3answers
191 views

How to print a line if that line or the next line do not contain a particular string

Input.txt: 8B0C remove 8B0D remove 8B0E remove 8B0F 8B10 remove 8B14 remove 8B15 remove 8B16 remove 8B17 remove 8AC0 8AC1 ...
0
votes
4answers
991 views

How to get data count between two strings with multiple occurences in the same file

I have the data in my file as START-OF-FIELDS ID NAME DEPT END-OF-FIELDS START-OF-DATA 1|joy|cs 2|sam|ec END-OF-DATA START-OF-FIELDS ID NAME DOB DEPT ADDRESS END-OF-FIELDS START-OF-DATA ...