Tagged Questions

2
votes
4answers
113 views

Using sed, how to format one word per line, removing white space

I'm trying to replaces patterns and cleanup a file containing multiple words to get one word per line. The result is achieved using this command line: sed -e '/^[[:space:]]*$/ d' \ # ...
4
votes
2answers
61 views

Use sed or awk to join lines in a certain pattern?

Not sure on how to turn this input: A B C apartner bpartner cpartner into this output: A apartner B bpartner c cpartner I have this pattern in a 80 line file that needs to be 40 line. Also, how ...
2
votes
3answers
78 views

Delete lines containing repeated text

I have a file containing two paths on each line. I want to remove the lines containing the same path twice. I work on Linux and Solaris. I would like a one-liner in sed or awk or perl. Example input ...
2
votes
3answers
64 views

I want to find duplicates in a file and add a character to the end of the line on the 1st match

I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line. eg my file (test.html) contains the following entries host= ...
2
votes
2answers
62 views

Replacing text between two HTML Comments

I am new to the world of sed/awk and regex in general and have been studying their usage, but have been floundering trying to meet my need: I have an htm page that has a single line notice that needs ...
3
votes
3answers
82 views

sed + how to remove character/s that start or ended on each number

How do I remove the . character(s) that start in the beginning of each number or end on each number? Remark – perl one liner also good alternative for sed. Example input: .23.12.44.5. .233.3.3.3 ...
3
votes
3answers
77 views

checking data in columns when a data or some may be missing or present?

I am not sure if this possible. say i have columns like : Team Colour Game Rainfall PlayerName XYZ Blue Cricket Yes Kapil suppose i need to search ...
4
votes
3answers
107 views

Why is sed giving me an error about an unterminated `s'?

I have a set of sed replacements in a bash script and am getting an error about an unterminated `s' command. Here's what the sed line looks like: sed -n -e "s/TMPFOO1/$FOO1/" -e "s/TMPFOO2/$FOO2/" ...
2
votes
3answers
178 views

How can I use SED or AWK to replace placeholders in a template file with variable content that contains special characters?

CentOS 6.3 I'm trying to get a small script to send an email containing a copy of email headers in the body (for the purpose of internal reporting). The template file contains the following: ...
0
votes
1answer
151 views

comparing lines with awk vs while read line

I have two files one with 17k lines and another one with 4k lines. I wanted to compare position 115 to position 125 with each line in the second file and if there is a match, write the entire line ...
1
vote
2answers
137 views

Unix command for pattern matching

I want to count the number of words and print the matched pattern lines which matches exactly with following pattern: abc-ERROR: The input File contains: # abc-ERROR: xyxxkkfgfr # def-Error: ...
4
votes
4answers
106 views

How to remove character and space from a string

Hello I have one file with output Name : ABC Name : CDE Name : ZYS I want to get the result ABC CDE ZYS How can i get it through SED command. I have tried. sed s/'Name' but ...
9
votes
3answers
304 views

Deleting all C comments with sed

I am trying to write a script that will delete all comments and everything in between inside C files in my current directory. I've been using sed, and this is what I have so far: sed -i '/ * [^()] ...
0
votes
2answers
36 views

How to delete lines where the given part of the line is more than 100 chars?

I know I can delete lines that are longer than 100 chars with this: sed '/^.\{100\}..*/d' But I have the following example line: <a ...
2
votes
2answers
94 views

add “#” in begining to selected lines in File

Requirement : ADD someting (i.e #) in begining of every line in file A which is grep from file B File A abcd abdc sdfg asdfa jon ram File B jon abcd grep file B from file A and add "#" ...
1
vote
2answers
99 views

Regex and piped commands with sed

I'm finding really hard to use sed command, plus I can't seem to find well written tutorials. Let me say that I worked with regular expression in other languages (Python, JavaScript, Java), so that ...
3
votes
1answer
42 views

How to edit files in-place while performing operation on the values matching the searched pattern?

Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ? Ideally a solution ...
1
vote
2answers
66 views

Match ids and print original file

I have two files Original: (5000 entries) Chr Position chr1 879108 chr1 881918 chr1 896874 … and a file with allele frequencies (2000 entries) Chr Position MAF chr1 881918 0.007 ...
2
votes
2answers
391 views

How can I replace text after a specific word using sed?

I have a file named .ignore. In need to replace the projdir. For example: ignore \..* ignore README projdir Snake I need to replace Snake with, for example, "PacMan". I read the man page, but I ...
2
votes
6answers
189 views

Is there a faster way to remove a line (given a line number) from a file?

A related question is here. I often have to edit a large file by removing a few lines from the middle of it. I know which lines I wish to remove and I typically do the following: sed ...
2
votes
1answer
47 views

extracting and copying parts of tex files into new tex files

I have a bunch of tex files in different folders on my system. The goal is to create new tex files (using a template, that has the title page, packages, new commands,...) that combines all the ...
2
votes
4answers
94 views

How do I correct both too many and too few space characters?

I have files that are edited by people other than me. They contain lines that look like this: <E> This is text </E> I use some Bash shell scripts do a bunch of text replacement ...
1
vote
2answers
90 views

Print fields from file only if line starts with a specific word

I have the following file: … LINK=dummy 172.17.100 => 10.218.11 [PATH=/etc] FILE=hosts 172.17.100 => 10.218.11 [PATH=/etc] FILE=network 172.17.100 => 10.218.11 ...
4
votes
4answers
1k views

Text between two tags

I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
4
votes
3answers
303 views

How to remove all white spaces just between brackets [] using bash? [duplicate]

Replace text between brackets Input testing on Linux [Remove white space] testing on Linux Output testing on Linux [Removewhitespace] testing on Linux So, how can we just remove all the ...
2
votes
2answers
62 views

sort a file based on length of the column/row

I need to sort a file based on the number of chars in the first column. I have no idea on how to go about this. (On Linux, so sed/awk/sort is available). An example: .abs is bla bla 12 .abc is bla ...
1
vote
2answers
339 views

Help inserting a new line of text after matching a line of text (sed)?

I need to insert a hidden HTML input tag into any form tag within a bunch of HTML files. I assume this is possible with sed, but need help forming the command. My idea is to search for any instance ...
3
votes
4answers
1k views

removing the first and the last character of every line from command line

I am trying to remove the first and the last characters of everyline in a text file and save the resulting truncated version in a new file. Does anyone have an idea about how to do that efficiently ...
3
votes
2answers
672 views

sed: multi-line replace of config block

I have some configuration files that basically look like (...content...) # BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY (... more content ...) # END DYNAMIC BLOCK (... even more content ...) Now, in ...
8
votes
4answers
417 views

How do I write a sed one-liner to add a character after every third character?

So, I have a string that looks like this: AUGGCCAUGGCGCCCAGAACUGAGAUCAAUAGUACCCGUAUUAACGGGUGA And I want to split the string into 3-character chunks delimited by a '+' sign. ...
1
vote
3answers
191 views

Script matching literal pattern over multiple lines?

I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print ...
7
votes
10answers
547 views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
1
vote
2answers
234 views

Deleting specific lines and duplicates from a 11gb wordlist text file

I have an 11gb wordlist file which is already sorted as each word is on its own line. I need to remove duplicates and lines starting from 077. I guess I need to run sed and sort -u together but I ...
1
vote
5answers
1k views

How to print all lines after a match up to the end of the file?

Input file1 is: dog 123 4335 cat 13123 23424 deer 2131 213132 bear 2313 21313 I give the match the pattern from in other file ( like dog 123 4335 from file2). I match the pattern of the line ...
3
votes
2answers
146 views

Extract lines containing PAT1 but not PAT2 with sed

I want to extract the lines containing PAT1 but not PAT2 from a file. For example, with PAT1='dog', PAT2='cat', and the following input: 1 cat chicken 2 bird dog apple 3 dog orange cat 4 cat juice ...
0
votes
2answers
160 views

Easiest way to find/replace in a file using a list?

I have a file, foo.txt, and a regexp I want to find in that file. Each time I find the regexp, I want to take a line from another file, bar.txt, and substitute it in for the regexp match I found in ...
0
votes
1answer
262 views

Remove a block of lines between two patterns [duplicate]

Possible Duplicate: Show only text between 2 matching pattern In a bash script using sed how can I remove a block of lines beginning with -pattern a- and ending with -pattern b- where the ...
3
votes
3answers
197 views

Delete a block of lines with a condition on the last line

I have a log-file where at the end of a series of lines you can see if this block is relevant. Now I'm looking for a command like sed to delete de blocks ending with "Content-Length: 0" and beginning ...
5
votes
2answers
366 views

Replacing missing value blank space with zero

I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt ...
1
vote
4answers
347 views

Replace text between brackets

I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like: SOMETHING [BAR, WIBBLE] SOMETHING [BAR] This gives the desired result of: SOMETHING [FOO, FOO] SOMETHING [FOO] ...
10
votes
4answers
2k views

Delete last line from the file

I use sed to quickly delete lines with specific position as sed '1d' sed '5d' But, what if I want to delete the last line of the file and I don't know the count of lines (I know I can get that ...
4
votes
3answers
262 views

Delete the matching line and several more from a file

I have a text file called file_a.txt. My first command is grep -A 12 ".production =" file_a.txt The output is a few block. Each block of string contains 13 rows I specifically want to delete all ...
1
vote
2answers
165 views

Substitute pattern within a file with the content of other file

I have a text file (devel.xml). I added the word REPLACETHIS to it in order to replace this string with the content within a different file (temp.txt). The closest thing I have is this: sed -i -e ...
2
votes
3answers
1k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
7
votes
5answers
3k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
2
votes
2answers
628 views

Replace the shortest match of a string pattern

I have this string: update mytable set mycol=myvalue where mycol=yourvalue; I need to convert it to: insert into mytemp select * from mytable where mycol=youvalue; I can accomplish it like this ...
2
votes
1answer
225 views

Using sed, how can I replace a line containing a pattern with contents from another file?

input_file.txt: one fffds two xxxx three ffff four ffde data_file.txt: six seven I want to replace the line containing two xxxx in input_file.txt with the content of data_file.txt. The output ...
3
votes
4answers
2k views

How to get all lines between first and last occurrences of patterns?

How can I trim a file (well input stream) so that I only get the lines ranging from the first occurrence of pattern foo to the last occurrence of pattern bar? For instance consider the following ...
6
votes
7answers
256 views

Swapping an unlimited number of columns

I have a file with columns. See below for an example: a b c ... z 1 2 3 ... 26 I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc.. z y x ...
3
votes
2answers
104 views

Column mismatch and substituting

input.txt (tab-delimted) TTTTOTTT00000000008 RTTTT899 5.00E-28 TTTTOTTT00000000046 RTTTWRR 3.00E-31 TTTTOTTT00000000051 2.00E-11 TTTTOTTT00000000051 7.00E-12 TTTTOTTT00000000054 ...

1 2 3