Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

10
votes
7answers
1k views

How can I prepend a tag to the beginning of several files?

I need to add PHP tags surrounding a file. It's easy to append them using find . -exec echo "?>" >> '{}' \; but how can I prepend the tag <?php?
8
votes
5answers
6k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
5
votes
1answer
6k views

Extracting a regex matched with 'sed' without printing the surrounding characters

To all the 'sed' doctors out there: I have a seemingly trivial 'sed' question to which I have not been able to find a solution. How can you get 'sed' to exctract a regular expression it has matched ...
14
votes
4answers
16k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
5
votes
4answers
5k views

Text between two tags

I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
4
votes
3answers
1k views

Show lines matching a pattern and the 4 lines before each

For example, from this file: CREATE SYNONYM I801XS07 FOR I8010.I801XT07 * ERROR at line 1: ORA-00955: name is already used by an existing object CREATE SYNONYM I801XS07 FOR ...
5
votes
3answers
385 views

Why does sed act differently depending on the output file?

If I run: cat messages.txt | sed -e 's/a/a/g' > messages.txt on one large file (2500+ lines) I find that the resulting file will only have about 900 lines after the command in cygwin and will ...
3
votes
2answers
3k views

Show only text between 2 matching pattern

I'm going to submit form using cURL, where some of the contents is come from other file, selected using sed If param1 is line matching pattern from other file using sed, below command will works ...
8
votes
4answers
8k views

Remove line containing certain string and the following line

I use this cat foo.txt | sed '/bar/d' to remove lines containing the string 'bar' in the file. I would like however to remove those lines and the line directly after it. Preferably in sed, awk or ...
10
votes
5answers
276 views

How to achieve portability with sed -i (in-place editing)?

I'm writing shell scripts for my server, which is a shared hosting running FreeBSD. I also want to be able to test them locally, on my PC running Linux. Hence, I'm trying to write them in a portable ...
8
votes
4answers
1k views

Replacing Multiple blank lines with a single blank line in vim / sed

Question more or less says it all. I'm aware that /^$/d will remove all blank lines, but I can't see how to say 'replace two or more blank lines with a single blank line' Any ideas?
7
votes
1answer
14k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
2
votes
6answers
1k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
4
votes
2answers
153 views

Why do I need to escape regex characters in sed to be interpreted as regex characters?

It seems e.g. cat sed_data.txt | sed 's/\b[0-9]\{3\}\b/NUMBER/g' that I must escape characters to form a regular expression. In this case I had to escape braces in order to be interpreted as a number ...
4
votes
3answers
534 views

Delete the matching line and several more from a file

I have a text file called file_a.txt. My first command is grep -A 12 ".production =" file_a.txt The output is a few block. Each block of string contains 13 rows I specifically want to delete all ...
3
votes
4answers
138 views

Rearrange text file by prepending data rows with header row

I have a file whose content looks like this: 2009 150 0 0 0 0000 75.316 0.0400390625 0.00007 0.00000 0.8980 76.216 0.0400390625 0.00007 1.00000 0.9046 77.217 0.0400390625 ...
3
votes
4answers
319 views

Piping paths with different types of quotes for slash substitution

I would like to use sed to convert a path with backslashes to the same path with forward slashes: E.g. I would like to pipe \\path\to\file\ and obtain /path/to/file None of the following commands ...
3
votes
2answers
928 views

Insert a multiline string into another string

I need to insert lines into an xml file : Insert <one> </one> into <tags> </tags> To obtain <tags> <one> </one> </tags> I tried this: ...
3
votes
2answers
6k views

sed one-liner to delete everything between a pair of brackets?

I am working with some text that is full of stuff between brackets [] that I don't want. Since I can delete the brackets myself, I don't need the one-liner to do that for me, but I do need a one-liner ...
2
votes
3answers
447 views

Using a reference to a bash string variable in sed

I'm a relative Linux novice. Suppose that I have a text file a.txt that contains the following text: A B C Now, if I want to change the text on line 2 (which contains B), I can use the following ...
2
votes
2answers
3k views

How can I replace text after a specific word using sed?

I have a file named .ignore. In need to replace the projdir. For example: ignore \..* ignore README projdir Snake I need to replace Snake with, for example, "PacMan". I read the man page, but I ...
2
votes
2answers
2k views

Sed: replacing digits

I am trying to write some install scripts for when I install/upgrade Ubuntu. I'm having trouble with fixing logrotate.d: /var/log/ufw.log { rotate 7 weekly missingok notifempty compress delaycompress ...
1
vote
5answers
489 views

How can I use variables when doing a sed?

I want to do: cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh in my program. But I want to use variables, e.g. old_run='old_name_952' new_run='old_name_953' I have ...
1
vote
4answers
536 views

Replace text between brackets

I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like: SOMETHING [BAR, WIBBLE] SOMETHING [BAR] This gives the desired result of: SOMETHING [FOO, FOO] SOMETHING [FOO] ...
22
votes
9answers
4k views

Addition with 'sed'

I am trying to perform a mathematical operation with sed, but it continues to treat my variables as strings. The input is of this kind: $ echo 12 | sed 's/[0-9]*/&+3/' $ 12+3 I'd like to have ...
14
votes
9answers
2k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
8
votes
4answers
3k views

Efficient in-place header removing for large files using sed?

The commands below may takes minutes depends on the file size. Is there any more effient method? sed -i 1d large_file
8
votes
1answer
2k views

Differences between sed on Mac OSX and other “standard” sed?

I am having some issues in using an answer provided on this site for this question about a sed command to replace a blank line with two other lines of content, and it was brought up if the sed command ...
6
votes
1answer
159 views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
5
votes
5answers
601 views

resolve all ip addresses in command output using standard command line tools

I have several log files that contain a bunch of ip addresses. I would love to be able to pipe the data through a program that would match and resolve ip addresses. I.E. cat /var/log/somelogfile | ...
8
votes
3answers
3k views

Zero-fill numbers to 2 digits with sed

Input: 201103 1 /mnt/hdd/PUB/SOMETHING 201102 7 /mnt/hdd/PUB/SOMETH ING 201103 11 /mnt/hdd/PUB/SO METHING 201104 3 /mnt/hdd/PUB/SOMET HING 201106 1 /mnt/hdd/PUB/SOMETHI NG Desired output: 201103 ...
7
votes
5answers
315 views

How to extract logs between two time stamps

I want to extract all logs between two timestamps. Some lines may not have the timestamp, but I want those lines also. In short, I want every line that falls under two time stamps. My log structure ...
5
votes
1answer
152 views

Execute a command within 'sed'

I have a list of random memory addresses saved in a text file, and I need to sed them out, decode them, and save them back to my file. However, the decoding operation I'm trying to perform is handled ...
5
votes
3answers
2k views

The way to use `/usr/bin/env sed -f ` in shebang?

Typing /usr/bin/env sed -f in terminal works. But if use it as a shebang, #!/usr/bin/env sed -f s/a/b/ The script will be fail to execute: /usr/bin/env: sed -f: No such file or directory I ...
4
votes
6answers
96 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?
4
votes
2answers
383 views

Append line to many files

I want to add some text to over 200,000 files I am trying this find . -name *.txt -print | xargs -I % echo "hello world" >> % But nothing is happening. When i run find . -name *.txt it work ...
4
votes
3answers
1k views

Inserting text at the beginning of a file with sed via the terminal in Linux [duplicate]

Possible Duplicate: How can I prepend a tag to the beginning of several files? How do I insert text at the beginning of a file via terminal?
3
votes
1answer
466 views

Get all regex matches between two patterns and print them to file

I've got a file with a bunch of long lines. I'd like to grab every group between two patterns and print them to a new file, one match per line. I could manage to do this with Python, but I'd prefer ...
2
votes
2answers
985 views

Comparing two files using Unix and Awk

I need to compare two files, File1 and File2 (Separated by space) using 4 fields (Field 1, 2, 4 and 5 of File 1 with field1, 2, 4 and 5 of File2). Logic: If column 1, 2 and 4 of File 1 matches with ...
2
votes
1answer
2k views

Replacing string based on line number

I have a situation where i want to replace a particular string in many files Replace a string AAA with another string BBB but there are lot of strings starting with AAA or ending in AAA ,and i want ...
2
votes
3answers
2k views

What is the purpose of -e in sed command?

I can't find any documentation about the sed -e switch, for simple replace, do I need it? e.g. sed 's/foo/bar/' VS sed -e 's/foo/bar/'
2
votes
2answers
9k views

Using sed to get rid of characters < > ,

I have a file with lines as follows: ... ... <230948203[234]>, ... ... <234[24]>, ... .. I would like to use sed to remove the characters < , and > from every line I tried using ...
1
vote
3answers
329 views

Switch from grep to sed

I have a files students.txt, with lines of the form: Surname, Forename: Day.Month.Year: Degree For example: Smith, John: 15.01.1986: MSc IT Taylor, Susan: 04.05.1987: MSc IT Thomas, Steve: ...
7
votes
3answers
2k views

How can I instruct BSD sed to interpret escape sequences like \n and \t?

I have a sed replacement command that I would like to be compatible with BSD sed as well as GNU sed. Extended regular expressions are not an issue as I do not need them in this case. My primary ...
7
votes
1answer
293 views

Change date “July 29th, 2011” to “20110729”

I have lots of HTML files contains date with format July 29th, 2011 I want to change date format July 29th, 2011 to 20110729, December 9th, 2010 to 20101209, etc. I think sed may works but still ...
5
votes
2answers
414 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
4
votes
5answers
292 views

Pattern matching across multiple lines

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output? I was using something like the following: awk ...
4
votes
2answers
338 views

Out of memory while using sed with multiline expressions on giant file

I am currently trying to remove all newlines that are not preceded by a closing parenthesis, so I came up with this expression: sed -r -i -e ":a;N;$!ba;s/([^\)])\n/\1/g;d" reallyBigFile.log It does ...
4
votes
3answers
322 views

How to parse a file to extract 3 digits numbers kept in a “group number”

I am a beginner, and trying to write a shell script to parse a text file that is extracted from a standardization pdf file. I would like for each test group (identified by Group 0, Group 1... to get ...
4
votes
3answers
451 views

How to skip file in sed if it contains regex?

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed: find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+ As ...