Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

2
votes
3answers
1k views

What is the purpose of -e in sed command?

I can't find any documentation about the sed -e switch, for simple replace, do I need it? e.g. sed 's/foo/bar/' VS sed -e 's/foo/bar/'
2
votes
2answers
9k views

Using sed to get rid of characters < > ,

I have a file with lines as follows: ... ... <230948203[234]>, ... ... <234[24]>, ... .. I would like to use sed to remove the characters < , and > from every line I tried using ...
3
votes
1answer
460 views

Use sed to fix broken newlines

I have a folder full of files, each with many hundreds of lines. Many of these files have occasional broken newlines: when I open them in vim, many have ^M^L interspersed where newlines should be. ...
15
votes
3answers
27k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
1
vote
3answers
3k views

grep piping into sed, replacing inline; but I want sed to print the filename and changed line. Is it possible?

Here's my command (break intentional): grep FOO "/Users/gjtorikian/blah" -l | xargs sed -i '' '/FOO/{s/FOO/BAR/g; w /dev/stdout }' At the high-level: grep for FOO in the blah directory; pipe in ...
3
votes
2answers
926 views

Insert a multiline string into another string

I need to insert lines into an xml file : Insert <one> </one> into <tags> </tags> To obtain <tags> <one> </one> </tags> I tried this: ...
5
votes
2answers
11k views

Removing all spaces, tabs, newlines, etc from a variable?

This is the error I am getting and it's failing because of a variable whose value is supposed to be 2 (I am getting this using a select * from tabel). I am getting spaces in that variable. + 0 != ...
3
votes
3answers
196 views

How to perform an action only on the first line?

sed 's/.\(.*\)/\1/' myfile Say myfile contains: abc def ghi I want to remove the first character only from the first line but the above removes it from all the line.
3
votes
5answers
488 views

changing pattern of a text file

A text file has contents something like chair table pen desk Now i want it to be changed and stored in a variable say var as below ('chair','chair'),('table','table'),('pen','pen'),('desk','desk') ...
3
votes
2answers
209 views

What is this sed command doing?

Can someone explain me what this sed command is trying to achieve? sed 's/^[[:space:]]*//;s/[[:space:]]*$// I understand that it is searching for whitespace characters from the beginning of the ...
5
votes
1answer
146 views

Confusion about sed and '>'

The following command works fine: sed s/input/raw_input/ p.py >p2.py && mv p2.py p.py However, the following command turns p.py into an empty file: sed s/input/raw_input/ p.py >p.py ...
5
votes
1answer
6k views

Extracting a regex matched with 'sed' without printing the surrounding characters

To all the 'sed' doctors out there: I have a seemingly trivial 'sed' question to which I have not been able to find a solution. How can you get 'sed' to exctract a regular expression it has matched ...
1
vote
2answers
163 views

Where does the -i option in sed store the lines matching a regex?

I am trying to release space in my server's hd so I'm matching old dates in a log file with sed and it is decreasing but my root partition is now using 98% percent of it's space instead of 91% it was ...
4
votes
2answers
465 views

Replace one line with STDIN in a Makefile

One Makefile target is supposed to be in charge of: grabbing the output of $ perl Markdown.pl src/index.md (markdown to html) using that string to replace one line, something like CONTENT ...
1
vote
2answers
329 views

Prepend x/ to line where x is a variable on each line

I want to add $x/ from data formatted as $x-xyz (xyz is a mix of text and number to be ignored). Example data AB-10C CD-02E Output AB/AB-10C CD/CD-02E How do I describe the variable as text ...
3
votes
4answers
361 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...
4
votes
3answers
6k views

Delete range of lines above pattern with sed (or awk)

I have the following code that will remove lines with the pattern banana and 2 lines after it: sed '/banana/I,+2 d' file So far, so good! But I need it to remove 2 lines before banana, but I can't ...
20
votes
10answers
4k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
6
votes
1answer
394 views

How to detect and delete lines containing ˆ@

I have a simple problem: In my file, the are lines containing the string ˆ@ˆ@ˆ@ˆ@ˆ@ˆ@. I just want to delete all lines with this string, using for example the sed or grep commands. And I would like ...
4
votes
2answers
664 views

Moving average on a log file with awk or other unix utilities?

Scenario: I have a log file that has a few number of entry "classes", like this: R0 dx=0.00500 rb=0.00000 sn=1 3145.88 2.59 0.08 se=21315 id=16190 R0 dx=0.00300 rb=-1.00000 sn=1 3150.40 2.38 0.05 ...
3
votes
2answers
290 views

Extracting bigrams with sed

How do I split an entire text consisting of only letters into bigrams. For example: odd even od -> de dd -> ev This is what I have so far but it does not produce the desired result. [some ...
3
votes
2answers
564 views

sed inline editing fails for multiple sed statements

I want to append #SB# at the beginning of every line where it matches a given string incron.sh sed -i -e'/test1/s/^/#SB#/g' file1 sed -i -e'/test2/s/^/#SB#/g' file1 sed -i -e'/test4/s/^/#SB#/g' ...
4
votes
6answers
3k views

How can I convert tab delimited data to comma delimited data?

I'm requesting a list of ec2 snapshots via amazon's ec2 command line tool: ec2-describe-snapshots -H --hide-tags > snapshots.csv The data looks something like this: SnapshotId VolumeId ...
0
votes
2answers
1k views

nmap script needed to output text into csv format

ultimately I am looking to insert some info into a database. So I'm trying to output a file from something (probably nmap) that is formatted like this: 10.1.150.1,Up,1/3/2012,host.domain.lan ...
3
votes
1answer
532 views

What's wrong with this sed command?

This command runs fine: $ sed -e '/foo/{g; d}' myfile But this one has an error: $ sed -e '/foo/{g; a bar}' myfile sed: -e expression #1, char 0: unmatched `{' What's wrong with it?
14
votes
4answers
16k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
3
votes
5answers
4k views

Using sed to convert newlines into spaces

Say I have a shell variable $string that holds some text with several newlines, e.g.: string="this is a test" I would like to convert this string into a new string new_string where all line breaks ...
1
vote
2answers
964 views

put tab before every output line on AIX/ksh

0:root@SERVER:/root # echo "something" | sed -e 's/^/\t/g' tsomething 0:root@SERVER:/root # AIX/ksh .. why doesn't it works? I just want to put a "tab" before every line!
8
votes
4answers
11k views

How can I use sed to replace a multi-line string?

I've noticed that, if I add \n to a patter for a substitute using sed, it does not match. Example: $ cat > alpha.txt This is a test Please do not be alarmed $ sed -i'.original' 's/a test\nPlease ...
1
vote
1answer
680 views

Insert after a second pattern in sed

I'm using sed to edit a text file. The command: sed -n -e '/pattern1/,/pattern2/p' test01.txt show the lines between pattern1 and pattern2. But how can I insert a new line after this pattern2? my ...
2
votes
1answer
363 views

Getting data section with maximum element

I have some data stored in JSON-like format. { { value1: 14, value2: 12, value3: 1 }, { value1: 4, value3: -1 } } How can I get subsection ...
8
votes
4answers
3k views

Efficient in-place header removing for large files using sed?

The commands below may takes minutes depends on the file size. Is there any more effient method? sed -i 1d large_file
8
votes
6answers
1k views

Delete lines beginning with #

How do I delete lines beginning with a #, given that there can be whitespace on the left and right of the #? # Master socket provides access to userdb information. It's typically
1
vote
1answer
977 views

reference problem in using sed

I'm trying to use sed command to find/replace a pattern in file. It gives: sed: -e expression #1, char 27: invalid reference \1 on `s' command's RHS My command is: sed ...
4
votes
2answers
2k views

How to search and replace multiple needles by one word via one expression?

Assume you have a text file: foo fnord bar bizz foo poit And now I would want to replace both "foo" and "bar" into "narf". I know I could use: sed -e 's/foo/narf/g' -e 's/bar/narf/g' fileName ...
2
votes
4answers
590 views

Clean up concatenated XML files

I'm trying to fix my Delicious export script to work with the new "trick" of only serving 1000 bookmarks at a time. The script concatenates results (XML) from multiple calls in a single file, and now ...
5
votes
4answers
2k views

How to perform a sed in-place substitution that only creates backups of files that were changed?

I ran the following to replace a term used in all files in the current working directory: $ find . -type f -print0 | xargs -0 sed -i'.bup' -e's/Ms. Johnson/Mrs. Melbin/g' This performed the word ...
2
votes
2answers
375 views

Remove copyright symbol

I'm trying to parse a RSS feed on the command line. The code works so far but the feed contains a copyright symbol which I try to remove (it is latin1 encoded). How do I remove the copyright symbol ...
9
votes
1answer
252 views

How does this sed command work?

I've come across this solution for printing a specific line from a text file: sed '123!d;q' file Why doesn't sed quit after the first line of input in this case?
2
votes
1answer
402 views

Replace complex string in several files

A hacker got into a web server and added this string (removed some characters for security purposes and added line breaks for readability) in all index.php files: <?php ...
1
vote
2answers
384 views

pattern search and display the last occuring pattern?

I have a log file containing startup and shutdown times for everday. I want to see the last pattern pattern for startup and shutdown(which is located at the end of the file being updated everyday). ...
1
vote
4answers
2k views

Replacing XML entity values with positive lookaround

I'm trying to do some pretty simple sedding of a file: sed 's:(?<=<Type>)default(?=</Type>):super:' < myfile.xml The concept's pretty simple, using positive lookaround, find the ...
2
votes
2answers
587 views

Adding 2 lines in text file stanza

I have a text file divided in stanzas. Each stanza starts with a [keyword] and ends with a blank line. Something like this: [keyword1] param1 = value1 param2 = value2 param3 = value3 [keyword2] ...
7
votes
1answer
14k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
2
votes
1answer
672 views

Using sed once to search/replace different strings

I'm using sed to find configuration directives in a php file. Essentially, I need to set the strings database_name_here, username_here and password_here with their appropriate values in the ...
5
votes
3answers
966 views

Regex & Sed/Perl: Match word that ISN'T preceded by another word

I'd like to use sed or perl to replace all occurrences of a word that doesn't have a certain word in front of it. For example, I have a text file that contains a plot of a movie and I want to replace ...
7
votes
2answers
3k views

How to insert a new line after every second line?

sed/bash/wharever :D INPUT: sadf asdf yxcv cxv eqrt asdf OUTPUT: sadf asdf yxcv cxv eqrt asdf
5
votes
2answers
413 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...
2
votes
2answers
3k views

Strip trailing whitespace from files

The answer to removing trailing whitespace with sed has most of the answer, but I want sed -i 's/[ \t]*$//' $1 to be able to take arbitrary number of file arguments as a shell script on the command ...
3
votes
2answers
930 views

simple command to strip header and footer from a file

I want a simple little command to strip an XML-Header and Footer from a file: <?xml version="1.0" encoding="UTF-8"?> <conxml> <MsgPain001> ...