Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

53
votes
6answers
13k views

Is there a basic tutorial for grep, awk and sed?

I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
22
votes
9answers
5k views

Addition with 'sed'

I am trying to perform a mathematical operation with sed, but it continues to treat my variables as strings. The input is of this kind: $ echo 12 | sed 's/[0-9]*/&+3/' $ 12+3 I'd like to have ...
21
votes
10answers
4k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
16
votes
3answers
36k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
14
votes
9answers
2k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
14
votes
4answers
18k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
13
votes
6answers
520 views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
12
votes
5answers
14k views

SED: insert something to the last line?

The command inserts a tag to the beginning of a file: sed -i "1s/^/<?php /" file How can I insert something to the end of each file with SED?
12
votes
7answers
1k views

How can I prepend a tag to the beginning of several files?

I need to add PHP tags surrounding a file. It's easy to append them using find . -exec echo "?>" >> '{}' \; but how can I prepend the tag <?php?
12
votes
4answers
16k views

How can I use sed to replace a multi-line string?

I've noticed that, if I add \n to a patter for a substitute using sed, it does not match. Example: $ cat > alpha.txt This is a test Please do not be alarmed $ sed -i'.original' 's/a test\nPlease ...
12
votes
8answers
1k views

How to perform a multi line grep

How would you perform a grep for text that appears on two lines? For example: pbsnodes is a command I use that returns the utilization of a linux cluster root$ pbsnodes node1 state = free ...
11
votes
4answers
8k views

Delete last line from the file

I use sed to quickly delete lines with specific position as sed '1d' sed '5d' But, what if I want to delete the last line of the file and I don't know the count of lines (I know I can get that ...
11
votes
5answers
401 views

How to achieve portability with sed -i (in-place editing)?

I'm writing shell scripts for my server, which is a shared hosting running FreeBSD. I also want to be able to test them locally, on my PC running Linux. Hence, I'm trying to write them in a portable ...
11
votes
6answers
5k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
11
votes
4answers
2k views

Replacing Multiple blank lines with a single blank line in vim / sed

Question more or less says it all. I'm aware that /^$/d will remove all blank lines, but I can't see how to say 'replace two or more blank lines with a single blank line' Any ideas?
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...
9
votes
8answers
1k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
9
votes
3answers
3k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
9
votes
2answers
3k views

How to remove multiple lines per occurrence in a file?

Say I have this 857835 line file, containing stuff like this: a1 rubbish1 rubbish2 rubbish3 rubbish4 a1 rubbish5 rubbish6 rubbish7 rubbish8 And I wish to remove all occurences of a1 and the next ...
9
votes
3answers
2k views

Substituting strings in a very large file

I have a very long series of urls with no separating character, in the same format as below: http://example.comhttp://example.nethttp://example.orghttp://etc... I want each URL to be on a new line. ...
9
votes
4answers
23k views

How to strip multipe spaces to one using sed?

sed on AIX is not doing what I think it should. I'm trying to replace multiple spaces with a single space in the output of IOSTAT: # iostat System configuration: lcpu=4 drives=8 paths=2 vdisks=0 ...
9
votes
1answer
256 views

How does this sed command work?

I've come across this solution for printing a specific line from a text file: sed '123!d;q' file Why doesn't sed quit after the first line of input in this case?
9
votes
3answers
5k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
9
votes
2answers
2k views

Differences between sed on Mac OSX and other “standard” sed?

I am having some issues in using an answer provided on this site for this question about a sed command to replace a blank line with two other lines of content, and it was brought up if the sed command ...
9
votes
2answers
1k views

Using sed to color the output from a command on solaris

I have a ksh script that must work on both linux and solaris. I'm trying to color the output of specific commands. It works on linux (specifically RHEL6), but not on solaris (SunOS 5.10). Command ...
9
votes
4answers
116 views

Script to extract selected entries from a bibtex file

I have a large bibtex file with many entries where each entry has the general structure @ARTICLE{AuthorYear, item = {...}, item = {...}, item = {...}, etc } (in some cases ARTICLE might be a ...
8
votes
6answers
230 views

Grab certain contents of a file

So I know tools exist for this problem because I've heard about them, but I don't know what they are. I want to do something like filter out all data but the usernames in /etc/passwd. For example, I ...
8
votes
4answers
872 views

How do I write a sed one-liner to add a character after every third character?

So, I have a string that looks like this: AUGGCCAUGGCGCCCAGAACUGAGAUCAAUAGUACCCGUAUUAACGGGUGA And I want to split the string into 3-character chunks delimited by a '+' sign. ...
8
votes
4answers
10k views

Remove line containing certain string and the following line

I use this cat foo.txt | sed '/bar/d' to remove lines containing the string 'bar' in the file. I would like however to remove those lines and the line directly after it. Preferably in sed, awk or ...
8
votes
5answers
8k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
8
votes
6answers
1k views

Delete lines beginning with #

How do I delete lines beginning with a #, given that there can be whitespace on the left and right of the #? # Master socket provides access to userdb information. It's typically
8
votes
2answers
8k views

How to replace a string with a string containing slash with sed?

i am looking for a way to replace a string in a file with a string that contains a slash by using sed. connect="192.168.100.61/foo" srcText="foo.bar=XPLACEHOLDERX" echo $srcText | sed ...
8
votes
4answers
3k views

Efficient in-place header removing for large files using sed?

The commands below may takes minutes depends on the file size. Is there any more effient method? sed -i 1d large_file
8
votes
3answers
3k views

Zero-fill numbers to 2 digits with sed

Input: 201103 1 /mnt/hdd/PUB/SOMETHING 201102 7 /mnt/hdd/PUB/SOMETH ING 201103 11 /mnt/hdd/PUB/SO METHING 201104 3 /mnt/hdd/PUB/SOMET HING 201106 1 /mnt/hdd/PUB/SOMETHI NG Desired output: 201103 ...
8
votes
4answers
2k views

How to delete line if longer than XY?

How can i delete a line if it is longer than e.g.: 2048 chars?
8
votes
4answers
7k views

grep — removing text after delimiter token

I have a file in which I need to eliminate everything after the first ; on every line. So a file like this: sdfsdsdf; fsdfsddf;sdfsd; Will result in this: sdfsdsdf fsdfsddf I have looked into ...
8
votes
1answer
21k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
8
votes
2answers
4k views

How to insert a new line after every second line?

sed/bash/wharever :D INPUT: sadf asdf yxcv cxv eqrt asdf OUTPUT: sadf asdf yxcv cxv eqrt asdf
8
votes
6answers
352 views

How to get multiple lines out of a file by a regex?

How to get multiple lines out of a file by a regex? I often would like to get multiple lines / modify multiple lines by a regex. A example case: I am trying to read part of an XML/SGML file (they ...
8
votes
2answers
3k views

Is there a way to detect null bytes (␀, NUL, \0) in sed?

Related to another question, in order to fuzzily detect binary files, is there a way to detect ␀ bytes in sed?
8
votes
2answers
8k views

Replace string with contents of a file using sed

I have two different files: File1 /home/user1/ /home/user2/bin /home/user1/a/b/c File2 <TEXT1> <TEXT2> I want to replace the <TEXT1> of File2 with the contents of File1 ...
8
votes
3answers
663 views

Deleting all C comments with sed

I am trying to write a script that will delete all comments and everything in between inside C files in my current directory. I've been using sed, and this is what I have so far: sed -i '/ * [^()] ...
8
votes
4answers
4k views

Remove files older than 5 days in UNIX (date in file name, not timestamp)

I want to delete log files which are older than 5 days from a directory. But deletion should not be based on the timestamp of file. It should be based on the name of file. For Example todays date is ...
7
votes
7answers
349 views

Swapping an unlimited number of columns

I have a file with columns. See below for an example: a b c ... z 1 2 3 ... 26 I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc.. z y x ...
7
votes
10answers
1k views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
7
votes
3answers
611 views

How to remove a specific character in a string but only if there are no numbers in that line in Linux

I've been stumped with this seemingly simple-to-fix problem.. well, for a while. Here's example output from the file I need to edit: $cat file George Washington Geneva Convention 123,281,029 USD ...
7
votes
5answers
332 views

SED : Case Match Pattern Replacement

I have a source code spread across several files. It has a pattern abcdef which I need to replace with pqrstuvxyz. The pattern could be Abcdef (Sentence Case) then it needs to be replaced with ...
7
votes
3answers
2k views

How can I instruct BSD sed to interpret escape sequences like \n and \t?

I have a sed replacement command that I would like to be compatible with BSD sed as well as GNU sed. Extended regular expressions are not an issue as I do not need them in this case. My primary ...
7
votes
2answers
7k views

Extracting a regex matched with 'sed' without printing the surrounding characters

To all the 'sed' doctors out there: I have a seemingly trivial 'sed' question to which I have not been able to find a solution. How can you get 'sed' to exctract a regular expression it has matched ...
7
votes
1answer
302 views

Change date “July 29th, 2011” to “20110729”

I have lots of HTML files contains date with format July 29th, 2011 I want to change date format July 29th, 2011 to 20110729, December 9th, 2010 to 20101209, etc. I think sed may works but still ...