sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

182
votes
5answers
370k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
75
votes
3answers
120k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
67
votes
7answers
28k views

Is there a basic tutorial for grep, awk and sed? [closed]

I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
63
votes
7answers
92k views

How can I use sed to replace a multi-line string?

I've noticed that, if I add \n to a pattern for substituting using sed, it does not match. Example: $ cat > alpha.txt This is a test Please do not be alarmed $ sed -i'.original' 's/a test\nPlease ...
38
votes
7answers
84k views

Delete First line of a file

How can I delete the first line of a file and keep the changes? I tried this but it erases the whole content of the file. $sed 1d file.txt > file.txt
37
votes
4answers
5k views

Reverse grepping

Let's say, I have a really big text file (about 10.000.000 lines). I need to grep it from the end and save result to a file. What's the most efficient way to accomplish task?
29
votes
2answers
92k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
27
votes
9answers
11k views

Addition with 'sed'

I am trying to perform a mathematical operation with sed, but it continues to treat my variables as strings. The input is of this kind: $ echo 12 | sed 's/[0-9]*/&+3/' $ 12+3 I'd like to have ...
23
votes
9answers
4k views

How to delete every second line from a file?

File: Data inserted into table. Total count 13 No error occurred Data inserted into table. Total count 45 No error occurred Data inserted into table. Total count 14 No error occurred Data inserted ...
23
votes
10answers
10k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
22
votes
6answers
50k views

SED: insert something to the last line?

The command inserts a tag to the beginning of a file: sed -i "1s/^/<?php /" file How can I insert something to the end of each file with SED?
22
votes
6answers
58k views

How to parse JSON with shell scripting in Linux?

I have a JSON output from which I need to extract a few parameters in Linux. This is the JSON output: { "OwnerId": "121456789127", "ReservationId": "r-48465168", "Groups": ...
22
votes
2answers
8k views

Differences between sed on Mac OSX and other “standard” sed?

I am having some issues in using an answer provided on this site for this question about a sed command to replace a blank line with two other lines of content, and it was brought up if the sed command ...
21
votes
5answers
72k views

How to strip multipe spaces to one using sed?

sed on AIX is not doing what I think it should. I'm trying to replace multiple spaces with a single space in the output of IOSTAT: # iostat System configuration: lcpu=4 drives=8 paths=2 vdisks=0 ...
20
votes
11answers
18k views

Decoding URL encoding (percent encoding)

I want to decode URL encoding, is there any built-in tool for doing this or could anyone provide me with a sed code that will do this? I did search a bit through unix.stackexchange.com and on the ...
20
votes
2answers
160k views

Using 'sed' to find and replace [duplicate]

I know this question has probably been answered before. I have seen many threads about this in various places, but the answers are usually hard to extract for me. I am looking for help with an ...
20
votes
6answers
3k views

How to achieve portability with sed -i (in-place editing)?

I'm writing shell scripts for my server, which is a shared hosting running FreeBSD. I also want to be able to test them locally, on my PC running Linux. Hence, I'm trying to write them in a portable ...
19
votes
11answers
26k views

How to print all lines after a match up to the end of the file?

Input file1 is: dog 123 4335 cat 13123 23424 deer 2131 213132 bear 2313 21313 I give the match the pattern from in other file ( like dog 123 4335 from file2). I match the pattern of the line ...
19
votes
5answers
35k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
18
votes
11answers
4k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
18
votes
4answers
34k views

Delete last line from the file

I use sed to quickly delete lines with specific position as sed '1d' sed '5d' But, what if I want to delete the last line of the file and I don't know the count of lines (I know I can get that ...
18
votes
7answers
43k views

How to read first and last line from cat output?

I have text file. Task - get first and last line from file after $ cat file | grep -E "1|2|3|4" | commandtoprint $ cat file 1 2 3 4 5 Need this without cat output (only 1 and 5). ~$ cat file | ...
18
votes
1answer
1k views

How does the command sed '1!G;h;$!d' reverse the contents of a file?

My question is related to the sed-specific solution given in this answer for this question of reverse grepping. The sed/grep solution that I am unable to decipher is the following one: sed ...
15
votes
5answers
14k views

Convert file contents to lower case

I have temp file with some lower-case and upper-case contents. Input Contents of my temp file: hi Jigar GANDHI jiga I want to convert all upper to lower. Command I tried the following command: ...
15
votes
7answers
2k views

How can I prepend a tag to the beginning of several files?

I need to add PHP tags surrounding a file. It's easy to append them using find . -exec echo "?>" >> '{}' \; but how can I prepend the tag <?php?
15
votes
4answers
2k views

Is there an alternative to sed that supports unicode?

For example: sed 's/\u0091//g' file1 Right now, I have to do hexdump to get hex number and put into sed as follows: $ echo -ne '\u9991' | hexdump -C 00000000 e9 a6 91 ...
14
votes
5answers
32k views

Remove line containing certain string and the following line

I use this cat foo.txt | sed '/bar/d' to remove lines containing the string bar in the file. I would like however to remove those lines and the line directly after it. Preferably in sed, awk or ...
14
votes
3answers
5k views

The way to use `/usr/bin/env sed -f ` in shebang?

Typing /usr/bin/env sed -f in terminal works. But if use it as a shebang, #!/usr/bin/env sed -f s/a/b/ The script will be fail to execute: /usr/bin/env: sed -f: No such file or directory I ...
14
votes
5answers
30k views

Multiline pattern match using sed, awk or grep

Is it possible to do a multiline pattern match using sed, awk or grep? Take for example, I would like to get all the lines between { and } So it should be able to match 1. {} 2. {.....} 3. ...
14
votes
4answers
4k views

Replacing Multiple blank lines with a single blank line in vim / sed

Question more or less says it all. I'm aware that /^$/d will remove all blank lines, but I can't see how to say 'replace two or more blank lines with a single blank line' Any ideas?
14
votes
1answer
30k views

Only return the matched string in sed [duplicate]

Possible Duplicate: Extracting a regex matched with 'sed' without printing the surrounding characters How do I make this only print test: echo "atestb" | sed -n 's/\(test\)/\1/p'
14
votes
2answers
4k views

Make sed ask for confirmation before each replacement?

Is there a way to make sed ask me for confirmation before each replace? Something similar to 'c' when using replace inside vim. Does sed do this at all?
13
votes
9answers
631 views

Find all occurances in a file with sed

Using OPEN STEP 4.2 OS... I am currently using the following sed Command: sed -n '1,/141.299.99.1/p' TESTFILE | tail -3 This command will find one instance in a file with the ip of 141.299.99.1 and ...
13
votes
8answers
5k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
13
votes
2answers
55k views

How to insert text after a certain string in a file?

Right now I'm using echo "Hello World" >> file.txt to append some text to a file but I also need to add text below a certain string let's say "[option]", is it possible with sed? EG: Input ...
13
votes
7answers
21k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
13
votes
8answers
3k views

How to perform a multi line grep

How would you perform a grep for text that appears on two lines? For example: pbsnodes is a command I use that returns the utilization of a linux cluster root$ pbsnodes node1 state = free ...
13
votes
8answers
7k views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
13
votes
3answers
12k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
13
votes
4answers
26k views

Text between two tags

I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
13
votes
3answers
16k views

Replace string with contents of a file using sed

I have two different files: File1 /home/user1/ /home/user2/bin /home/user1/a/b/c File2 <TEXT1> <TEXT2> I want to replace the <TEXT1> of File2 with the contents of File1 ...
13
votes
1answer
26k views

Replacing string based on line number

I have a situation where i want to replace a particular string in many files Replace a string AAA with another string BBB but there are lot of strings starting with AAA or ending in AAA ,and i want ...
13
votes
1answer
10k views

Substitute text with sed and keep part of the original text

I am trying to convert <id>1</id> <Name>ENTERPRISE RESOURCE PLANNING</Name> to: <column name="id">1</column> <column name="Name">ENTERPRISE RESOURCE ...
12
votes
9answers
2k views

How can I delete the 5th word of every line in a file?

I want to delete the 5th word of each line in a file. The current content of the file: File is not updated or and will be removed System will shut down f within 10 seconds Please save your work ...
12
votes
6answers
2k views

How to delete the last column of a file in Linux

I want to delete the last column of a txt file, while I do not know what the column number is. How could I do this? Example: Input: 1223 1234 1323 ... 2222 123 1233 1234 1233 ... 3444 125 0000 5553 ...
12
votes
3answers
10k views

What is the purpose of -e in sed command?

I can't find any documentation about the sed -e switch, for simple replace, do I need it? e.g. sed 's/foo/bar/' VS sed -e 's/foo/bar/'
12
votes
3answers
8k views

Zero-fill numbers to 2 digits with sed

Input: 201103 1 /mnt/hdd/PUB/SOMETHING 201102 7 /mnt/hdd/PUB/SOMETH ING 201103 11 /mnt/hdd/PUB/SO METHING 201104 3 /mnt/hdd/PUB/SOMET HING 201106 1 /mnt/hdd/PUB/SOMETHI NG Desired output: 201103 ...
12
votes
4answers
2k views

How to report “sed” in-place changes

When using sed to replace strings in-place, is there a way to make it report the changes it does (without relying on a diff of old and new files)? For instance, how can I change the command line ...
11
votes
2answers
23k views

How to replace a string with a string containing slash with sed?

i am looking for a way to replace a string in a file with a string that contains a slash by using sed. connect="192.168.100.61/foo" srcText="foo.bar=XPLACEHOLDERX" echo $srcText | sed ...
11
votes
7answers
11k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" ...