sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

237
votes
6answers
456k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
6
votes
1answer
826 views

How to ensure that string interpolated into `sed` substitution escapes all metachars

I have a script that reads a text stream and generates a file of sed commands that is later run with sed -f. The generated sed commands are like: ...
12
votes
3answers
27k views

Extracting a regex matched with 'sed' without printing the surrounding characters

To all the 'sed' doctors out there: How can you get 'sed' to extract a regular expression it has matched in a line? In other words words, I want just the string corresponding to the regular ...
6
votes
4answers
565 views

Search for a string and print everything before and after within a range

I have this file: sometext1{ string1 } sometext2{ string2 string3 } sometext3{ string4 string5 string6 } I want to search this file for a specific string and print everything before this string ...
24
votes
7answers
4k views

How to achieve portability with sed -i (in-place editing)?

I'm writing shell scripts for my server, which is a shared hosting running FreeBSD. I also want to be able to test them locally, on my PC running Linux. Hence, I'm trying to write them in a portable ...
15
votes
7answers
2k views

How can I prepend a tag to the beginning of several files?

I need to add PHP tags surrounding a file. It's easy to append them using find . -exec echo "?>" >> '{}' \; but how can I prepend the tag <?php?
14
votes
7answers
24k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
5
votes
3answers
3k views

Insert text from file inline after matching pattern in another file

I'm trying to take the contents of a file and insert it after a matching pattern in another file using sed. My question is very similar to this question, but I wish to insert the contents of a file ...
87
votes
4answers
135k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
74
votes
8answers
106k views

How can I use sed to replace a multi-line string?

I've noticed that, if I add \n to a pattern for substituting using sed, it does not match. Example: $ cat > alpha.txt This is a test Please do not be alarmed $ sed -i'.original' 's/a test\nPlease ...
13
votes
4answers
29k views

Text between two tags

I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
13
votes
3answers
17k views

Replace string with contents of a file using sed

I have two different files: File1 /home/user1/ /home/user2/bin /home/user1/a/b/c File2 <TEXT1> <TEXT2> I want to replace the <TEXT1> of File2 with the contents of File1 ...
6
votes
2answers
9k views

Show only text between 2 matching pattern

I'm going to submit form using cURL, where some of the contents is come from other file, selected using sed If param1 is line matching pattern from other file using sed, below command will works ...
19
votes
5answers
37k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
8
votes
2answers
15k views

sed one-liner to delete everything between a pair of brackets?

I am working with some text that is full of stuff between brackets [] that I don't want. Since I can delete the brackets myself, I don't need the one-liner to do that for me, but I do need a one-liner ...
19
votes
4answers
39k views

Delete last line from the file

I use sed to quickly delete lines with specific position as sed '1d' sed '5d' But, what if I want to delete the last line of the file and I don't know the count of lines (I know I can get that ...
13
votes
7answers
13k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" ...
7
votes
3answers
3k views

How do I stop sed from adding extra newline characters

I am running the following 2 sed commands. The first one adds newline characters where I want them, the second also adds newline characters where I want them, BUT also adds an extra one at the end of ...
6
votes
3answers
407 views

How to print a line if that line or the next line do not contain a particular string

Input.txt: 8B0C remove 8B0D remove 8B0E remove 8B0F 8B10 remove 8B14 remove 8B15 remove 8B16 remove 8B17 remove 8AC0 8AC1 ...
6
votes
1answer
4k views

How to grep all lines where starting line ends with a pattern and ending line ends with another?

I want to start grepping from the lines that has CK at the end of line and stop grepping when the line has D at the end. I tried grep "$CK" "$D" file..txt, but it didn't work. Input: kkkkkkkkkkk ...
7
votes
6answers
46k views

How can I use variables when doing a sed?

I want to do: cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh in my program. But I want to use variables, e.g. old_run='old_name_952' new_run='old_name_953' I have ...
6
votes
1answer
243 views

How can I extract/change lines in a text file whose data are separated into fields?

How can I manipulate field-based data from the commandline? For example How can I print only lines whose Nth field is foo? How can I print only lines whose Nth field isn't foo? How can I print only ...
4
votes
2answers
890 views

Why do I need to escape regex characters in sed to be interpreted as regex characters?

It seems e.g. cat sed_data.txt | sed 's/\b[0-9]\{3\}\b/NUMBER/g' that I must escape characters to form a regular expression. In this case I had to escape braces in order to be interpreted as a number ...
9
votes
6answers
41k views

Can sed replace new line characters?

Is there an issue with sed and new line character? I have a file test.txt with the following contents aaaaa bbbbb ccccc ddddd The following does not work: sed -r -i 's/\n/,/g' test.txt ...
2
votes
3answers
9k views

Using a reference to a bash string variable in sed

I'm a relative Linux novice. Suppose that I have a text file a.txt that contains the following text: A B C Now, if I want to change the text on line 2 (which contains B), I can use the following ...
15
votes
5answers
32k views

Multiline pattern match using sed, awk or grep

Is it possible to do a multiline pattern match using sed, awk or grep? Take for example, I would like to get all the lines between { and } So it should be able to match 1. {} 2. {.....} 3. ...
27
votes
6answers
58k views

SED: insert something to the last line?

The command inserts a tag to the beginning of a file: sed -i "1s/^/<?php /" file How can I insert something to the end of each file with SED?
27
votes
12answers
22k views

Decoding URL encoding (percent encoding)

I want to decode URL encoding, is there any built-in tool for doing this or could anyone provide me with a sed code that will do this? I did search a bit through unix.stackexchange.com and on the ...
24
votes
6answers
70k views

How to parse JSON with shell scripting in Linux?

I have a JSON output from which I need to extract a few parameters in Linux. This is the JSON output: { "OwnerId": "121456789127", "ReservationId": "r-48465168", "Groups": ...
12
votes
4answers
10k views

sed on OSX insert at a certain line

So I've been using 'sed' on linux for a while, but have had a bit of difficulty trying to use it on OSX since 'POSIX sed' and 'GNU sed' have so many little differences. Currently I'm struggling with ...
13
votes
3answers
11k views

What is the purpose of -e in sed command?

I can't find any documentation about the sed -e switch, for simple replace, do I need it? e.g. sed 's/foo/bar/' VS sed -e 's/foo/bar/'
6
votes
6answers
2k views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
6
votes
5answers
7k views

How to get all lines between first and last occurrences of patterns?

How can I trim a file (well input stream) so that I only get the lines ranging from the first occurrence of pattern foo to the last occurrence of pattern bar? For instance consider the following ...
3
votes
4answers
1k views

How to use sed command to modify a file's entry line by line in shell script?

This is my file1: #$BQ #{ VOL @home } #database daba #relation tcdeatid #copy 1 #{ version 0 } #opendb #clear #.lruno := 72 #.infno := 1 #.tid.noel := 101 #.tid.info ...
3
votes
2answers
5k views

Comparing two files using Unix and Awk

I need to compare two files, File1 and File2 (Separated by space) using 4 fields (Field 1, 2, 4 and 5 of File 1 with field1, 2, 4 and 5 of File2). Logic: If column 1, 2 and 4 of File 1 matches with ...
10
votes
8answers
601 views

sed in-place line deletion on full filesystem?

Due to an application bug as yet undiagnosed, I have several hundred servers with a full disk. There is one file that has been filled up with duplicate lines—not a log file, but a user environment ...
4
votes
5answers
619 views

How to replace only the Nth occurrence of a pattern in a file?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...
4
votes
3answers
705 views

extract from line to line and then save to separate file

I tried my luck with grep and sed but somehow I don't manage to get it right. I have a log file which is about 8 GB in size. I need to analyze a 15 minute time period of suspicious activity. I ...
4
votes
3answers
2k views

Show lines matching a pattern and the 4 lines before each

For example, from this file: CREATE SYNONYM I801XS07 FOR I8010.I801XT07 * ERROR at line 1: ORA-00955: name is already used by an existing object CREATE SYNONYM I801XS07 FOR ...
3
votes
2answers
3k views

Find the line number which contains the pattern using custom regex delimiter

As in example I'm trying to get line numbers which contains the pattern. My pattern contains slashes so I wanted to add custom delimiter. This simple one works: sed -n '/file/=' temp.txt Using ...
1
vote
1answer
171 views

How to nest global matches with sed?

If I do: sed 's/match/replace/g' I know that sed will substitute replace for every occurrence of match on a line. But what if...? echo "match <please dont match this?>" | sed ...
10
votes
1answer
362 views

regarding portable sed -e… d b or ! b?

In this edit Stéphane Chazelas POSIXifies (again) my sed formatting by inserting an -expression break and another -expression statement. Now, I might just ask him why in the comments, I suppose, but ...
6
votes
1answer
324 views

Variable in sed

I have a file that contains filenames with absolute path. In the middle of that path, is a variable directory, whenever I hard code the variable part, it works fine, but when I replace it with the ...
5
votes
4answers
871 views

Why does sed act differently depending on the output file?

If I run: cat messages.txt | sed -e 's/a/a/g' > messages.txt on one large file (2500+ lines) I find that the resulting file will only have about 900 lines after the command in cygwin and will ...
4
votes
3answers
621 views

Removal of lines with no more or fewer than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
2
votes
2answers
2k views

How do I move a line in a text file up or down by one line?

I have some text files, and I'd like to be able to move an arbitrary line in any of the files up or down one line (lines at the beginning or end of the file would stay where they are). I have some ...
2
votes
2answers
92 views

Search and Replace question

I need to find all occurrences of AAsomeArbitraryStringBB and replace it with CCsomeArbitraryStringDD. So AAHelloBB Text AAByeByeBB becomes CCHelloDD Text CCByeByeDD. It's important to note ...
1
vote
3answers
2k views

Replace specified character between two strings?

i have something to accomplish. i need to replace all occurrence of & inside or between <ex> </ex> to #. actual example below: a & b & c <ex> a & b & c ...
69
votes
7answers
30k views

Is there a basic tutorial for grep, awk and sed? [closed]

I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
30
votes
2answers
103k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...