sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

70
votes
2answers
133k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
62
votes
7answers
20k views

Is there a basic tutorial for grep, awk and sed? [closed]

I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
40
votes
6answers
53k views

How can I use sed to replace a multi-line string?

I've noticed that, if I add \n to a pattern for substituting using sed, it does not match. Example: $ cat > alpha.txt This is a test Please do not be alarmed $ sed -i'.original' 's/a test\nPlease ...
40
votes
3answers
77k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
36
votes
4answers
3k views

Reverse grepping

Let's say, I have a really big text file (about 10.000.000 lines). I need to grep it from the end and save result to a file. What's the most efficient way to accomplish task?
25
votes
9answers
8k views

Addition with 'sed'

I am trying to perform a mathematical operation with sed, but it continues to treat my variables as strings. The input is of this kind: $ echo 12 | sed 's/[0-9]*/&+3/' $ 12+3 I'd like to have ...
22
votes
10answers
7k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
17
votes
6answers
31k views

SED: insert something to the last line?

The command inserts a tag to the beginning of a file: sed -i "1s/^/<?php /" file How can I insert something to the end of each file with SED?
16
votes
5answers
27k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
15
votes
5answers
40k views

Delete First line of a file

How can I delete the first line of a file and keep the changes? I tried this but it erases the whole content of the file. $sed 1d file.txt > file.txt
15
votes
4answers
21k views

Delete last line from the file

I use sed to quickly delete lines with specific position as sed '1d' sed '5d' But, what if I want to delete the last line of the file and I don't know the count of lines (I know I can get that ...
15
votes
6answers
2k views

How to achieve portability with sed -i (in-place editing)?

I'm writing shell scripts for my server, which is a shared hosting running FreeBSD. I also want to be able to test them locally, on my PC running Linux. Hence, I'm trying to write them in a portable ...
15
votes
2answers
5k views

Differences between sed on Mac OSX and other “standard” sed?

I am having some issues in using an answer provided on this site for this question about a sed command to replace a blank line with two other lines of content, and it was brought up if the sed command ...
14
votes
10answers
3k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
14
votes
7answers
2k views

How can I prepend a tag to the beginning of several files?

I need to add PHP tags surrounding a file. It's easy to append them using find . -exec echo "?>" >> '{}' \; but how can I prepend the tag <?php?
14
votes
6answers
9k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
13
votes
9answers
528 views

Find all occurances in a file with sed

Using OPEN STEP 4.2 OS... I am currently using the following sed Command: sed -n '1,/141.299.99.1/p' TESTFILE | tail -3 This command will find one instance in a file with the ip of 141.299.99.1 and ...
13
votes
2answers
58k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
13
votes
5answers
48k views

How to strip multipe spaces to one using sed?

sed on AIX is not doing what I think it should. I'm trying to replace multiple spaces with a single space in the output of IOSTAT: # iostat System configuration: lcpu=4 drives=8 paths=2 vdisks=0 ...
13
votes
4answers
858 views

Is there an alternative to sed that supports unicode?

For example: sed 's/\u0091//g' file1 Right now, I have to do hexdump to get hex number and put into sed as follows: $ echo -ne '\u9991' | hexdump -C 00000000 e9 a6 91 ...
13
votes
8answers
4k views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
12
votes
4answers
22k views

Remove line containing certain string and the following line

I use this cat foo.txt | sed '/bar/d' to remove lines containing the string 'bar' in the file. I would like however to remove those lines and the line directly after it. Preferably in sed, awk or ...
12
votes
5answers
15k views

Remove comma between the quotes only in a comma delimited file

I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row 123,"ABC, DEV 23",345,534.202,NAME I need to ...
12
votes
5answers
27k views

How to parse JSON with shell scripting in Linux?

I have a JSON output from which I need to extract a few parameters in Linux. This is the JSON output: { "OwnerId": "121456789127", "ReservationId": "r-48465168", "Groups": ...
12
votes
8answers
3k views

How to perform a multi line grep

How would you perform a grep for text that appears on two lines? For example: pbsnodes is a command I use that returns the utilization of a linux cluster root$ pbsnodes node1 state = free ...
12
votes
3answers
8k views

Delete the first n bytes of files

I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way. I want to delete the first 31 bytes of each file ...
12
votes
4answers
3k views

Replacing Multiple blank lines with a single blank line in vim / sed

Question more or less says it all. I'm aware that /^$/d will remove all blank lines, but I can't see how to say 'replace two or more blank lines with a single blank line' Any ideas?
11
votes
8answers
3k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...
11
votes
3answers
5k views

Zero-fill numbers to 2 digits with sed

Input: 201103 1 /mnt/hdd/PUB/SOMETHING 201102 7 /mnt/hdd/PUB/SOMETH ING 201103 11 /mnt/hdd/PUB/SO METHING 201104 3 /mnt/hdd/PUB/SOMET HING 201106 1 /mnt/hdd/PUB/SOMETHI NG Desired output: 201103 ...
11
votes
3answers
12k views

Replace string with contents of a file using sed

I have two different files: File1 /home/user1/ /home/user2/bin /home/user1/a/b/c File2 <TEXT1> <TEXT2> I want to replace the <TEXT1> of File2 with the contents of File1 ...
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...
10
votes
3answers
1k views

How do I replace the last occurrence of a character in a string using sed?

How do I replace only the last occurrence of "-" in a string with a space using sed? For example: echo $MASTER_DISK_RELEASE swp-RedHat-Linux-OS-5.5.0.0-03 but I want to get the following output ( ...
10
votes
3answers
6k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...
10
votes
4answers
7k views

How to insert a new line after every second line?

sed/bash/wharever :D INPUT: sadf asdf yxcv cxv eqrt asdf OUTPUT: sadf asdf yxcv cxv eqrt asdf
10
votes
4answers
749 views

How to report “sed” in-place changes

When using sed to replace strings in-place, is there a way to make it report the changes it does (without relying on a diff of old and new files)? For instance, how can I change the command line ...
10
votes
2answers
2k views

Using sed to color the output from a command on solaris

I have a ksh script that must work on both linux and solaris. I'm trying to color the output of specific commands. It works on linux (specifically RHEL6), but not on solaris (SunOS 5.10). Command ...
9
votes
2answers
4k views

How to remove multiple lines per occurrence in a file?

Say I have this 857835 line file, containing stuff like this: a1 rubbish1 rubbish2 rubbish3 rubbish4 a1 rubbish5 rubbish6 rubbish7 rubbish8 And I wish to remove all occurences of a1 and the next ...
9
votes
5answers
1k views

SED : Case Match Pattern Replacement

I have a source code spread across several files. It has a pattern abcdef which I need to replace with pqrstuvxyz. The pattern could be Abcdef (Sentence Case) then it needs to be replaced with ...
9
votes
3answers
2k views

Substituting strings in a very large file

I have a very long series of urls with no separating character, in the same format as below: http://example.comhttp://example.nethttp://example.orghttp://etc... I want each URL to be on a new line. ...
9
votes
3answers
4k views

The way to use `/usr/bin/env sed -f ` in shebang?

Typing /usr/bin/env sed -f in terminal works. But if use it as a shebang, #!/usr/bin/env sed -f s/a/b/ The script will be fail to execute: /usr/bin/env: sed -f: No such file or directory I ...
9
votes
4answers
5k views

Efficient in-place header removing for large files using sed?

The commands below may takes minutes depends on the file size. Is there any more effient method? sed -i 1d large_file
9
votes
3answers
15k views

Extracting a regex matched with 'sed' without printing the surrounding characters

To all the 'sed' doctors out there: I have a seemingly trivial 'sed' question to which I have not been able to find a solution. How can you get 'sed' to exctract a regular expression it has matched ...
9
votes
2answers
325 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...
9
votes
1answer
271 views

How does this sed command work?

I've come across this solution for printing a specific line from a text file: sed '123!d;q' file Why doesn't sed quit after the first line of input in this case?
9
votes
2answers
5k views

Is there a way to detect null bytes (␀, NUL, \0) in sed?

Related to another question, in order to fuzzily detect binary files, is there a way to detect ␀ bytes in sed?
9
votes
1answer
6k views

Substitute text with sed and keep part of the original text

I am trying to convert <id>1</id> <Name>ENTERPRISE RESOURCE PLANNING</Name> to: <column name="id">1</column> <column name="Name">ENTERPRISE RESOURCE ...
9
votes
1answer
15k views

Only return the matched string in sed [duplicate]

Possible Duplicate: Extracting a regex matched with 'sed' without printing the surrounding characters How do I make this only print test: echo "atestb" | sed -n 's/\(test\)/\1/p'
9
votes
2answers
3k views

Make sed ask for confirmation before each replacement?

Is there a way to make sed ask me for confirmation before each replace? Something similar to 'c' when using replace inside vim. Does sed do this at all?
9
votes
4answers
182 views

Script to extract selected entries from a bibtex file

I have a large bibtex file with many entries where each entry has the general structure @ARTICLE{AuthorYear, item = {...}, item = {...}, item = {...}, etc } (in some cases ARTICLE might be a ...
8
votes
6answers
247 views

Grab certain contents of a file

So I know tools exist for this problem because I've heard about them, but I don't know what they are. I want to do something like filter out all data but the usernames in /etc/passwd. For example, I ...