sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

6
votes
2answers
3k views

Remove unknown unicode character from textfiles - sed, other bash/shell methods

I need to search and replace all occurences of an unknown character in some files having the same name. Opening such files with vi, I read <91> code for that character. Opening them with nano, I ...
6
votes
2answers
239 views

Regex `/pattern/g` and ed `:g/pattern/`: which came first, and why `g`?

The g option (e.g. s/pattern/replacement/g) for many tools that use regex-style pattern matching and the :g command in ed, ex, vi, and vim have pretty similar usage and meaning: match the given regex ...
6
votes
2answers
265 views

Grepping for a block of text with parts that can be optional

I have multiple entries that describes an event in a very large log file, say A.log. I would like to do two things with the event entries in the log file: Count the number of occurrences of each ...
6
votes
3answers
1k views

Multi-line replace

I spent ages trying to figure this out. As it happens, I don't need to do it any more; I found another way. But for the sake of my sanity, I'd like to know the answer anyway. It's quite easy to use ...
6
votes
3answers
112 views

How to use sed to manipulate continuously streaming output?

I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight ...
6
votes
3answers
1k views

How do I stop sed from adding extra newline characters

I am running the following 2 sed commands. The first one adds newline characters where I want them, the second also adds newline characters where I want them, BUT also adds an extra one at the end of ...
6
votes
2answers
178 views

How come sed is so fast?

I thought about whether this question is suitable for SE or not, I hope you agree it is. Some time ago I asked on SE how to find text in files and leave the file with only the matching lines that ...
5
votes
4answers
1k views

How to append end of every line with the line itself

This is in file.txt: redcar bluecar greencar Im looking for ways to make it become: redcar redcar bluecar bluecar greencar greencar I've tried many ways using sed with no luck
5
votes
8answers
450 views

How do I delete the first n lines and last line of a file using shell commands?

I have a file named Element_query containing the result of a query : SQL> select count (*) from element; [Output of the query which I want to keep in my file] SQL> spool off; I want to ...
5
votes
6answers
415 views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
5
votes
5answers
408 views

How to replace a word with new line

I have a text file with following data and each row ends with |END|. T|somthing|something|END|T|something2|something2|END| I am tryig to replace |END| with \n new line with sed. sed ...
5
votes
5answers
1k views

Sed: extending a string of numbers to a certain number of digits by padding with zeros

I'm currently trying to modify an echoed string of alphanumeric characters by piping it to sed. The area I'm interested in modifying is a block of numbers, which I would like to make 8 characters long ...
5
votes
4answers
444 views

Remove next-to-last extension in a file name

Suppose a folder with many files with bad extensions,for example Song1.avi.mp3 Song2.avi.mp3 Song32.web.mp3 Song23.mp4.mp3 Song2a.mp9.mp3 I want to remove only the second field (web, avi, mp4). I ...
5
votes
4answers
484 views

Reduce the over use of files in text processing

Currently, I have a file (Results.txt) which has 27k rows. The file content is as below. 45008657 | A. | | Long | 0 49210987 | A. | ...
5
votes
3answers
681 views

Why does sed act differently depending on the output file?

If I run: cat messages.txt | sed -e 's/a/a/g' > messages.txt on one large file (2500+ lines) I find that the resulting file will only have about 900 lines after the command in cygwin and will ...
5
votes
7answers
1k views

Replace a character except last x occurences

I have a file that has a bunch of hostnames correlated with IPs that looks like this: x-cluster-front-1 192.168.1.2 x-cluster-front-2 192.158.1.10 y-cluster-back-1 10.1.11.99 y-cluster-back-2 ...
5
votes
2answers
25k views

Using sed to get rid of characters < > ,

I have a file with lines as follows: ... ... <230948203[234]>, ... ... <234[24]>, ... .. I would like to use sed to remove the characters < , and > from every line I tried using ...
5
votes
5answers
251 views

Converting long numbers to hex with sed

I'm trying to make a sed command so that numbers longer than 3 digits were converted to hex. I.e. a string like 124 3275 7535 should result in 124 0xccb 0x1d6f. Here's what I currently have: sed ...
5
votes
3answers
1k views

Deleting Lines if there is a dot in the 5th Column

I have a file and I want to remove all the line if it has a dot in the 5th Column. Input File: sc2/80 20 . A C 80 PASS N=2 F=5;U=4 sc2/60 ...
5
votes
3answers
14k views

extract part of string using sed

ls lib/oracle-11.2.0.3.0.txt | sed 's/lib.\([oracle.*]\)\.txt/\1/' It is giving the whole string instead of just oracle part until .txt What am I doing wrong? I can do it using awk as follows, but, ...
5
votes
6answers
26k views

Using sed to find and replace complex string (preferrably with regex)

I have a file with the following contents: <username><![CDATA[name]]></username> <password><![CDATA[password]]></password> ...
5
votes
2answers
334 views

How is the inplace modification of a file being done?

What does the "inplace" modification of a file e.g. via sed -i or perl -i mean? My question is about how this inplace modification is done. Is the file copied the modification is done in the copy and ...
5
votes
2answers
518 views

Does `sed` exist in all *nixes?

My application will be running on all versions of Unix-based OS, say, Linux, MacOS, HP-UX, Sun Solaris , OSX, etc. I'm using the following sed command in a common place to delete the string matched ...
5
votes
3answers
701 views

Is it possible to match multiple specific line numbers (not range) with sed?

Consider: echo -e "a\nb\nc\nd\ne\nf\ng\nh" | sed '3,5a test' This will match lines 3, 4 and 5. But I am trying to match only lines 3 and 5 (not 4). And append 'test' after them.
5
votes
2answers
328 views

sed + how to delete the second character “.” from the line

how to delete the second character "." from the line what I have is this ( but its remove the first "." from output uname -r | sed s'/\./ /' 2 6.18-164.2.1.el5PAE while I need the following ...
5
votes
4answers
463 views

extract lines according to a particular column

I have a CSV file like CK,ck XYZ,xyz,xyzs ABC,abc,abcs PQR,pqr,pqrs LMN,lmn, IJK,ijk, I have to check if something is written in column 3 except whitespace, then display the whole line.the output ...
5
votes
4answers
418 views

reverse order of paragraphs in file

I have a file containing text in paragraphs (lines with text separated by one or more empty lines). I would like to reverse the order of paragraphs (i.e. last paragraph will become the first, ...), ...
5
votes
4answers
645 views

sed how to substitute when string has “http://” in it?

I have a file xx that has the following contents: @base_url = "http://dmstaffing-stage.herokuapp.com/" I want to use sed to eliminate this line (replace with nothing). I have used this sed ...
5
votes
4answers
235 views

How could I simplify this command to only use awk?

awk '/user/ {print $1 }' /etc/userdomains | sed 's/://' the format of /etc/userdomains is domain.tld: user otherdomain.tld: otheruser
5
votes
2answers
780 views

sed: read whole file into pattern space without failing on single-line input

Reading a whole file into pattern space is useful for substituting newlines, &c. and there are many instances advising the following: sed ':a;N;$!ba; [commands...]' However, it fails if the ...
5
votes
5answers
419 views

How to run a command only if sed replace strings

I want to enable multiverse repository in Ubuntu by a command line. sed -i "/^# deb.*multiverse/ s/^# //" /etc/apt/sources.list && apt-get update This command will run apt-get update every ...
5
votes
1answer
9k views

Delete specific line number(s) from a text file using sed?

I want to delete one or more specific line numbers from a file. How would I do this using sed?
5
votes
4answers
6k views

Adding a line of text to multiple files

So, I have a bunch of files in a directory, and I need to insert a line of text into each of them. They have essentially the following format: <VirtualHost *:80> ServerAdmin ...
5
votes
4answers
675 views

Is there a difference between read, head -1, and sed 1q?

The following commands seem to be roughly equivalent: read varname varname=$(head -1) varname=$(sed 1q) One difference is that read is a shell builtin while head and sed aren't. Besides that, is ...
5
votes
2answers
9k views

Compare two file columns

I have long text files with space-delimited fields: cat file1.txt Id leng sal mon 25671 34343 56565 5565 44888 56565 45554 6868 23343 23423 26226 6224 77765 88688 87464 6848 66776 23343 63463 ...
5
votes
3answers
4k views

How can sed output be formatted like printf's formatted printing?

Can sed replace text with a string formatted like printf's formatted printing? The following sed command replaces a line starting with the current value of "$domain" with several values specified in ...
5
votes
1answer
3k views

Replacing multiple lines in sed or awk [closed]

I'm trying to use sed or awk to replace 5 lines in a smb file but I just don't have any idea how to deal with the newlines.
5
votes
5answers
9k views

Remove multiple lines after a pattern match but only after pattern match

I am trying to remove all instances of a pattern match from a file if it matches a pattern. If there is a match, the (complete) line with the matching pattern and the next line get removed. The next ...
5
votes
4answers
2k views

sed behaves different on FreeBSD and on Linux?

I use both Linux and FreeBSD (specifically, I use Debian Linux and PC-BSD), and I found something weird about sed. I frequently need to convert "tab separated values" files to "comma separated ...
5
votes
4answers
301 views

Want to substitute only first occurence with sed

Original file claudio antonio claudio michele I want to change only the first occurrence of "claudio" with "claudia" so the file result claudia antonio claudio michele I have tried sed -e ...
5
votes
3answers
636 views

“parentheses not balanced” even though I escaped it?

Im trying to run this simple sed command sed '/VALUES\ \([0-9]/d!' It still gives me an error saying "RE error: parentheses not balanced" (yes the quotes are there). What can I do? I mean I could ...
5
votes
4answers
233 views

How to replace m part of Lines with n part of Lines?

I have a file "test" like this: RewiteEngine On RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.7$ [NC] RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.8$ [NC] RewriteCond %{REMOTE_ADDR} ...
5
votes
6answers
4k views

How to match exact string using `sed`? But not the part of it.?

I have a input file FILE1.TXT as below. 11 id1 12 13 AGE = 20 14 NAME = NAME1 15 16 id2 17 18 AGE = 30 19 NAME = NAME2 . . . 110 idXYZ 111 112 AGE = AGEXYZ 113 NAME = ...
5
votes
2answers
2k views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
5
votes
2answers
4k views

Single command taking two strings to extract string between them, like 'tr' (without expressions)

Saw here a way to use sed to get text between two other strings in a line, like: sed 's/.*starting_text\(.*\)ending_text.*/\1/' but I'd like a simple command (like tr, but for string extraction) ...
5
votes
1answer
15k views

Replacing string based on line number

I have a situation where i want to replace a particular string in many files Replace a string AAA with another string BBB but there are lot of strings starting with AAA or ending in AAA ,and i want ...
5
votes
2answers
162 views

How can I delete all unordered lines from a text file?

Overview Consider an ordered list interspersed with unordered elements, e.g.: Alligator Ant Falcon <-- Baboon Badger Armadillo <-- Caiman Cat How can this list be processed so that all ...
5
votes
3answers
965 views

Extracting fixed width records with no delimiter from a single line

I need to extract strings of text from a single file containing one very long line of text with no delimiters. Using the sample line below, these are the following known facts: ...
5
votes
3answers
544 views

Gedit or an other non-commandline editor with the ability to spell-check two languages at the same time

I'm searching for an editor with the ability to spell-check two languages at the same time (German and English). Gedit can't do it out of the box. But I want to use Gedit. It should be possible by ...
5
votes
3answers
238 views

Extracting data from complex file structure

I have a txt file that is a dump from a database, that contains one entry per line. The structure is like this: 1500 29/03/2010 18 02 09 47 17 45 28.248 0 0.01 130 12.721 7908 298,809 YES ...