Sed is a command-line stream editor for filtering and transforming text.
45
votes
6answers
7k views
Is there a basic tutorial for grep, awk and sed?
I've been a Linux user for a while, and I've a pretty decent understanding of most the common command line utilities. However, ones that come up and up again in relation to programming are grep, awk, ...
19
votes
9answers
2k views
Addition with 'sed'
I am trying to perform a mathematical operation with sed, but it continues to treat my variables as strings. The input is of this kind:
$ echo 12 | sed 's/[0-9]*/&+3/'
$ 12+3
I'd like to have ...
17
votes
9answers
3k views
How can I write to the second line of a file from the command line?
I have an external program that produces an output file (largish, 20K lines possible).
I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
12
votes
9answers
1k views
There must be a better way to replace single newlines only?
I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
10
votes
4answers
10k views
How to manipulate a CSV file with sed or awk?
How can I do the following to a CSV file using sed or awk?
Delete a column
Duplicate a column
Move a column
I have a big table with over 200 rows, and I'm not that familiar with sed.
9
votes
4answers
1k views
Delete last line from the file
I use sed to quickly delete lines with specific position as
sed '1d'
sed '5d'
But, what if I want to delete the last line of the file and I don't know the count of lines (I know I can get that ...
9
votes
6answers
912 views
How can I prepend a tag to the beginning of several files?
I need to add PHP tags surrounding a file. It's easy to append them using
find . -exec echo "?>" >> '{}' \;
but how can I prepend the tag <?php?
9
votes
2answers
2k views
How to remove multiple lines per occurrence in a file?
Say I have this 857835 line file, containing stuff like this:
a1
rubbish1
rubbish2
rubbish3
rubbish4
a1
rubbish5
rubbish6
rubbish7
rubbish8
And I wish to remove all occurences of a1 and the next ...
9
votes
3answers
766 views
Substituting strings in a very large file
I have a very long series of urls with no separating character, in the same format as below:
http://example.comhttp://example.nethttp://example.orghttp://etc...
I want each URL to be on a new line. ...
9
votes
3answers
293 views
Deleting all C comments with sed
I am trying to write a script that will delete all comments and everything in between inside C files in my current directory. I've been using sed, and this is what I have so far:
sed -i '/ * [^()] ...
8
votes
6answers
220 views
Grab certain contents of a file
So I know tools exist for this problem because I've heard about them, but I don't know what they are.
I want to do something like filter out all data but the usernames in /etc/passwd.
For example, I ...
8
votes
3answers
713 views
How to sort the string which combined with string + numeric using bash script?
This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected.
/home/files/profile1
/home/files/profile10
/home/files/profile11
...
8
votes
6answers
725 views
Delete lines beginning with #
How do I delete lines beginning with a #, given that there can be whitespace on the left and right of the #?
# Master socket provides access to userdb information. It's typically
8
votes
4answers
397 views
How do I write a sed one-liner to add a character after every third character?
So, I have a string that looks like this:
AUGGCCAUGGCGCCCAGAACUGAGAUCAAUAGUACCCGUAUUAACGGGUGA
And I want to split the string into 3-character chunks delimited by a '+' sign.
...
8
votes
2answers
2k views
How to replace a string with a string containing slash with sed?
i am looking for a way to replace a string in a file with a string that contains a slash by using sed.
connect="192.168.100.61/foo"
srcText="foo.bar=XPLACEHOLDERX"
echo $srcText | sed ...
8
votes
4answers
2k views
Efficient in-place header removing for large files using sed?
The commands below may takes minutes depends on the file size. Is there any more effient method?
sed -i 1d large_file
8
votes
3answers
2k views
Zero-fill numbers to 2 digits with sed
Input:
201103 1 /mnt/hdd/PUB/SOMETHING
201102 7 /mnt/hdd/PUB/SOMETH ING
201103 11 /mnt/hdd/PUB/SO METHING
201104 3 /mnt/hdd/PUB/SOMET HING
201106 1 /mnt/hdd/PUB/SOMETHI NG
Desired output:
201103 ...
8
votes
3answers
2k views
Delete the first n bytes of files
I've got an extreme problem, and all of the solutions I can imagine are complicated. According to my UNIX/Linux experience there must be an easy way.
I want to delete the first 31 bytes of each file ...
8
votes
1answer
222 views
How does this sed comand work?
I've come across this solution for printing a specific line from a text file:
sed '123!d;q' file
Why doesn't sed quit after the first line of input in this case?
8
votes
4answers
2k views
Remove files older than 5 days in UNIX (date in file name, not timestamp)
I want to delete log files which are older than 5 days from a directory. But deletion should not be based on the timestamp of file. It should be based on the name of file. For Example todays date is ...
7
votes
10answers
506 views
What's a good way to filter a text file to remove empty lines?
I have a .csv file (on a mac) that has a bunch of empty lines, e.g.:
"1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
7
votes
5answers
2k views
Remove comma between the quotes only in a comma delimited file
I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row
123,"ABC, DEV 23",345,534.202,NAME
I need to ...
7
votes
4answers
4k views
grep — removing text after delimiter token
I have a file in which I need to eliminate everything after the first ; on every line.
So a file like this:
sdfsdsdf;
fsdfsddf;sdfsd;
Will result in this:
sdfsdsdf
fsdfsddf
I have looked into ...
7
votes
2answers
3k views
Replace string with contents of a file using sed
I have two different files:
File1
/home/user1/
/home/user2/bin
/home/user1/a/b/c
File2
<TEXT1>
<TEXT2>
I want to replace the <TEXT1> of File2 with the contents of File1 ...
7
votes
1answer
236 views
Change date “July 29th, 2011” to “20110729”
I have lots of HTML files contains date with format July 29th, 2011
I want to change date format July 29th, 2011 to 20110729, December 9th, 2010 to 20101209, etc.
I think sed may works but still ...
7
votes
4answers
6k views
Copy/rename multiple files using regular expression (shell script)
I have a collection of files matching a pattern such as 'assignment02.cc', 'assignment02.h', 'assignment02.txt', etc. I would like to copy/rename these files into 'assignment03.cc', 'assignment03.h', ...
6
votes
4answers
2k views
Remove line containing certain string and the following line
I use this
cat foo.txt | sed '/bar/d'
to remove lines containing the string 'bar' in the file.
I would like however to remove those lines and the line directly after it. Preferably in sed, awk or ...
6
votes
2answers
864 views
sed one-liner to delete any line that does not contain lowercase letters
So, basically
THIS LINE WOULD BE DELETED
and
(THIS LINE WOULD ALSO BE DELETED)
but
Indeed, THIS LINE WOULD NOT
6
votes
6answers
2k views
How can you combine all lines that end with a backslash character?
Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash?
For example, given the file:
foo bar \
bash \
baz
dude \
happy
...
6
votes
3answers
737 views
How to delete line if longer then XY?
How can i delete a line, if it's longer then e.g.: 2048 chars?
6
votes
1answer
223 views
How to detect and delete lines containing ˆ@
I have a simple problem:
In my file, the are lines containing the string ˆ@ˆ@ˆ@ˆ@ˆ@ˆ@. I just want to delete all lines with this string, using for example the sed or grep commands.
And I would like ...
6
votes
3answers
7k views
What characters do I need to escape when using sed in a sh script?
Take the following script:
#!/bin/sh
sed 's/(127\.0\.1\.1)\s/\1/' [some file]
If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
6
votes
3answers
648 views
Replacing Multiple blank lines with a single blank line in vim / sed
Question more or less says it all. I'm aware that /^$/d will remove all blank lines, but I can't see how to say 'replace two or more blank lines with a single blank line'
Any ideas?
6
votes
3answers
200 views
Locate postion then make a change using sed
This script uses sed to change all "" to "new stuff". How would one change just the "" after the yyy: using sed or anything else?
cat >sample.txt <<EOF
xxx:
""
yyy:
""
}
EOF
sed ...
6
votes
4answers
3k views
How can I use sed to replace a multi-line string?
I've noticed that, if I add \n to a patter for a substitute using sed, it does not match. Example:
$ cat > alpha.txt
This is
a test
Please do not
be alarmed
$ sed -i'.original' 's/a test\nPlease ...
6
votes
1answer
429 views
Why isn't sed greedy in this simple case?
$echo "foo 65 bar" | sed -n -e 's/.*\([0-9]\+\).*/\1/p'
5
Why is the output not 65? Shouldn't sed greedily match the [0-9]\+ part? How do I tell sed to match all of 65?
6
votes
2answers
636 views
Using sed to color the output from a command on solaris
I have a ksh script that must work on both linux and solaris. I'm trying to color the output of specific commands. It works on linux (specifically RHEL6), but not on solaris (SunOS 5.10).
Command ...
6
votes
1answer
342 views
How do you save a complex regex for multiple reuse in sed?
In using sed, I often create rather complicated and intricate regexes that I need to match twice in a file. Is there a way for me to save this regex and just reference it twice?
Maybe something that ...
5
votes
4answers
459 views
How to append end of every line with the line itself
This is in file.txt:
redcar
bluecar
greencar
Im looking for ways to make it become:
redcar redcar
bluecar bluecar
greencar greencar
I've tried many ways using sed with no luck
5
votes
7answers
249 views
Swapping an unlimited number of columns
I have a file with columns. See below for an example:
a b c ... z
1 2 3 ... 26
I'd like to swap all columns where the 1st becomes the last, the second becomes the one before last...etc..
z y x ...
5
votes
3answers
3k views
Processing bash variable with sed
Been banging my head off a wall on this
bash variable LATLNG contains a latitude & longitude value in brackets like so
(53.3096,-6.28396)
I want to parse these into a variable called LAT and ...
5
votes
3answers
2k views
How to sed only that lines that contains given string?
INPUT:
Select ASDF 325 sdfg sdflk lk
Select TRG 46sdg rasdftz fsgs 45
Select ASDF 6ffg sdfg 4456 sdrg
OUTPUT:
Select ASDF 325 XXXX sdflk lk
Select TRG 46sdg rasdftz fsgs 45
Select ASDF 6ffg XXXX ...
5
votes
4answers
766 views
sed on OSX insert at a certain line
So I've been using 'sed' on linux for a while, but have had a bit of difficulty trying to use it on OSX since 'POSIX sed' and 'GNU sed' have so many little differences. Currently I'm struggling with ...
5
votes
3answers
1k views
The way to use `/usr/bin/env sed -f ` in shebang?
Typing /usr/bin/env sed -f in terminal works.
But if use it as a shebang,
#!/usr/bin/env sed -f
s/a/b/
The script will be fail to execute:
/usr/bin/env: sed -f: No such file or directory
I ...
5
votes
5answers
351 views
resolve all ip addresses in command output using standard command line tools
I have several log files that contain a bunch of ip addresses. I would love to be able to pipe the data through a program that would match and resolve ip addresses.
I.E.
cat /var/log/somelogfile | ...
5
votes
4answers
209 views
How could I simplify this command to only use awk?
awk '/user/ {print $1 }' /etc/userdomains | sed 's/://'
the format of /etc/userdomains is
domain.tld: user
otherdomain.tld: otheruser
5
votes
5answers
200 views
How to remove multiple blank lines from a file?
I have some text-files I use to take notes in - just plain text, usually just using cat >> file. Occasionally I use a blank line or two (just return - the new-line character) to specify a new ...
5
votes
2answers
2k views
How to insert a new line after every second line?
sed/bash/wharever :D
INPUT:
sadf
asdf
yxcv
cxv
eqrt
asdf
OUTPUT:
sadf
asdf
yxcv
cxv
eqrt
asdf
5
votes
4answers
6k views
How to strip multipe spaces to one using sed?
sed on AIX is not doing what I think it should.
I'm trying to replace multiple spaces with a single space in the output of IOSTAT:
# iostat
System configuration: lcpu=4 drives=8 paths=2 vdisks=0
...
5
votes
2answers
312 views
Replacing missing value blank space with zero
I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt
...
