sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

4
votes
3answers
910 views

Delete the matching line and several more from a file

I have a text file called file_a.txt. My first command is grep -A 12 ".production =" file_a.txt The output is a few block. Each block of string contains 13 rows I specifically want to delete all ...
4
votes
1answer
2k views

Substitute pattern within a file with the content of other file

I have a text file (devel.xml). I added the word REPLACETHIS to it in order to replace this string with the content within a different file (temp.txt). The closest thing I have is this: sed -i -e ...
4
votes
3answers
708 views

How to skip file in sed if it contains regex?

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed: find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+ As ...
4
votes
1answer
1k views

Use sed to fix broken newlines

I have a folder full of files, each with many hundreds of lines. Many of these files have occasional broken newlines: when I open them in vim, many have ^M^L interspersed where newlines should be. ...
4
votes
2answers
154 views

Replace string in a file with another where both are saved in variables

I want to replace a string found in a file with another string, but both have a special character (in this example it is a . char) e.g 1.0 and 2.0. So this is the command currently used: sed -i ...
4
votes
1answer
76 views

Insert missing data within blocks

I`m a biologist and I am trying to solve this problem, please guide me on where to start. I do not know which forum to post this in, please let me know if this place is not appropriate. I have blocks ...
4
votes
2answers
150 views

sed regex issue

I'm trying to replace a string in a file: $RELEASE = '1234'; The whitespace between RELEASE and the = symbol could be either spaces OR tabs. The quotes around the version number could be either ...
4
votes
1answer
290 views

sed : insert sth before subsequent lines that begin the same, but are not the same

I have a LaTeX file with one glossary entry per line: ... \newglossaryentry{ajahn}{name=Ajahn,description={\textit{(Thai)} From the Pali \textit{achariya}, a Buddhist monk's preceptor: `teacher'; ...
4
votes
5answers
89 views

Multiline Text Processing : Editing sshd_config

I have multiple blocks at the end of my sshd_config files like theses: Match User FOO ChrootDirectory /srv/www/FOO AllowTCPForwarding no X11Forwarding no ForceCommand internal-sftp ...
4
votes
2answers
69 views

How to exchange words in a filename using the shell?

One of my customers is a school. On their file server all projects have a directory with the same naming scheme. <last name>, <first name> <second name>? - <project ...
4
votes
1answer
130 views

Strip color on OS X with BSD sed (or any other tool)

I'm on OS X, which has a BSD version of sed (which I'm guessing is inferior to a GNU version on Linux), and all the techniques for removing color on commandlinefu.com did not work. I tried replacing ...
4
votes
1answer
193 views

How to follow and seek in a file being piped through a filter, in less

I basically wish to seek, search and follow a growing log file after passing it through a sed filter. I figured I need to follow a file with tail -f pipe it to sed and then pipe that to less. For ...
3
votes
4answers
368 views

Decreasing a number with sed

I extracted the metadata of a PDF in a .txt file using pdftk, and now I am trying to decrease the BookmarkPageNumber value for each bookmark by an integer. The .txt has these lines: BookmarkBegin ...
3
votes
7answers
144 views

Insert a word in the first line of the file

Input: firstline secondline thirdline ...some magic happens here... :) Output: insertedtextfirstline secondline thirdline Question: How can I insert the insertedtext to the start of the first ...
3
votes
5answers
270 views

removing extensions in a column

I have a file like this ILMN_1343291 TGTGTTGAGAGCTTCTCAGACTATCCACCTTTGGGTCGCTTTGCTGTTCG NM_001402.5 ILMN_1343295 CTTCAACAGCGACACCCACTCCTCCACCTTTGACGCTGGGGCTGGCATTG NM_002046.3 ILMN_1651209 ...
3
votes
3answers
239 views

Parse multiple sections of data into seperate files

I have a concatenated log file with multiple logs inside that I'm trying to parse out into individual log files (I will later rename them to the date/time of each). Each log is separated by "--- LOG ...
3
votes
4answers
223 views

How to truncate the second column to given length

Given input of the form XY981743 foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte how can I truncate only the second column? The delimiter is TAB and the ...
3
votes
2answers
590 views

Delete every Nth line in shell

I'm trying to delete a line after N lines using awk and I can't seem to get it right. The file format is like this YYYYYY XXXXXX XXXXXX YYYYYY XXXXXX XXXXXX The real example would be office3 3 1 ...
3
votes
4answers
205 views

Extract paragraph separated with *** using AWK

I have a file like below: blablabla blablabla *** thingsIwantToRead1 thingsIwantToRead2 thingsIwantToRead3 blablabla blablabla I want to extract the paragraph with thingsIwantToRead. When I had to ...
3
votes
4answers
593 views

Is is possible to modify a file in a read only directory? [duplicate]

I have already read: Is there a way to modify a file in-place? I am curious if there is a way to modify a file in place using a command that will not create temporary files. Let's say I create a ...
3
votes
6answers
132 views

how can I use sed to replace the lists that begins with (*) in a file

Using sed how can I replace the lines that start with a star and replace it with numbers starting from 1? I need to replace the (*) in the fist of a list with numbers using sed for example > this ...
3
votes
4answers
599 views

How to Prefix a column values with an apostrophe ( ' )?

I have a CSV file with multiple columns and 1000's of records, I need to prefix all the values of one of the columns (lets say 2nd column) with an apostrophe ' except in the first line or header line, ...
3
votes
2answers
5k views

How can I filter those “Permission denied” from find output? [duplicate]

I started using sed recently. One handy way I use it is to ignore unimportant lines of a log file: tail -f example.com-access.log | sed '/127.0.0.1/d;/ELB-/d;/408 0 "-" "-"/d;' But when I try to ...
3
votes
4answers
1k views

shorten long lines in a log file

I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
3
votes
3answers
1k views

Replace %20 with a space in filenames? [duplicate]

I have a list of files that are using %20 to indicate a space in their names. I'm currently trying to do the following to change them all from %20 to a space " ". for x in *.txt do mv $x $(echo $x ...
3
votes
4answers
781 views

How to use sed to selectively remove leading zeros

How can I substitute/remove 0's in this, using sed: 2001:0db8:03cd:0000:0000:ef45:0006:0123 to make it looks like this: 2001:db8:3cd:::ef45:6:123 There are 8 pieces of 4's divided by : in the ...
3
votes
3answers
231 views

How to perform an action only on the first line?

sed 's/.\(.*\)/\1/' myfile Say myfile contains: abc def ghi I want to remove the first character only from the first line but the above removes it from all the line.
3
votes
2answers
242 views

What is this sed command doing?

Can someone explain me what this sed command is trying to achieve? sed 's/^[[:space:]]*//;s/[[:space:]]*$// I understand that it is searching for whitespace characters from the beginning of the ...
3
votes
6answers
3k views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
3
votes
3answers
968 views

Regex not matching

why this does not match..? sed -e '/--Updated?[[:space:]]+Date/d' inputfile this..: --Updated Date: 2013-11-06 15:32:13 d? is because sometimes I have Update Date, and sometimes Updated Date. ...
3
votes
4answers
335 views

Split the input file with line breaks

I am trying to insert line breaks on a file based on the same names which happens to be the second field in my file. So, my input file is like below. 17412193|name1|organization ...
3
votes
2answers
3k views

Why `sed` does not return exit status if regex does not matched ?

I'm trying to search and replace some line from files, but when I run sed if that regex not match then sed does not return any status code, So instead of adding one more condition with grep, is there ...
3
votes
3answers
273 views

Extracting unique piece of text from file with awk, sed or grep

From the output of pactl list sink-inputs, I need to grab the sink input number for VLC. Before that, I'm trying to extract the piece that contains the output for only VLC. All the methods that I ...
3
votes
7answers
286 views

How to add a line after nth occurrence of a keyword using sed?

Using sed, I want to add check after nth occurrence Input: DCR DCR DCR Output: DCR DCR check DCR Is it possible using sed?
3
votes
3answers
1k views

Need to insert single quotes in text file for use as SQL query using sed

I have created a line of text that looks like this: INSERT INTO radcheck(id, username, attribute, op, value) VALUES (,,00:23:32:c2:a9:e8,Auth-Type,:=,Accept); I want to use sed to make it look like ...
3
votes
2answers
127 views

Extract text including parens

I have some text like this: Sentence #1 (n tokens): Blah Blah Blah [... ... ...] ( #start first set here ... (other possible parens and text here) ) #end first set here (...) (...) Sentence #2 ...
3
votes
3answers
178 views

sed syntax to remove 2 characters only when it is box and space

I have a sed expression to remove the first 2 characters from all the lines in a file. sed 's/^..//' raw.txt > converted.txt However, the problem is some lines are large and get appended to the ...
3
votes
3answers
3k views

Delete XML node containing certain element

I want to remove all Placemarks from a KML file that contain the element <tessellate>. The following block should be wholly removed: <Placemark> ...
3
votes
4answers
1k views

Replacing pattern after nth match is found on each line?

I have a file containing lines: india;austria;japan;chile china;US;nigeria;mexico;russia I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd ...
3
votes
5answers
2k views

Remove lines based on pattern by keeping first n lines

I need to remove lines from a text file based on pattern but I need to keep the first n lines of that pattern. Input % 1 % 2 % 3 % 4 % 5 text1 text2 text3 output %1 %2 text1 text2 text3 I used ...
3
votes
6answers
141 views

Extracting lines based on conditions

Each line in a comma-separated file has 5 fields. a,b,c,d,e f,g,c,i, j,k,c,m,n o,p,c,r,s t,u,c,w, x,y,z,aa,bb How can I extract the lines which have c in the 3rd field and their 5th field is NOT ...
3
votes
7answers
1k views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
3
votes
1answer
493 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
3
votes
6answers
2k views

Remove last comma from a bash for loop generated string

I am dealing with a situation where I need to create a comma separated list from an array into a heredoc and remove the last comma. I am using bash for piped into sed which is erasing all commas ...
3
votes
4answers
10k views

extract only the substring after double quotes - grep

I have a file which is as below. <a href="http://firstlink.com" title="title1"> <a href="http://secondlink.com" title="title2"> <a href="http://thirdlink.com" title="title3"> <a ...
3
votes
2answers
11k views

Removing text between two specific strings

I have a file as below: mime PI Name: ISHO SUCCESS RATE RT, Value: 95.663826 scr PI Name: RRC Access Failures due to UU, Value: 0.13394141 prog PI Name: RRC Access Failures due to UU, Value: ...
3
votes
3answers
805 views

Easily get a particular column from output without sed or awk

Is there a quicker way of getting a couple of column of values than futzing with sed and awk? For instance, if I have the output of ls -hal / and I want to just get the file and directory names and ...
3
votes
2answers
407 views

Extract lines containing PAT1 but not PAT2 with sed

I want to extract the lines containing PAT1 but not PAT2 from a file. For example, with PAT1='dog', PAT2='cat', and the following input: 1 cat chicken 2 bird dog apple 3 dog orange cat 4 cat juice ...
3
votes
4answers
210 views

How do I extract the first integer from text string in a column of a tab-delimited file?

I work in Medical Genetics and often have delimited text files where in one column (ex. column 5) there is a text string with a "mutation" in our jargon: c.2458C>T or c.45_46delAA or ...
3
votes
5answers
260 views

How to replace only the third occurrence of the file in unix using only sed command?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...