sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

4
votes
5answers
995 views

Multi-line grep beween two tags, only show the last match

I'm having a program that intermingles data segments into log output: log message log message ----BEGIN INLINE DATA---- data data data -----END INLINE DATA----- log messge I'm looking for something ...
0
votes
0answers
27 views

Variable Substitution in sed [duplicate]

I have this script and I need to count the lines inside a file, then pass the number of lines counted to a for loop so that I can get the contents of each line and put it in a variable. My question ...
24
votes
6answers
58k views

Delete First line of a file

How can I delete the first line of a file and keep the changes? I tried this but it erases the whole content of the file. $sed 1d file.txt > file.txt
0
votes
3answers
193 views

Search in iptables.log file takes a long time

I have used iptables to save information about connections by modifying the /etc/syslog.conf file to save information in /var/log/iptables.log. So, every X hours, I need to extract information from ...
2
votes
4answers
10k views

How exactly do I create a sed script and use it to edit a file? [duplicate]

I know how to edit a file using sed one step at a time, but how would I create an actual sed script and then use it on the file to do the same things that the individual sed commands would do? For ...
1
vote
2answers
287 views

How would I use sed to detect a certain amount of numbers before it inserts a character? ###-###-####

I need to format phone numbers in a text file. They appear as (example) 8014516912 How would I format it so that I can insert a - after the first 3 numbers, and then another - after another 3 ...
2
votes
1answer
2k views

How would I use sed to insert a character at a certain character position on only ONE line?

I am aware that this will do what I want on all lines: sed 's/^\(.\{4\}\)/\1-/' textFile But how would I enter a - after the 4th character slot on ONLY the 5th line? Or even better, on a range of ...
4
votes
2answers
7k views

How can I get my sed command to make permanent changes to a file?

Is this possible? I read somewhere that the following command would do it: sed -e [command] [file] but it appeared to do the same thing as just sed [command] [file] (it did not save the ...
3
votes
2answers
1k views

Replace a pattern in a file with a long string

I'm writing a little shell script to generate a directory listing. To make the output easily customizable, the script just builds a HTML-table and than should replace a specific token in a template ...
5
votes
3answers
1k views

Deleting Lines if there is a dot in the 5th Column

I have a file and I want to remove all the line if it has a dot in the 5th Column. Input File: sc2/80 20 . A C 80 PASS N=2 F=5;U=4 sc2/60 ...
3
votes
2answers
13k views

Removing text between two specific strings

I have a file as below: mime PI Name: ISHO SUCCESS RATE RT, Value: 95.663826 scr PI Name: RRC Access Failures due to UU, Value: 0.13394141 prog PI Name: RRC Access Failures due to UU, Value: ...
2
votes
1answer
216 views

How to extract single value from single line tuples?

I have a dump file of a database in the following form ('value1','value2',value3','value4'),('value1','value2',value3','value4'),... I want first to have each tuple in a single line and next i ...
0
votes
1answer
92 views

Can i use & variable of sed for doing operation inside $()? [duplicate]

I am not sure whether the below statement can be done placing & inside $() so that each time & is replaced by 1,2,3,4 and 1 is added to it. echo 1 2 3 4 | tr ' ' '\n' | sed "s/.*/$(echo expr ...
3
votes
1answer
886 views

Efficient way to print lines from a massive file using awk, sed, or something else? [duplicate]

If I had a plain-text file containing 8 million lines and I want to print out lines 4,000,010 to 4,000,000 to the screen, which would be more efficient: awk or sed? There is no pattern to the text, ...
0
votes
2answers
679 views

any command that can format html or xml from shell? [closed]

There must be something on the lines of sed? Normally in sed you type something like: echo day | sed s/day/night/ the shell prints "night". Is there something that can take a file (containing ...
0
votes
4answers
66 views

cut+sed function joined

Input dir/foo Output dir./ so I need: echo 'dir/foo' | MAGIC Q: What is MAGIC? (something like a cut+sed joined:D)
4
votes
2answers
2k views

sed: match two lines

I have a text file, where records are split on two lines. I would like to put them on one line. The records look something like this: 2013-07-29 apt-get install git 2013-07-29 apt-get install rsync ...
4
votes
3answers
2k views

Count the number of rows in each column of a text file

I have a text file with the following format: 5 3 1 2 3 4 ..... ..... i.e. space separated 3 columns of numbers/ However, some of the rows may be like: 2 3 1 So, I want to detect if the text file has ...
1
vote
3answers
376 views

Reversing the value key pairs of array using sed or pattern replacement or brace expansion?

#!/bin/bash arr=(a b c d) declare -A brr for i in ${!arr[@]} do brr[${arr[$i]}]=$i done echo ${brr[@]} #0 1 2 3 Is there any way to reverse the values and key pairs ...
4
votes
1answer
5k views

How to assign result of sed to variable

Below is a complete copy of a demo I'm using to to figure out to get the sed command to get rid of the spaces in a persons name, and compress it down to not have spaces. Once this is done, I want to ...
2
votes
1answer
414 views

How to use awk or sed to convert csv diffs into more readable format

Can anyone give me the example of how to use awk or sed (not sure which one, as I haven't use either of these much, as I have mostly been using grep and cut to work with csv data) to convert ...
2
votes
3answers
3k views

Printing match groups with grep

I have the following grep: grep -Po 'href="(http://.*)">Transcript.* - ([0-9]*)\. (.*)</a>' I would like to print out the match groups. Is there a way to do this without Perl/Python? I ...
2
votes
2answers
292 views

Sanitize Apache access log files?

We have this code in a shell script that pipes output for Apache to log. declare -a values=( $taintRequestVals ) for item in ${!values[@]} do cat $apacheLog | sed ...
2
votes
1answer
57 views

How can i change the format of text using sed?

cat > inputfile X1 a1,b1,c1,d1.................. dn X2 a2,b2,c2,d2.................. and so on upto n ................................................ ...
13
votes
8answers
3k views

How to perform a multi line grep

How would you perform a grep for text that appears on two lines? For example: pbsnodes is a command I use that returns the utilization of a linux cluster root$ pbsnodes node1 state = free ...
17
votes
6answers
2k views

How to achieve portability with sed -i (in-place editing)?

I'm writing shell scripts for my server, which is a shared hosting running FreeBSD. I also want to be able to test them locally, on my PC running Linux. Hence, I'm trying to write them in a portable ...
5
votes
3answers
353 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
5
votes
2answers
4k views

Single command taking two strings to extract string between them, like 'tr' (without expressions)

Saw here a way to use sed to get text between two other strings in a line, like: sed 's/.*starting_text\(.*\)ending_text.*/\1/' but I'd like a simple command (like tr, but for string extraction) ...
4
votes
3answers
467 views

Join every other column with sed or awk

I have a large text file (666000 colums) in the format A B C D E F Desired output AB CD EF How can we do it in sed or awk. I have tried a couple of things but nothing seems to be working. Please ...
6
votes
5answers
302 views

Get filesystem name and usage with `df -h`

df -h | grep test | sed -e 's/*%.^ //g;s/.*[ ^I]//' Output: /test /test/drv0 /test/drv1 df -h | grep test | cut -d '%' -f1 | sed -e 's/*%.^ //g;s/.*[ ^I]//' Output: 10 20 15 I want to ...
3
votes
1answer
4k views

Get all regex matches between two patterns and print them to file

I've got a file with a bunch of long lines. I'd like to grab every group between two patterns and print them to a new file, one match per line. I could manage to do this with Python, but I'd prefer ...
0
votes
2answers
916 views

How to remove n number of lines from a file using sed command? [duplicate]

I want to remove a particular number of lines after the given pattern is matched. Can anyone please tell me how that can be done using the sed command ?
2
votes
4answers
264 views

AWK Search massive file and write to variable name

I want to pull data that I will need to reformat from a simulation file I am working with to prepare it as input for running other simulations. The target data is an optimized chemical z-matrix. The ...
0
votes
1answer
860 views

Sed - unknown option to `s'

I execute a sed command in my script: sed -i "${1}s/${2}=.*/${2}=${REPLY}/" $3 this is the echo: sed -i 13s/log4j.appender.ROOT2.File=.*/log4j.appender.ROOT2.File=/app/TET/log.log/ ...
1
vote
1answer
2k views

How to find out the content of a XML file using Unix Sed/Awk? [duplicate]

I have a XML file(MyXML.xml) like this : <?xml version="1.0" encoding="UTF-8"?> <S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"> <S:Body> ...
1
vote
1answer
93 views

How to match a literal '*' with sed?

I have this line in my file called test.php 'dbs_password' => 'a8b*cyP0', and want to replace into: 'dbs_password' => 'password-here', So I use this sed command with syntax: $ sed -e ...
1
vote
2answers
156 views

Sed script inserting and appending in wrong place only in one particular place

I am trying to convert man pages to tex, and I keep getting \item[ in the beginning of the file and ] \hfill \\ After the part of the man pages where it shows ENV(1L). Everything else seems to ...
1
vote
3answers
58 views

Stuck in separating chars into strings in my approach. Found a working approach but can not understand it

I am trying to figure out how to break a string to be on character per line. I.e ahebhaaa to be: a h e b h a a a I tried: $ echo ahebhaaa | sed 's/\(.\)\(.\)/\1\n\2/g' I.e. my ...
4
votes
2answers
622 views

Why do I need to escape regex characters in sed to be interpreted as regex characters?

It seems e.g. cat sed_data.txt | sed 's/\b[0-9]\{3\}\b/NUMBER/g' that I must escape characters to form a regular expression. In this case I had to escape braces in order to be interpreted as a number ...
1
vote
2answers
744 views

Improve sed command to replace first instance of character and all following characters?

Input: <e1 name="file1" id="id1" anotherId="id2"> Desired output: file1 I can get what I need with this: echo '<e1 name="file1" id="id1" anotherId="id2">' | sed ...
0
votes
1answer
756 views

Assign sed result to a variable

I have a sed command and I would like to assign the result to a variable. My command is: sed -n 's/^Release:\s*\(.*\){?dist}/\1/p' file.spec And I tried var = $(sed -n ...
2
votes
3answers
191 views

Using sed to insert latex commands around headers of a document

I have a documen with headers in ALL CAPS, for example: NAME env - run a program in a modified environment and I want to insert \item[ before the header and ] \hfill \\ after a header ...
1
vote
4answers
9k views

How to extract line from the file on specific condition

File: chromosome position ref alt chr1 1398 A T chr1 2980 A C chr2 3323 C T,A chr2 3749 T G chr3 5251 C T,G chr3 ...
3
votes
5answers
5k views

How to remove duplicate letters using sed?

Using sed, how can I remove duplicate letters from HEADERS within a text file? NNAAMMEE nice - run a program with modified scheduling priority SSYYNNOOPPSSIISS nice [-n ...
3
votes
1answer
235 views

How can I write a sed script to delete numbers from a line?

Sample File a.b.gg3d 0000000001 0000000001 y a.b.112a 0000000000 0000000001 y a.b.gg3s 0000000001 0000000001 y a.b.1111 0000000000 0000000001 m I have a file that includes about three thousand ...
3
votes
2answers
2k views

How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?

I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below. col0,col1,col2,col3,col4,col5,col6, ...
1
vote
2answers
2k views

sed : Ignore line starting whitespace for match

I want to replace the default listening port of httpd to 9090. I can edit the line in httpd.conf file using below sed -i "/^Listen/c\Listen 9090" /etc/httpd/conf/httpd.conf But the line Listen 80 ...
5
votes
4answers
6k views

SED : Delete 4 line above and below 5 line after pattern match

I have Hugh file with following details : define host{ use generic-host ; Name of host template to use host_name ServerA_172.29.16.102 ...
1
vote
1answer
80 views

Problem with sed

root@ROUTER:~# cat maccheck.txt logread | egrep ': STA |DHCPACK' | awk '{print $1" "$2" "$3" "$9}' | sed -e 's/\( [0-9] \)/0\1/' | sed "s/.\{15\}/&:/; s/: /:/g" | cut -d : -f 1,2,4,5,6,7,8,9 | ...
4
votes
6answers
114 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?