sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

1
vote
2answers
431 views

iterate through files ('for file in…) and also include sub directories?

I am using sed to do a bunch of changes to files. I think the question is relevant for any operation that want to iterate through files that match a pattern and include subdirectories both in the ...
1
vote
1answer
185 views

best programming language for I/O intensive tasks [closed]

I need to process "big data" (nearly 1TB which can be partitioned into smaller chunks). Consider several programming languages (Java, Python, C++, Perl) and other scripting options (Perl, awk, sed, ...
4
votes
1answer
241 views

Matching Five Columns in two Files using Awk

I have two Input files. File1: s2/80 20 . A T 86 F=5;U=4 s2/20 10 . G T 90 F=5;U=4 s2/90 60 . C G 30 ...
3
votes
3answers
91 views

Using sed (?) to extract one instance of a string that is repeated in the original string

INPUT: <a href=FII/FOO./>FOO</a> OUTPUT: FOO Q: how can I replace the INPUT with the OUTPUT? The FII and "./>" is CONSTANT. The FOO could vary to any english letter and "-" and "." ...
4
votes
1answer
215 views

Reverse a line in sed?

echo 1234 | sed ' /\n/ !G s/\(.\)\(.*\n\)/&\2\1/ //D s/.// ' I'm unable to comprehend the above sed code. my understanding is: PATTERN SPACE=1234 first ...
1
vote
2answers
180 views

Replace a pattern which ends at newline with the same pattern without the newline

I have a javascript file with two new lines after a particular word "[FACT]", I want to replace it with one newline. I need to do it in bash script. Normally, I would have used sed but it turns out ...
4
votes
1answer
104 views

Is strace a suitable tool for finding out where a program wrote to the filesystem?

I'd like to determine where a certain app (as an example, surf) writes to the filesystem in order to make sure it leaves no traces. Without a dedicated sandboxing system this probably isn't easy, but ...
4
votes
1answer
7k views

Using sed to extract text between 2 tags

I have a .xml file and I'm trying to do a "groupinstall" on a RHEL6 machine as there are several hundred libraries in that .xml file... (close to 16 000 lines). I'm therefore trying to extract the ...
0
votes
4answers
70 views

How to make 1:52 to 01:52 with eg.: sed?

INPUT: a|2013.06.13. 13:22|xx b|2013.06.13. 1:52 |xx c|2013.06.13. 2:26 |xx OUTPUT: a|2013.06.13. 13:22|xx b|2013.06.13. 01:52 |xx c|2013.06.13. 02:26 |xx How can I make the 1:52 to 01:52 in ...
4
votes
3answers
1k views

using sed to replace pattern with hash values

I want to search the file and replace specific pattern with its hash (SHA1) values. For example, let file.txt has the following content: one S56G one two three four five V67X six and I want to ...
4
votes
5answers
2k views

extract fields from “tail -f” of a syslog stream

Example line from syslog file: Aug 1 10:25:50 10.10.10.1 id=firewall sn=XXXX time="2012-08-01 14:35:18 UTC" fw=x.x.x.x pri=6 c=1024 m=537 msg="Connection Closed" f=11 n=195273698 ...
3
votes
4answers
538 views

How to Prefix a column values with an apostrophe ( ' )?

I have a CSV file with multiple columns and 1000's of records, I need to prefix all the values of one of the columns (lets say 2nd column) with an apostrophe ' except in the first line or header line, ...
6
votes
2answers
588 views

How to report “sed” in-place changes

When using sed to replace strings in-place, is there a way to make it report the changes it does (without relying on a diff of old and new files)? For instance, how can I change the command line ...
0
votes
0answers
26 views

Variable Substitution in sed [duplicate]

I have this script and I need to count the lines inside a file, then pass the number of lines counted to a for loop so that I can get the contents of each line and put it in a variable. My question ...
1
vote
3answers
328 views

Reversing the value key pairs of array using sed or pattern replacement or brace expansion?

#!/bin/bash arr=(a b c d) declare -A brr for i in ${!arr[@]} do brr[${arr[$i]}]=$i done echo ${brr[@]} #0 1 2 3 Is there any way to reverse the values and key pairs ...
1
vote
4answers
6k views

How exactly do I create a sed script and use it to edit a file? [duplicate]

I know how to edit a file using sed one step at a time, but how would I create an actual sed script and then use it on the file to do the same things that the individual sed commands would do? For ...
5
votes
3answers
746 views

Deleting Lines if there is a dot in the 5th Column

I have a file and I want to remove all the line if it has a dot in the 5th Column. Input File: sc2/80 20 . A C 80 PASS N=2 F=5;U=4 sc2/60 ...
1
vote
2answers
206 views

How would I use sed to detect a certain amount of numbers before it inserts a character? ###-###-####

I need to format phone numbers in a text file. They appear as (example) 8014516912 How would I format it so that I can insert a - after the first 3 numbers, and then another - after another 3 ...
2
votes
1answer
1k views

How would I use sed to insert a character at a certain character position on only ONE line?

I am aware that this will do what I want on all lines: sed 's/^\(.\{4\}\)/\1-/' textFile But how would I enter a - after the 4th character slot on ONLY the 5th line? Or even better, on a range of ...
1
vote
2answers
8k views

Removing text between two specific strings

I have a file as below: mime PI Name: ISHO SUCCESS RATE RT, Value: 95.663826 scr PI Name: RRC Access Failures due to UU, Value: 0.13394141 prog PI Name: RRC Access Failures due to UU, Value: ...
2
votes
1answer
191 views

How to extract single value from single line tuples?

I have a dump file of a database in the following form ('value1','value2',value3','value4'),('value1','value2',value3','value4'),... I want first to have each tuple in a single line and next i ...
3
votes
1answer
654 views

Efficient way to print lines from a massive file using awk, sed, or something else? [duplicate]

If I had a plain-text file containing 8 million lines and I want to print out lines 4,000,010 to 4,000,000 to the screen, which would be more efficient: awk or sed? There is no pattern to the text, ...
0
votes
1answer
86 views

Can i use & variable of sed for doing operation inside $()? [duplicate]

I am not sure whether the below statement can be done placing & inside $() so that each time & is replaced by 1,2,3,4 and 1 is added to it. echo 1 2 3 4 | tr ' ' '\n' | sed "s/.*/$(echo expr ...
4
votes
2answers
964 views

sed: match two lines

I have a text file, where records are split on two lines. I would like to put them on one line. The records look something like this: 2013-07-29 apt-get install git 2013-07-29 apt-get install rsync ...
0
votes
4answers
64 views

cut+sed function joined

Input dir/foo Output dir./ so I need: echo 'dir/foo' | MAGIC Q: What is MAGIC? (something like a cut+sed joined:D)
3
votes
3answers
1k views

Count the number of rows in each column of a text file

I have a text file with the following format: 5 3 1 2 3 4 ..... ..... i.e. space separated 3 columns of numbers/ However, some of the rows may be like: 2 3 1 So, I want to detect if the text file has ...
1
vote
1answer
340 views

How to use awk or sed to convert csv diffs into more readable format

Can anyone give me the example of how to use awk or sed (not sure which one, as I haven't use either of these much, as I have mostly been using grep and cut to work with csv data) to convert ...
2
votes
3answers
2k views

Printing match groups with grep

I have the following grep: grep -Po 'href="(http://.*)">Transcript.* - ([0-9]*)\. (.*)</a>' I would like to print out the match groups. Is there a way to do this without Perl/Python? I ...
6
votes
5answers
22k views

Multiline pattern match using sed, awk or grep

Is it possible to do a multiline pattern match using sed, awk or grep? Take for example, I would like to get all the lines between { and } So it should be able to match 1. {} 2. {.....} 3. ...
2
votes
1answer
4k views

How to assign result of sed to variable

Below is a complete copy of a demo I'm using to to figure out to get the sed command to get rid of the spaces in a persons name, and compress it down to not have spaces. Once this is done, I want to ...
2
votes
2answers
203 views

Sanitize Apache access log files?

We have this code in a shell script that pipes output for Apache to log. declare -a values=( $taintRequestVals ) for item in ${!values[@]} do cat $apacheLog | sed ...
2
votes
1answer
56 views

How can i change the format of text using sed?

cat > inputfile X1 a1,b1,c1,d1.................. dn X2 a2,b2,c2,d2.................. and so on upto n ................................................ ...
2
votes
4answers
238 views

AWK Search massive file and write to variable name

I want to pull data that I will need to reformat from a simulation file I am working with to prepare it as input for running other simulations. The target data is an optimized chemical z-matrix. The ...
3
votes
3answers
371 views

Join every other column with sed or awk

I have a large text file (666000 colums) in the format A B C D E F Desired output AB CD EF How can we do it in sed or awk. I have tried a couple of things but nothing seems to be working. Please ...
3
votes
2answers
271 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
6
votes
5answers
265 views

Get filesystem name and usage with `df -h`

df -h | grep test | sed -e 's/*%.^ //g;s/.*[ ^I]//' Output: /test /test/drv0 /test/drv1 df -h | grep test | cut -d '%' -f1 | sed -e 's/*%.^ //g;s/.*[ ^I]//' Output: 10 20 15 I want to ...
3
votes
1answer
3k views

Get all regex matches between two patterns and print them to file

I've got a file with a bunch of long lines. I'd like to grab every group between two patterns and print them to a new file, one match per line. I could manage to do this with Python, but I'd prefer ...
0
votes
2answers
839 views

How to remove n number of lines from a file using sed command? [duplicate]

I want to remove a particular number of lines after the given pattern is matched. Can anyone please tell me how that can be done using the sed command ?
0
votes
1answer
630 views

Sed - unknown option to `s'

I execute a sed command in my script: sed -i "${1}s/${2}=.*/${2}=${REPLY}/" $3 this is the echo: sed -i 13s/log4j.appender.ROOT2.File=.*/log4j.appender.ROOT2.File=/app/TET/log.log/ ...
4
votes
2answers
417 views

Why do I need to escape regex characters in sed to be interpreted as regex characters?

It seems e.g. cat sed_data.txt | sed 's/\b[0-9]\{3\}\b/NUMBER/g' that I must escape characters to form a regular expression. In this case I had to escape braces in order to be interpreted as a number ...
1
vote
1answer
1k views

How to find out the content of a XML file using Unix Sed/Awk? [duplicate]

I have a XML file(MyXML.xml) like this : <?xml version="1.0" encoding="UTF-8"?> <S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"> <S:Body> ...
22
votes
10answers
6k views

How can I write to the second line of a file from the command line?

I have an external program that produces an output file (largish, 20K lines possible). I need to insert a new line between the existing line 1 and line 2. I've been looking at awk and sed - I use ...
1
vote
1answer
78 views

How to match a literal '*' with sed?

I have this line in my file called test.php 'dbs_password' => 'a8b*cyP0', and want to replace into: 'dbs_password' => 'password-here', So I use this sed command with syntax: $ sed -e ...
1
vote
2answers
145 views

Sed script inserting and appending in wrong place only in one particular place

I am trying to convert man pages to tex, and I keep getting \item[ in the beginning of the file and ] \hfill \\ After the part of the man pages where it shows ENV(1L). Everything else seems to ...
1
vote
3answers
58 views

Stuck in separating chars into strings in my approach. Found a working approach but can not understand it

I am trying to figure out how to break a string to be on character per line. I.e ahebhaaa to be: a h e b h a a a I tried: $ echo ahebhaaa | sed 's/\(.\)\(.\)/\1\n\2/g' I.e. my ...
1
vote
2answers
670 views

Improve sed command to replace first instance of character and all following characters?

Input: <e1 name="file1" id="id1" anotherId="id2"> Desired output: file1 I can get what I need with this: echo '<e1 name="file1" id="id1" anotherId="id2">' | sed ...
3
votes
5answers
3k views

How to remove duplicate letters using sed?

Using sed, how can I remove duplicate letters from HEADERS within a text file? NNAAMMEE nice - run a program with modified scheduling priority SSYYNNOOPPSSIISS nice [-n ...
2
votes
3answers
159 views

Using sed to insert latex commands around headers of a document

I have a documen with headers in ALL CAPS, for example: NAME env - run a program in a modified environment and I want to insert \item[ before the header and ] \hfill \\ after a header ...
0
votes
1answer
469 views

Assign sed result to a variable

I have a sed command and I would like to assign the result to a variable. My command is: sed -n 's/^Release:\s*\(.*\){?dist}/\1/p' file.spec And I tried var = $(sed -n ...
2
votes
2answers
2k views

How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?

I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below. col0,col1,col2,col3,col4,col5,col6, ...