Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

3
votes
2answers
590 views

How do remove identical lines in one file from another, using sed?

I have two files, one being a superset of the other. I want to remove the identical lines in the smaller files from the larger file. One possible complication is that the lines contain backslashes. ...
3
votes
3answers
373 views

How to parse JSON with shell scripting in Linux?

I have a JSON output from which I need to extract a few parameters in Linux. This is the JSON output: { "OwnerId": "121456789127", "ReservationId": "r-48465168", "Groups": ...
3
votes
3answers
89 views

How to select 30 files and do a bunch of sed commands on them

I want new copies of the files, so I want to do something like: dir *.rb foreach file make a copy of the file, e.g. 'blob.rb' to 'blob_processed.rb' do the 50 sed commands to process the ...
3
votes
3answers
145 views

checking data in columns when a data or some may be missing or present?

I am not sure if this possible. say i have columns like : Team Colour Game Rainfall PlayerName XYZ Blue Cricket Yes Kapil suppose i need to search ...
3
votes
5answers
497 views

changing pattern of a text file

A text file has contents something like chair table pen desk Now i want it to be changed and stored in a variable say var as below ('chair','chair'),('table','table'),('pen','pen'),('desk','desk') ...
3
votes
6answers
101 views

sed remove characters from new line

After some processing, I have my input file like below. file1.txt 12345|john|student 43321|jack|professor 78965|alex|lecturer I need to process the above file further and so I need a line ...
3
votes
3answers
196 views

awk remove the rows based on another column value

I have a file like below which I have sorted based on the username field. UserID score UserName 1234 200 Jack 5678 150 Jill 8543 200 Jill 5678 100 John I am trying to remove ...
3
votes
1answer
152 views

grep on line before the match plus the match

zzzzzzzzz aaaaaaaaa bbbbbbbbb & ccccccccc & ddddddddd hhhhhhhhh eeeeeeeee fffffffff & ggggggggg & in the above line, what i want is to grep/sed/awk (any method is fine) line that ...
3
votes
5answers
3k views

Using sed to find and replace complex string (preferrably with regex)

I have a file with the following contents: <username><![CDATA[name]]></username> <password><![CDATA[password]]></password> ...
3
votes
5answers
174 views

grep device name and look for next value :

I have this output from a find command: abc,10.11.13.14,def,1.2.3.4,geh,6.7.54.23 where abc,def and geh are device names and could be of any length and others are IP address belong to devices. ...
3
votes
6answers
197 views

Extracting help message from script itself

I want that my help message be extracted from the script itself. #!/bin/bash # # foo - do things # Author: John Doe <jhon@doe> # ---------------------------------------------- # SYNOPSIS # ...
3
votes
4answers
364 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...
3
votes
4answers
522 views

Adding a TeX macro to the beginning of a file using sed (why doesn't my method work?)

I need to add a line to the beginning of a file. The line in question is \def\submit{} I've got a semi-working solution using sed. I don't know sed, but got it from somewhere on the net, but it ...
3
votes
1answer
547 views

Why can't I sed two [or more..] empty lines to one empty line?

Why can't I sed two [or more..] empty lines to one empty line? What is the trick? echo -e "hello\n\n\nhello2" | sed 's/^$\n^$/\n/g' hello hello2
3
votes
4answers
204 views

extract the characters before () using grep

I need to extract the characters before the character '('. My file structure is: rose(good)ook daisy()no lilly(bad)fine sunflower()nice I need output file as in the form of: rose ...
3
votes
2answers
260 views

Basic sed command on large one-line file: couldn't re-allocate memory

I have a 250 MB text file, all in one line. In this file I want to replace a characters with b characters: sed -e "s/a/b/g" < one-line-250-mb.txt It fails with: sed: couldn't re-allocate ...
3
votes
3answers
161 views

sed - how to (not) match unmatched brackets

How can I do my operation only on lines that close their brackets? For example I want to change ".is" lines and add a bracket but not for lines with unclosed brackets. so these are changed ...
3
votes
1answer
169 views

Efficient way to print lines from a massive file using awk, sed, or something else?

If I had a plain-text file containing 8 million lines and I want to print out lines 4,000,010 to 4,000,000 to the screen, which would be more efficient: awk or sed? There is no pattern to the text, ...
3
votes
2answers
698 views

Removing unwanted lines in text file

All I need is the sed or awk code to remove the even numbered lines in a text file. Nothing fancy I just need to remove irrelevant data on the even lines of a text file.
3
votes
4answers
196 views

Replace all occurences of \ except \N

I have a file like this: 1,2,subjects,\mat\hs,unix\,\Nato,\N,123,\N I want to replace all occurences of \ with #, except for \N, such that the output looks like this: ...
3
votes
2answers
350 views

How to only substitute lines that match several patterns in sed?

$ echo -e 'CH12\nCH23au' | sed '/^CH/s=^=<b>=' <b>CH12 <b>CH23au I know I can match lines starting with CH by ^CH but how can I match multiple patterns? Example: Input: CH12 ...
3
votes
2answers
707 views

What is the best way to find a list of several strings within a large text file

The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file? I am asking this to all Unix/Linux experts as a general ...
3
votes
2answers
344 views

Parsing a text-file table and aggregating information

I have long text file with the following columns, space-delimited: Id Pos Ref Var Cn SF:R1 SR He Ho NC cm|371443199 22 G A R Pass:8 0 1 0 0 ...
3
votes
2answers
94 views

What is the usage of pattern before substitute command in sed

There is an example in this link about sed: To delete the first number on all lines that start with a "#" use: sed '/^#/ s/[0-9][0-9]*//' What is the benefit of first pattern(/^#/)? It could be ...
3
votes
5answers
4k views

Using sed to convert newlines into spaces

Say I have a shell variable $string that holds some text with several newlines, e.g.: string="this is a test" I would like to convert this string into a new string new_string where all line breaks ...
3
votes
2answers
6k views

sed one-liner to delete everything between a pair of brackets?

I am working with some text that is full of stuff between brackets [] that I don't want. Since I can delete the brackets myself, I don't need the one-liner to do that for me, but I do need a one-liner ...
3
votes
2answers
126 views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...
3
votes
3answers
176 views

extract only the substring after double quotes - grep

I have a file which is as below. <a href="http://firstlink.com" title="title1"> <a href="http://secondlink.com" title="title2"> <a href="http://thirdlink.com" title="title3"> <a ...
3
votes
1answer
85 views

Can someone clarify this Bash script

I'm learning something about Bash scripting, and I came across this example: The task is to write a Bash script that reads a file, given with the first argument, pulls out those records that have the ...
3
votes
3answers
523 views

print lines between start and end using sed

I need to print the lines between two delimiters. For example, given the input fluff1 fluff2 BEGIN good3 good4 END fluff5 fluff6 BEGIN good7 good8 END more fluff I want the output to be good3 ...
3
votes
3answers
77 views

how to form a sed expression containing escaped characters

Given a sed expression (and GNU sed 4.2.2 on ArchLinux) /match/i\tline1\n\tline2 which should insert two tab-indented lines above the match, I find that the escaping of the first character (in the ...
3
votes
2answers
411 views

Colorizing tail output with sed

How would I go about colorizing the output of tail with sed? echo "`tput setaf 1`foo`tput op`" works as expected echo "foo" | sed -e 's/(foo)/`tput setaf 1`\0`tput op`/g' however, does not. What am ...
3
votes
3answers
369 views

Delete a block of lines with a condition on the last line

I have a log-file where at the end of a series of lines you can see if this block is relevant. Now I'm looking for a command like sed to delete de blocks ending with "Content-Length: 0" and beginning ...
3
votes
2answers
259 views

Print row contain maximum and minimum value

I need to read the file (contains 16K rows)and print the entire row if any of columns and all columns contains max value (100) and all columns contain min value (0).The ouput example is given ...
3
votes
1answer
3k views

Only return the matched string in sed [duplicate]

Possible Duplicate: Extracting a regex matched with 'sed' without printing the surrounding characters How do I make this only print test: echo "atestb" | sed -n 's/\(test\)/\1/p'
3
votes
1answer
541 views

What's wrong with this sed command?

This command runs fine: $ sed -e '/foo/{g; d}' myfile But this one has an error: $ sed -e '/foo/{g; a bar}' myfile sed: -e expression #1, char 0: unmatched `{' What's wrong with it?
3
votes
2answers
942 views

simple command to strip header and footer from a file

I want a simple little command to strip an XML-Header and Footer from a file: <?xml version="1.0" encoding="UTF-8"?> <conxml> <MsgPain001> ...
3
votes
2answers
4k views

Sed: Replace N first occurrences of a character

I am looking to replace the 5 first occurences of the whitespace character per line inside a sed script. Here's what I have so far sed -e "s/ /;/" -e "s/ /;/" -e "s/ /;/" -e "s/ /;/" -e "s/ /;/" ...
3
votes
2answers
2k views

Make sed ask for confirmation before each replacement?

Is there a way to make sed ask me for confirmation before each replace? Something similar to 'c' when using replace inside vim. Does sed do this at all?
3
votes
3answers
1k views

Substitute part of text file using bash script

I'm writing a shell script (bash) to fetch and build several bits of software. The script also writes several small config files and needs to alter a couple of pre-existing config files. What is the ...
3
votes
2answers
214 views

what is the meaning of this sed command?

sed -e 's/72;/72, next_val = 0x11111111;/;/@/{h;s/test/next/g;x;G}' fmt_vuln.c > fmt_vuln2.c could you tell me about what is the meaning of ;/@/{h;s/test/next/g;x;G}?
3
votes
2answers
114 views

How to convert *text* to {\i text} with sed?

I would like to replace every occurrence of *text* into {\i text}. text *text* text *text* text *text* *text text text* text should become text {\i text} text {\i text} text {\i text} {\i text ...
3
votes
2answers
1k views

sed: multi-line replace of config block

I have some configuration files that basically look like (...content...) # BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY (... more content ...) # END DYNAMIC BLOCK (... even more content ...) Now, in ...
3
votes
2answers
2k views

Removing Color Codes From Output [duplicate]

Possible Duplicate: Removing control chars (including console codes / colours) from script output I'm working on a script to work alongside a program that I'm writing. What i'm trying to do ...
3
votes
3answers
253 views

Sed processed file displays differently in vi vs cat

I can't tell if sed is mucking my file up. In vi or less it displays properly, but cat and more insert other characters. why are they showing up differently I am on a redhat linux system with a ...
3
votes
6answers
1k views

Text file look-up by column

I have a file in this format: [#] OWNER_NAME NAME SIZE [6] Robottinosino Software 200 [42] Robottinosino Ideas worth zero 188 [12] ...
3
votes
3answers
1k views

Using sed to edit lines in a file with a variable

I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
3
votes
1answer
696 views

Chinese characters instead of Latin being written to file

When I run sed like this and print to a console everything is fine: sed '/Q/{ s/Q//g r /Users/ericbrotto/Desktop/question.txt }' Commision.txt But when I do this and output t a file: sed '/Q/{ ...
3
votes
2answers
165 views

Why isn't this sed regex matching?

I've recently started learning sed. I did $seq 10 | sed '/[^049]/d' I was expecting 4 9 10 as output. But I got 4 9 Where am I making mistake in understanding this regex?
3
votes
2answers
1k views

sed or tr one-liner to delete all numeric digits

So, I have this textfile, and it consists of mostly alphanumeric characters. It's a standard document. But since I copied it and pasted it from a PDF, there are page numbers in there. I don't much ...