sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

3
votes
4answers
5k views

extract only the substring after double quotes - grep

I have a file which is as below. <a href="http://firstlink.com" title="title1"> <a href="http://secondlink.com" title="title2"> <a href="http://thirdlink.com" title="title3"> <a ...
3
votes
2answers
1k views

Basic sed command on large one-line file: couldn't re-allocate memory

I have a 250 MB text file, all in one line. In this file I want to replace a characters with b characters: sed -e "s/a/b/g" < one-line-250-mb.txt It fails with: sed: couldn't re-allocate ...
3
votes
5answers
193 views

grep device name and look for next value :

I have this output from a find command: abc,10.11.13.14,def,1.2.3.4,geh,6.7.54.23 where abc,def and geh are device names and could be of any length and others are IP address belong to devices. ...
3
votes
3answers
2k views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 5l0 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
3
votes
6answers
217 views

Extracting help message from script itself

I want that my help message be extracted from the script itself. #!/bin/bash # # foo - do things # Author: John Doe <jhon@doe> # ---------------------------------------------- # SYNOPSIS # ...
3
votes
4answers
452 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...
3
votes
4answers
535 views

Adding a TeX macro to the beginning of a file using sed (why doesn't my method work?)

I need to add a line to the beginning of a file. The line in question is \def\submit{} I've got a semi-working solution using sed. I don't know sed, but got it from somewhere on the net, but it ...
3
votes
4answers
154 views

pattern match and capturing in unix

I have a file like this: A1-1915435_1_MIMAT0000069_MirBase_miR-16_microRNA_1_22-A1-1915435_1_ENSG00000206737_ENST00000384010_U1_snRNA_73_115 ...
3
votes
3answers
577 views

Extracting part of lines with specific pattern using awk,sed

I have a question regarding the awk/sed operators. I have a big file which has following set of lines repeated Expression loweWallrhoPhi : sum=-6.97168e-09 Expression leftWallrhoPhi : ...
3
votes
2answers
137 views

How to add new lines from another file with sed

I'm trying to find the sed command so that I can put filename into filename1. These are my two separate files. INPUT filename has: Cindy 11 22 54 Chester 48 12 84 INPUT filename1 has: Name ...
3
votes
2answers
493 views

Get text between a word and the last line?

According to the answers in Stack Overflow question How to use sed/grep to extract text between two words?, we can get text between two words: sed -n "/first/,/second/p" file But what if I want to ...
3
votes
3answers
134 views

“Tuning” tr and sed expression

I have MD5 file which lists filename and MD5 sum in the following way: MD5 (file14n0001.xml.gz) = 04eb2524d271077ec8f1a0021779b171 MD5 (file14n0002.xml.gz) = 07e2b29645e9211cce9daf1ae27e7b99 MD5 ...
3
votes
2answers
403 views

Counting comma separated characters in a row

How can I count how many elements are separated with comma in a given column of a file (tab_delimited), using awk or sed? Exemple: AC 1,23,5 DC 2,8,89 Would return: AC 3 DC 3
3
votes
4answers
271 views

Use the sed command to search text file for entries 10000 and under

I'm trying to use the sed command to search a file for all entries 10000 and under but I think this incomplete sed search string might be for entries 10000 and over: sed -n ...
3
votes
2answers
2k views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...
3
votes
3answers
255 views

sed - how to (not) match unmatched brackets

How can I do my operation only on lines that close their brackets? For example I want to change ".is" lines and add a bracket but not for lines with unclosed brackets. so these are changed ...
3
votes
1answer
572 views

Efficient way to print lines from a massive file using awk, sed, or something else? [duplicate]

If I had a plain-text file containing 8 million lines and I want to print out lines 4,000,010 to 4,000,000 to the screen, which would be more efficient: awk or sed? There is no pattern to the text, ...
3
votes
5answers
3k views

How to remove duplicate letters using sed?

Using sed, how can I remove duplicate letters from HEADERS within a text file? NNAAMMEE nice - run a program with modified scheduling priority SSYYNNOOPPSSIISS nice [-n ...
3
votes
4answers
2k views

Removing unwanted lines in text file

All I need is the sed or awk code to remove the even numbered lines in a text file. Nothing fancy I just need to remove irrelevant data on the even lines of a text file.
3
votes
4answers
428 views

Replace all occurences of \ except \N

I have a file like this: 1,2,subjects,\mat\hs,unix\,\Nato,\N,123,\N I want to replace all occurences of \ with #, except for \N, such that the output looks like this: ...
3
votes
3answers
99 views

how to form a sed expression containing escaped characters

Given a sed expression (and GNU sed 4.2.2 on ArchLinux) /match/i\tline1\n\tline2 which should insert two tab-indented lines above the match, I find that the escaping of the first character (in the ...
3
votes
1answer
9k views

Replacing string based on line number

I have a situation where i want to replace a particular string in many files Replace a string AAA with another string BBB but there are lot of strings starting with AAA or ending in AAA ,and i want ...
3
votes
2answers
771 views

Colorizing tail output with sed

How would I go about colorizing the output of tail with sed? echo "`tput setaf 1`foo`tput op`" works as expected echo "foo" | sed -e 's/(foo)/`tput setaf 1`\0`tput op`/g' however, does not. What am ...
3
votes
4answers
571 views

Delete a block of lines with a condition on the last line

I have a log-file where at the end of a series of lines you can see if this block is relevant. Now I'm looking for a command like sed to delete de blocks ending with "Content-Length: 0" and beginning ...
3
votes
2answers
1k views

What is the best way to find a list of several strings within a large text file

The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file? I am asking this to all Unix/Linux experts as a general ...
3
votes
2answers
512 views

Parsing a text-file table and aggregating information

I have long text file with the following columns, space-delimited: Id Pos Ref Var Cn SF:R1 SR He Ho NC cm|371443199 22 G A R Pass:8 0 1 0 0 ...
3
votes
2answers
99 views

What is the usage of pattern before substitute command in sed

There is an example in this link about sed: To delete the first number on all lines that start with a "#" use: sed '/^#/ s/[0-9][0-9]*//' What is the benefit of first pattern(/^#/)? It could be ...
3
votes
2answers
3k views

sed: substitute text with a string that contains special characters

I have a lot of text file with specific tag called "Comment", example I have a file called 3dchess.sh, I need change the tag: Comment=3D chess for X11 to Comment='<span ...
3
votes
1answer
129 views

Can someone clarify this Bash script

I'm learning something about Bash scripting, and I came across this example: The task is to write a Bash script that reads a file, given with the first argument, pulls out those records that have the ...
3
votes
7answers
1k views

Is there a faster way to remove a line (given a line number) from a file?

A related question is here. I often have to edit a large file by removing a few lines from the middle of it. I know which lines I wish to remove and I typically do the following: sed ...
3
votes
2answers
362 views

Print row contain maximum and minimum value

I need to read the file (contains 16K rows)and print the entire row if any of columns and all columns contains max value (100) and all columns contain min value (0).The ouput example is given ...
3
votes
6answers
2k views

Text file look-up by column

I have a file in this format: [#] OWNER_NAME NAME SIZE [6] Robottinosino Software 200 [42] Robottinosino Ideas worth zero 188 [12] ...
3
votes
2answers
2k views

simple command to strip header and footer from a file

I want a simple little command to strip an XML-Header and Footer from a file: <?xml version="1.0" encoding="UTF-8"?> <conxml> <MsgPain001> ...
3
votes
2answers
2k views

sed or tr one-liner to delete all numeric digits

So, I have this textfile, and it consists of mostly alphanumeric characters. It's a standard document. But since I copied it and pasted it from a PDF, there are page numbers in there. I don't much ...
3
votes
3answers
2k views

Substitute part of text file using bash script

I'm writing a shell script (bash) to fetch and build several bits of software. The script also writes several small config files and needs to alter a couple of pre-existing config files. What is the ...
3
votes
3answers
132 views

Strip off word pattern at the end of string using sed

I have following string: ksh -x SCRIPT2.ksh $BUSSINESSDATE & PID=$!; ksh -x SCRIPT1.ksh $BUSSINESSDATE & PID=$!; I want to strip off the last word & PID=$!;. Desired result is: ksh -x ...
3
votes
3answers
111 views

Extract lines between and including 2 patterns

I have a file with data structured as follows 1, p1, p2, p3, p4, p5, p6, p7 2, p9, p10,, 2, q1, q2, q3, q4, q5, q6, q7 2, q9, q10,, 2, r1, r2, r3, r4, r5, r6, r7 2, r9, r10,, 1, s1, s2, s3, s4, s5, ...
3
votes
2answers
158 views

Sed: modify every non-first word repetition for every word in text

Can you help me to do something like that using sed? qq ab xyz ab qq aa ab Becomes: qq ab xyz +ab+ +qq+ aa +ab+
3
votes
4answers
94 views

Replace a marker in a text file with arbitrary new text

I have a file which contains some text and - on a single line - a marker which indicates where new content should be added foo bar %SUBSTITUTE% foo The line line substitute should be replaced by a ...
3
votes
2answers
132 views

How to read needle part of the sed command from a file?

I want to replace a long part of a file with something else preferably by sed but want to read that long part from a file. I already found this: sed -e '/<TEXT1>/{r File1' -e 'd}' File2 from ...
3
votes
2answers
43 views

Help with iterating fieldA in CSV file, where fieldB has specified value

I've been trying to setup a script to run on gitbash, to process a CSV file so each record that has a specific value, the last field is changed from empty ("") to a value that iterates from 1 to 16. ...
3
votes
3answers
1k views

Replacing string in all files found by grep. Can't get it to work

I'm trying to do what title says. So I find all needed files with grep: grep --include=\*.{php,ini,conf,sh} -ril -P "'([\d\w\-\_\.]+)(@domain.com)'" '/var/www_data/somepath/' Now I assume it's ...
3
votes
3answers
64 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
3
votes
2answers
190 views

Replace a backslash with sed

I've been trying to create a script that would replace every occurence of C:\ in two different types of file by /root. My arborescence looks like : - Batch - Place of the script - Conf - Appli1 ...
3
votes
1answer
140 views

Is there a way to pass a variable from an awk statement to a bash function as an argument? [closed]

I am trying to look a for some specific values from (such as 1 or 4 and 2 or 3) a text file using an awk statement in a bash script. If this value is found in the file (within the awk statement), then ...
3
votes
2answers
120 views

How to convert *text* to {\i text} with sed?

I would like to replace every occurrence of *text* into {\i text}. text *text* text *text* text *text* *text text text* text should become text {\i text} text {\i text} text {\i text} {\i text ...
3
votes
3answers
557 views

Script matching literal pattern over multiple lines?

I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print ...
3
votes
3answers
484 views

Sed processed file displays differently in vi vs cat

I can't tell if sed is mucking my file up. In vi or less it displays properly, but cat and more insert other characters. why are they showing up differently I am on a redhat linux system with a ...
3
votes
3answers
2k views

Using sed to edit lines in a file with a variable

I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
3
votes
1answer
857 views

Chinese characters instead of Latin being written to file

When I run sed like this and print to a console everything is fine: sed '/Q/{ s/Q//g r /Users/ericbrotto/Desktop/question.txt }' Commision.txt But when I do this and output t a file: sed '/Q/{ ...