sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

3
votes
4answers
330 views

Split the input file with line breaks

I am trying to insert line breaks on a file based on the same names which happens to be the second field in my file. So, my input file is like below. 17412193|name1|organization ...
3
votes
2answers
3k views

Why `sed` does not return exit status if regex does not matched ?

I'm trying to search and replace some line from files, but when I run sed if that regex not match then sed does not return any status code, So instead of adding one more condition with grep, is there ...
3
votes
3answers
245 views

Extracting unique piece of text from file with awk, sed or grep

From the output of pactl list sink-inputs, I need to grab the sink input number for VLC. Before that, I'm trying to extract the piece that contains the output for only VLC. All the methods that I ...
3
votes
2answers
125 views

Extract text including parens

I have some text like this: Sentence #1 (n tokens): Blah Blah Blah [... ... ...] ( #start first set here ... (other possible parens and text here) ) #end first set here (...) (...) Sentence #2 ...
3
votes
3answers
177 views

sed syntax to remove 2 characters only when it is box and space

I have a sed expression to remove the first 2 characters from all the lines in a file. sed 's/^..//' raw.txt > converted.txt However, the problem is some lines are large and get appended to the ...
3
votes
4answers
1k views

Replacing pattern after nth match is found on each line?

I have a file containing lines: india;austria;japan;chile china;US;nigeria;mexico;russia I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd ...
3
votes
5answers
2k views

Remove lines based on pattern by keeping first n lines

I need to remove lines from a text file based on pattern but I need to keep the first n lines of that pattern. Input % 1 % 2 % 3 % 4 % 5 text1 text2 text3 output %1 %2 text1 text2 text3 I used ...
3
votes
7answers
1k views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
3
votes
3answers
419 views

sed fails to remove newline character

I've been using sed for quite some time but here is a quirk I came around with, which I am not able to resolve. Let me explain my problem with the actual case. Scene#1 printf "ls" | xclip ...
3
votes
3answers
931 views

Need to insert single quotes in text file for use as SQL query using sed

I have created a line of text that looks like this: INSERT INTO radcheck(id, username, attribute, op, value) VALUES (,,00:23:32:c2:a9:e8,Auth-Type,:=,Accept); I want to use sed to make it look like ...
3
votes
1answer
447 views

Fastest `uniq` tool in linux

I have large text file (1.5 G), I want to know what the fastest and more reliable tool in linux is. I usualy use: awk '!x[$0]++' file.txt But when I use htop command I see my memory usage is ...
3
votes
6answers
2k views

Remove last comma from a bash for loop generated string

I am dealing with a situation where I need to create a comma separated list from an array into a heredoc and remove the last comma. I am using bash for piped into sed which is erasing all commas ...
3
votes
4answers
9k views

extract only the substring after double quotes - grep

I have a file which is as below. <a href="http://firstlink.com" title="title1"> <a href="http://secondlink.com" title="title2"> <a href="http://thirdlink.com" title="title3"> <a ...
3
votes
3answers
722 views

Easily get a particular column from output without sed or awk

Is there a quicker way of getting a couple of column of values than futzing with sed and awk? For instance, if I have the output of ls -hal / and I want to just get the file and directory names and ...
3
votes
2answers
385 views

Extract lines containing PAT1 but not PAT2 with sed

I want to extract the lines containing PAT1 but not PAT2 from a file. For example, with PAT1='dog', PAT2='cat', and the following input: 1 cat chicken 2 bird dog apple 3 dog orange cat 4 cat juice ...
3
votes
5answers
233 views

How to replace only the third occurrence of the file in unix using only sed command?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...
3
votes
2answers
461 views

Shell script to assign values from a lookup table is too slow

I'm trying to assign some variables from a lookup file with a shell script. I have something working but it seems unnecessary slow. Script: while read line do ...
3
votes
2answers
618 views

Sed — replace a character in a matched line in place?

In a file containing lines like this one: # lorem ipsum blah variable I would like to remove # (comment) character in the same line that contains a specific string, in place. Is sed good for this? ...
3
votes
5answers
3k views

How to use a shell variable inside sed's s command? [duplicate]

we know that we can get the value of a variable using $ sign: x=3 echo $x 3 Is there other ways that we can use to get the value without using $ sign. I'm asking this because $ sign is a special ...
3
votes
3answers
295 views

Rearrange text file by prepending data rows with header row

I have a file whose content looks like this: 2009 150 0 0 0 0000 75.316 0.0400390625 0.00007 0.00000 0.8980 76.216 0.0400390625 0.00007 1.00000 0.9046 77.217 0.0400390625 ...
3
votes
1answer
2k views

grep on line before the match plus the match

zzzzzzzzz aaaaaaaaa bbbbbbbbb & ccccccccc & ddddddddd hhhhhhhhh eeeeeeeee fffffffff & ggggggggg & in the above line, what i want is to grep/sed/awk (any method is fine) line that ...
3
votes
2answers
304 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
3
votes
3answers
3k views

Delete XML node containing certain element

I want to remove all Placemarks from a KML file that contain the element <tessellate>. The following block should be wholly removed: <Placemark> ...
3
votes
4answers
5k views

How to get all lines between first and last occurrences of patterns?

How can I trim a file (well input stream) so that I only get the lines ranging from the first occurrence of pattern foo to the last occurrence of pattern bar? For instance consider the following ...
3
votes
4answers
475 views

Piping paths with different types of quotes for slash substitution

I would like to use sed to convert a path with backslashes to the same path with forward slashes: E.g. I would like to pipe \\path\to\file\ and obtain /path/to/file None of the following commands ...
3
votes
4answers
967 views

Strip spaces after single capital letters with sed

I am writing a bash script to automatically generate some other files, and I have to format some strings a certain way. Specifically, the last problem I'm having is formatting a string that has ...
3
votes
2answers
7k views

Strip trailing whitespace from files

The answer to removing trailing whitespace with sed has most of the answer, but I want sed -i 's/[ \t]*$//' $1 to be able to take arbitrary number of file arguments as a shell script on the command ...
3
votes
2answers
731 views

How do remove identical lines in one file from another, using sed?

I have two files, one being a superset of the other. I want to remove the identical lines in the smaller files from the larger file. One possible complication is that the lines contain backslashes. ...
3
votes
4answers
155 views

tab separation of the file

I have a file with seq likes this ATCGTTTCTCAGCCTTTTGGCAAGACCAAGTGTAGTATCTGTTCTTATCAGATATTGGTATCTGCAGGCACTCCTGGAACCAAT ...
3
votes
2answers
1k views

Search a string with last two characters as numeric

Suppose I have a file like this: adasfddfd09 dsassd90897 323sdsdsdsd sdddsdf56 dfdf45fdfdf I want to search only those strings which have last two characters numeric with sed, grep and awk.
3
votes
3answers
104 views

How to select 30 files and do a bunch of sed commands on them

I want new copies of the files, so I want to do something like: dir *.rb foreach file make a copy of the file, e.g. 'blob.rb' to 'blob_processed.rb' do the 50 sed commands to process the ...
3
votes
3answers
220 views

checking data in columns when a data or some may be missing or present?

I am not sure if this possible. say i have columns like : Team Colour Game Rainfall PlayerName XYZ Blue Cricket Yes Kapil suppose i need to search ...
3
votes
5answers
741 views

changing pattern of a text file

A text file has contents something like chair table pen desk Now i want it to be changed and stored in a variable say var as below ('chair','chair'),('table','table'),('pen','pen'),('desk','desk') ...
3
votes
3answers
3k views

Extracting part of lines with specific pattern using awk,sed

I have a question regarding the awk/sed operators. I have a big file which has following set of lines repeated Expression loweWallrhoPhi : sum=-6.97168e-09 Expression leftWallrhoPhi : ...
3
votes
2answers
887 views

Replacing lines containing a pattern with sed

How can I replace a given character in a line matching a pattern with sed? For example: I'd like to match every line beginning with a letter, and replace the newline at the end with a tab. I'm trying ...
3
votes
3answers
329 views

String replacement in file

I have the following file: <?xml version="1.0" encoding="utf-8"?> <!--Generated by crowdin.net--> <string name="test" >- test</string> <string name="test" ...
3
votes
2answers
2k views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...
3
votes
6answers
1k views

sed remove characters from new line

After some processing, I have my input file like below. file1.txt 12345|john|student 43321|jack|professor 78965|alex|lecturer I need to process the above file further and so I need a line ...
3
votes
4answers
762 views

extract the characters before () using grep

I need to extract the characters before the character '('. My file structure is: rose(good)ook daisy()no lilly(bad)fine sunflower()nice I need output file as in the form of: rose ...
3
votes
3answers
2k views

awk remove the rows based on another column value

I have a file like below which I have sorted based on the username field. UserID score UserName 1234 200 Jack 5678 150 Jill 8543 200 Jill 5678 100 John I am trying to remove ...
3
votes
5answers
202 views

grep device name and look for next value :

I have this output from a find command: abc,10.11.13.14,def,1.2.3.4,geh,6.7.54.23 where abc,def and geh are device names and could be of any length and others are IP address belong to devices. ...
3
votes
3answers
2k views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 5l0 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
3
votes
6answers
230 views

Extracting help message from script itself

I want that my help message be extracted from the script itself. #!/bin/bash # # foo - do things # Author: John Doe <jhon@doe> # ---------------------------------------------- # SYNOPSIS # ...
3
votes
4answers
480 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...
3
votes
4answers
538 views

Adding a TeX macro to the beginning of a file using sed (why doesn't my method work?)

I need to add a line to the beginning of a file. The line in question is \def\submit{} I've got a semi-working solution using sed. I don't know sed, but got it from somewhere on the net, but it ...
3
votes
4answers
239 views

Using sed for brackets

I would like to find out, if, in my file, there is a right number of brackets. I can make my file look like (((()))()(())) ((()))() How can I use sed to compute those brackets and print yes or no ...
3
votes
2answers
226 views

Find the line number which contains the pattern with custom delimiter

As in example I'm trying to get line numbers which contains the pattern. My pattern contains slashes so I wanted to add custom delimiter. This simple one works: sed -n '/file/=' temp.txt Using ...
3
votes
4answers
174 views

pattern match and capturing in unix

I have a file like this: A1-1915435_1_MIMAT0000069_MirBase_miR-16_microRNA_1_22-A1-1915435_1_ENSG00000206737_ENST00000384010_U1_snRNA_73_115 ...
3
votes
2answers
313 views

How to add new lines from another file with sed

I'm trying to find the sed command so that I can put filename into filename1. These are my two separate files. INPUT filename has: Cindy 11 22 54 Chester 48 12 84 INPUT filename1 has: Name ...
3
votes
2answers
580 views

Get text between a word and the last line?

According to the answers in Stack Overflow question How to use sed/grep to extract text between two words?, we can get text between two words: sed -n "/first/,/second/p" file But what if I want to ...