sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

9
votes
4answers
8k views

Concatenate lines by first column by awk or sed

How can I use awk in the following situation? I want to concatenate lines that start with the same column. Only the first column is kept after the join (in this case aaa, www, hhh). The file may be ...
9
votes
6answers
9k views

Want to substitute only first occurence with sed

Original file claudio antonio claudio michele I want to change only the first occurrence of "claudio" with "claudia" so the file result claudia antonio claudio michele I have tried sed -e ...
9
votes
6answers
3k views

How to get multiple lines out of a file by a regex?

How to get multiple lines out of a file by a regex? I often would like to get multiple lines / modify multiple lines by a regex. A example case: I am trying to read part of an XML/SGML file (they ...
9
votes
2answers
7k views

Is there a way to detect null bytes (␀, NUL, \0) in sed?

Related to another question, in order to fuzzily detect binary files, is there a way to detect ␀ bytes in sed?
9
votes
3answers
2k views

Print a line only if the next line does NOT contain a particular match

I am trying to search a log file for logged activities that did not complete. For example, I log a "Starting activity for ID 1234..." and if successful, the next line will be "Activity 1234 ...
9
votes
2answers
2k views

Deleting all C comments with sed

I am trying to write a script that will delete all comments and everything in between inside C files in my current directory. I've been using sed, and this is what I have so far: sed -i '/ * [^()] ...
9
votes
3answers
11k views

How do I grep without leading whitespaces?

I'm greping through a large codebase, and leading whitespaces and tabulation seem to be quite annoying. Is there any way to get rid of it? grep -R "something" ./ E.g, instead of: foo/bar.cpp: ...
9
votes
1answer
468 views

What is the point of using multiple exclamation marks in sed?

POSIX sed documentation said: A function can be preceded by one or more '!' characters, in which case the function shall be applied if the addresses do not select the pattern space. Zero or ...
8
votes
6answers
12k views

How to grep for text in a file and display the paragraph that has the text?

Below is the text in the file: Pseudo name=Apple Code=42B state=fault Pseudo name=Prance Code=43B state=good I need to grep for "42B" and get the output from the above text like: Pseudo ...
8
votes
6answers
262 views

Grab certain contents of a file

So I know tools exist for this problem because I've heard about them, but I don't know what they are. I want to do something like filter out all data but the usernames in /etc/passwd. For example, I ...
8
votes
8answers
882 views

Split text file into lines with fixed number of words

Related, but no satisfactory answers: How can I split a large text file into chunks of 500 words or so? I'm trying to take a text file (http://mattmahoney.net/dc/text8.zip) with > 10^7 words all in ...
8
votes
8answers
3k views

awk or sed to lowercase/uppercase only one character in string?

Is there a way how to uppercase/lowercase only one character in some string? Input Example: syslog_apr_24_30 syslog_mar_01_17 Desired output: syslog_Apr_24_30 syslog_Mar_01_17 Note please the ...
8
votes
4answers
17k views

Processing bash variable with sed

Been banging my head off a wall on this bash variable LATLNG contains a latitude & longitude value in brackets like so (53.3096,-6.28396) I want to parse these into a variable called LAT and ...
8
votes
4answers
5k views

How do I write a sed one-liner to add a character after every third character?

So, I have a string that looks like this: AUGGCCAUGGCGCCCAGAACUGAGAUCAAUAGUACCCGUAUUAACGGGUGA And I want to split the string into 3-character chunks delimited by a '+' sign. ...
8
votes
6answers
2k views

How to append Line to previous Line?

I have a Log file which need to be parsed and analysed. File contains something similar like below: File: 20141101 server contain dump 20141101 server contain nothing {uekdmsam ikdas jwdjamc ...
8
votes
7answers
20k views

Print odd-numbered lines, print even-numbered lines

I want to print the odd-numbered and even-numbered lines from files. I found this shell script which makes use of echo. #!/bin/bash # Write a shell script that, given a file name as the argument ...
8
votes
6answers
3k views

Delete lines beginning with #

How do I delete lines beginning with a #, given that there can be whitespace on the left and right of the #? # Master socket provides access to userdb information. It's typically
8
votes
3answers
3k views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
8
votes
2answers
456 views

How is the inplace modification of a file being done?

What does the "inplace" modification of a file e.g. via sed -i or perl -i mean? My question is about how this inplace modification is done. Is the file copied the modification is done in the copy and ...
8
votes
7answers
1k views

How can I “merge” patterns in a single line?

I am doing grep and sed and I get 2 lines of a file that I am interested in. How can I get these lines in a single line ending with the new line character? Now I am getting: pattern1 pattern2 ...
8
votes
6answers
49k views

How can I use variables when doing a sed?

I want to do: cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh in my program. But I want to use variables, e.g. old_run='old_name_952' new_run='old_name_953' I have ...
8
votes
3answers
7k views

Regex alternation/or operator (foo|bar) in GNU or BSD Sed

I cannot seem to make it work. GNU sed documentation says to escape the pipe, but that doesn't work, nor does using a straight pipe without the escape. Adding parens makes no difference. $ echo ...
8
votes
3answers
486 views

sed - how to capitalize every 3rd word?

Given: main_east_library main_west_roof main_north_roof minor_south_roof How can I used sed (specifically, not awk, tr, etc.) to create: main_east_Library main_west_Roof main_north_Roof ...
8
votes
4answers
32k views

How to add a carriage return before every newline?

I have a file that only uses \n for new lines, but I need it to have \r\n for each new line. How can I do this? For example, I solved it in Vim using :%s/\n/\r\n/g, but I would like to use a script ...
8
votes
4answers
14k views

grep — removing text after delimiter token

I have a file in which I need to eliminate everything after the first ; on every line. So a file like this: sdfsdsdf; fsdfsddf;sdfsd; Will result in this: sdfsdsdf fsdfsddf I have looked into ...
8
votes
2answers
16k views

sed one-liner to delete everything between a pair of brackets?

I am working with some text that is full of stuff between brackets [] that I don't want. Since I can delete the brackets myself, I don't need the one-liner to do that for me, but I do need a one-liner ...
8
votes
7answers
1k views

How to efficiently split up a large text file wihout splitting multiline records?

I have a big text file (~50Gb when gz'ed). The file contains 4*N lines or N records; that is every record consists of 4 lines. I would like to split this file into 4 smaller files each sized roughly ...
8
votes
4answers
3k views

How to perform a sed in-place substitution that only creates backups of files that were changed?

I ran the following to replace a term used in all files in the current working directory: $ find . -type f -print0 | xargs -0 sed -i'.bup' -e's/Ms. Johnson/Mrs. Melbin/g' This performed the word ...
8
votes
4answers
3k views

sed behaves different on FreeBSD and on Linux?

I use both Linux and FreeBSD (specifically, I use Debian Linux and PC-BSD), and I found something weird about sed. I frequently need to convert "tab separated values" files to "comma separated ...
8
votes
2answers
6k views

Removing Color Codes From Output [duplicate]

Possible Duplicate: Removing control chars (including console codes / colours) from script output I'm working on a script to work alongside a program that I'm writing. What i'm trying to do ...
8
votes
7answers
461 views

Convert rows to columns

I have a file that includes details about VMs running in a hypervisor. We run some command and redirect the output to a file. And the is data available in the below format. Virtual Machine : OL6U5 ...
8
votes
3answers
242 views

Locate postion then make a change using sed

This script uses sed to change all "" to "new stuff". How would one change just the "" after the yyy: using sed or anything else? cat >sample.txt <<EOF xxx: "" yyy: "" } EOF sed ...
8
votes
3answers
3k views

How to use [\w]+ in regular expression in sed?

I am on Windows, but I guess my question is still rightly placed here. C:\Users\User>grep --version GNU grep 2.6.3 C:\Users\User>sed --version GNU sed version 4.2.1 I noticed that the ...
8
votes
4answers
7k views

Remove files older than 5 days in UNIX (date in file name, not timestamp)

I want to delete log files which are older than 5 days from a directory. But deletion should not be based on the timestamp of file. It should be based on the name of file. For Example todays date is ...
8
votes
4answers
497 views

How to use sed to manipulate continuously streaming output?

I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight ...
8
votes
1answer
2k views

Is there an option to make sed fail if pattern not found?

I need to find and replace some patterns in some files, but I need it to return 1 or something if a pattern is not found. Can I do this with sed alone or do I need to check whether the pattern exists ...
7
votes
6answers
14k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
7
votes
8answers
4k views

Grep rest of line…after match

I have a file containing only two lines, with the following structure: $ cat /tmp/pwpower.log 000D6F0000D34227, -114.10 000D6F0001A405C4, -130.09 The values are power values of my solar plant. ...
7
votes
3answers
2k views

How to remove a specific character in a string but only if there are no numbers in that line in Linux

I've been stumped with this seemingly simple-to-fix problem.. well, for a while. Here's example output from the file I need to edit: $cat file George Washington Geneva Convention 123,281,029 USD ...
7
votes
2answers
37k views

Using sed to get rid of characters < > ,

I have a file with lines as follows: ... ... <230948203[234]>, ... ... <234[24]>, ... .. I would like to use sed to remove the characters < , and > from every line I tried using ...
7
votes
4answers
4k views

Replace a string containing newline characters

With the bash shell, in a file with rows like the following ones first "line" <second>line and so on I would like to replace one or more occurrences of "line"\n<second> with other ...
7
votes
4answers
25k views

How to get only filename using sed

How can I get only the filename using sed? I've this out_file=$(echo $in_file|sed "s/\(.*\.\).*/\1mp4/g") But I get the path too /root/video.mp4, and I want only video.mp4.
7
votes
3answers
105 views

Remove certain characters from multiple files with whitepsaces

I'm trying to rename a bunch of music tracks in a directory, but I got this error: When moving multiple files, last argument must be a directory This is the script: for file in * ; do mv ...
7
votes
3answers
24k views

Sed : Replace pattern on every second occurence?

Is there a way to tell sed to replace the pattern on every second occurrence? Or at least on every second line? (Sure it's possible with a script, but I was asking myself if sed can do it to). ...
7
votes
7answers
587 views

Removing numeric values in certain columns whilst keeping minus signs?

I put my hands up to being a complete beginner, so apologies if I'm doing this wrong. Say I have the following data frame that continues indefinitely horizontally and vertically with negative numbers ...
7
votes
6answers
547 views

reverse order of paragraphs in file

I have a file containing text in paragraphs (lines with text separated by one or more empty lines). I would like to reverse the order of paragraphs (i.e. last paragraph will become the first, ...), ...
7
votes
4answers
366 views

Matching and pasting to line

So I have 2 very big text files, consisting of lines like so: First: Robert:Dillain:Other:Other:Other Julian:Brude:Other:Other:Other Megan:Flikk:Other:Other:Other Samantha:Minot:Other:Other:Other ...
7
votes
2answers
18k views

Why does sed outputs “char 53: unterminated `s' command”

Why this bash script ssh $SERVER bash <<EOF sed -i "s/database_name: [^ ]*/database_name: kartable_$ME" $PARAM_FILE exit EOF output -> sed: -e expression #1, char 53: unterminated `s' ...
7
votes
6answers
10k views

sed - remove the very last occurrence of a string (a comma) in a file?

I have a very large csv file. How would you remove the very last , with sed (or similar) ? ... [11911,0,"BUILDER","2014-10-15","BUILDER",0,0], [11912,0,"BUILDER","2014-10-15","BUILDER",0,0], ...
7
votes
4answers
2k views

How to split long sed expression into multiple lines?

sed 's/[long1][long2]/[long3][long4]/' file.txt I would like to split this command onto multiple lines - f.x. something like this: sed 's/ [long1] [long2] / [long3] [long4] /' ...