sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

6
votes
3answers
12k views

Filter a .CSV file based on the 5th column values of a file and print those records into a new file

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , ...
4
votes
1answer
911 views

Grep on single line [duplicate]

I have a script that streams a pattern in a single line (no linebreaks). I want to grep stock_ticker in that line and output it as soon as I have found one. Now the script is never ending and is ...
8
votes
7answers
1k views

How to efficiently split up a large text file wihout splitting multiline records?

I have a big text file (~50Gb when gz'ed). The file contains 4*N lines or N records; that is every record consists of 4 lines. I would like to split this file into 4 smaller files each sized roughly ...
-1
votes
1answer
51 views

How to add a word at the first of every lines? [duplicate]

File1: Data will be inserted on Thursday Messages has been sent at 6pm Office hours is from 9am to 6pm All the employees are present Output what I want: DMS Data will be inserted on ...
1
vote
3answers
55 views

Adding a pattern to each field of the following line

I need some help here please. I have the following input: MX04A; DMX04A; DMX04A; LMX04A; LMX04A; -17.2; -15.3; -14.3; -13.6; -16.8; -15.4; -16.0; -15.3; LH36A; DLH36A; DLH36A; -11; -117.2; -11; ...
2
votes
3answers
644 views

Get last set of non-empty lines

I'm running a cron job which should get only the last result of iostat -d 1 2. This requires some parsing: What is the simplest way to get the last set of non-empty lines from standard input to ...
19
votes
11answers
4k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
2
votes
3answers
76 views

Extract specific value from blocks of data

I have the blocks of data which are differentiated by the global Ids as below. I need to extract the last block of data with the highest id value, in the below example, id="82". <gc type="global" ...
1
vote
1answer
77 views

error in awk/sed usage of file manipulation line-by-line [duplicate]

I have a requirement of appending the string " 1" at the end of every line of the file. After the execution of below two commands, an empty file is created. awk '{ print $0 " 1" }' < file.txt > ...
2
votes
5answers
84 views

bash expanding to same value instead of line by line value in sed command

I have a file containing numbers in it say numbers.txt as follows. 9374541632553 51243747879841 32030098896914 84654557358238 11000656847765 I want to convert it to the following format. ...
4
votes
2answers
661 views

Passing parsed output of sed to find (in this direction)

Well, I think you can find dozens of questions on this platform how to pipe find output to sed, but I haven't found anything for the reverse direction so far. What I want to do is modify my input, and ...
2
votes
2answers
58 views

Awk - keep word “example” and “EXAMPLE” with one input

I am attempting to to only keep customers that contain certain words. Here is an example < newname.txt awk 'BEGIN {FS=","} $7~/USA|United States/' > usa.txt The problem I am having is if they ...
2
votes
4answers
185 views

Is it possible, with `sed` or otherwise, to apply a regex transform to an input stream that may not contain newlines?

I have a shell pipeline that contains a sed expression: source | sed -e 's:xxxxx:yyyyy:g' | sink It works but it has a latent flaw because sed works on whole lines. This means that the sink will ...
4
votes
5answers
811 views

How to remove newlines between data of each record which are located between 2 patterns?

I have a large file to parse and reformat, preferably with sed (under bash). The file contains repetitive sequences starting with PATTERN_START and ending with PATTERN_END. These sequences are ...
4
votes
6answers
710 views

Extracting lines based on conditions

Each line in a comma-separated file has 5 fields. a,b,c,d,e f,g,c,i, j,k,c,m,n o,p,c,r,s t,u,c,w, x,y,z,aa,bb How can I extract the lines which have c in the 3rd field and their 5th field is NOT ...
2
votes
3answers
360 views

How to 'drop'/delete characters from in front of a string?

I have a string that I would like to manipulate. The string is H08W2345678 how would I be able to manipulate it so the output is just W2345678? Similarly if the I wanted to drop the last 4 ...
1
vote
4answers
118 views

How to use regex in sed?

I need to revoke the sudo access to the user using sed in scripts. I want to comment the user only if the line contains /bin/su - root Let us consider the file as below, in which I have to comment ...
2
votes
2answers
210 views

How to replace value for a given condition in specific column of file

I have file separetd by pipe | and I want that when column 6 contain letter I print 0758000 in column 7 and when the column 6 contain letter A print 0800000 in column 7, I can not find how to do it!!! ...
5
votes
2answers
21k views

Using sed to extract text between 2 tags

I have a .xml file and I'm trying to do a "groupinstall" on a RHEL6 machine as there are several hundred libraries in that .xml file... (close to 16 000 lines). I'm therefore trying to extract the ...
0
votes
1answer
134 views

Bash : rewriting a file which is read in same line

I'm working on a bash script for partially automating router connection in command line. I'm a newbie in Bash. This is part where I have a problem: #!/bin/bash STRING2="ESSID" MYPWD=$(pwd) sudo ...
1
vote
1answer
70 views

Convert multilines to single line with SED

I need additional help, based on similar question I have located here: Convert multilines to single line Current SED example (sed -e'y/)},={/(((((/' \ -e's/-\([^(I]*\)[^0-9]*\([0-9]*\)[( ]*/- \2 ...
0
votes
1answer
109 views

Find part of String and replace entire string in a file

I have a file named A.xml which contains a strings such as "Ticket_Release1" and "Ticket_V2". I want to find and replace all the strings in the file starting with Ticket_ with something called ...
0
votes
1answer
62 views

sed - find string made of 4 numbers with a preceding character and append character

I'm trying to build a sed command that will allow me to search and replace some values in a large csv file. I have this column which contains years, like 2005 - 2006 2001 - 2003 and so on, but I also ...
0
votes
0answers
68 views

Searching and replacing string using sed

Searching and replacing a string in a file. Input file name - Input.txt Following is the content: DATETIMEOFFSET(20) datetimeoffset(99) DatetimeOffset(10) lots of data1 lots of data2 ...
1
vote
2answers
95 views

Insert multi lines after match using Mac's sed command

I have 2 files:- a.xml <project> <modelVersion>4.0.0</modelVersion> <parent> ... </parent> </project> b.xml <project> ...
2
votes
2answers
57 views

Grouping lines into heterogeneous subsets

I have file with n lines.  (Each line refers to a “question”, and therefore they are labeled Q.1, Q.2, Q.3, ..., Q.n.)  Each line (question) has a “Marks” attribute, which has the value 2, 3, 4, 5, or ...
0
votes
1answer
287 views

How can I find and replace only in a range of columns?

I have a text file with 9267 lines. I want to find "A" with 5 spaces after it and replace it with "A" with 5 spaces before and after it on all lines, but only between columns 67 and 88. Text outside ...
-1
votes
1answer
54 views

String replacement in sed

How to replace BINARY(any integer of 64 bit) with BYTEA: Example1: Input: BINARY(28) Output: BYTEA Example2: Input: BINARY(234) Output: BYTEA
2
votes
3answers
93 views

Read a column from a file and append a specific column of another file

I have two files as such: fileA: Name,Site Name,Product UPEI,Product Name,NMS Flag,Product Model, SW FileB: NE_Name SW ABC 4.4 ASD 4.3 ... ... Note that ...
1
vote
1answer
97 views

To use Pandoc pipe data with AWK

Data ID | Size | Blocks -------|-----------|--------- 206 |80760288 | 157744 207 |80760288 | 157744 328 |241520288 | 471728 Wanted output by AWK ID |BlockSize | Size | ...
0
votes
6answers
1k views

sed to replace with backspace

I have trail.txt, which contains spice netlist. Now I want to replace all expanded parameters to one line only i.e. remove + and append that line to previous line. trail.txt data + net06706 ...
7
votes
4answers
424 views

How to insert lines with numbers from X to Y into another file after line Z?

I created two files: echo -e "1\n2\n3\n4\n5" > 123.txt echo -e "a\nb\nc\nd\ne" > abc.txt I want to get the file 123.txt with the following contents: 1 2 3 b c d 4 5 So in other words ...
4
votes
2answers
70 views

Adding a new column using information from subheadings

Replacing lines containing a pattern with sed Thanks to @don_crissti and @Archemar. I reckon my question is not a duplication of the above question. So here is an example of input. Edit: All ...
3
votes
2answers
1k views

Replacing lines containing a pattern with sed

How can I replace a given character in a line matching a pattern with sed? For example: I'd like to match every line beginning with a letter, and replace the newline at the end with a tab. I'm trying ...
1
vote
3answers
217 views

Swap line containing pattern with next line

I currently have the following code which nicely outputs some information about the wireless networks around me: sudo iw wlan0 scan | sed -nr '/SSID: |signal: |primary channel: /p' | sed ...
2
votes
0answers
248 views

Bash - Convert escape character colored text to BBCode

I want to convert a terminal escape-character colored text file to BBCode colored text. For this i've created an Android logfile with logcat -Cd > /sdcard/logcat.txt. The -C switch adds the color ...
2
votes
2answers
115 views

sed multiple entries in one command to remove data

I am using the following sed command to remove emails that contain hotmail - Is it possible to check for multiple entries at the same time? Preferably load it from list.txt (one entry per line). sed ...
-1
votes
2answers
72 views

Substitute a string containing && with sed [closed]

I've wrote some script for automated technical acceptance tests. Since we're not allowed to write password in clear text, the script I used is associated with many variables, one of which is the ...
1
vote
2answers
35 views

sed remove command - Include upper/lower case

I am using the following command to get rid of lines that contain hotmail sed -i '/^[^\t]*\t[^\t]*hotmail/d' ./file The test data is "foxva****omes****","scott@hotmail.com","8*** ...
6
votes
4answers
1k views

sed ranges aren't always able to match only one line

Consider the following range: 1,/pattern/. If pattern matches on the first line, the range matches for the whole file: $ cat 1.sh #!/usr/bin/env bash set -eu seq 1 4 | sed -rn '1,/'"$1"'/p' $ ./1.sh ...
4
votes
2answers
317 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
3
votes
4answers
8k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
3
votes
4answers
1k views

How to use sed command to modify a file's entry line by line in shell script?

This is my file1: #$BQ #{ VOL @home } #database daba #relation tcdeatid #copy 1 #{ version 0 } #opendb #clear #.lruno := 72 #.infno := 1 #.tid.noel := 101 #.tid.info ...
23
votes
7answers
57k views

How to read first and last line from cat output?

I have text file. Task - get first and last line from file after $ cat file | grep -E "1|2|3|4" | commandtoprint $ cat file 1 2 3 4 5 Need this without cat output (only 1 and 5). ~$ cat file | ...
1
vote
1answer
50 views

Helping with sed - remove specific lines with certain string

These are the contents of the text file: cat dog elephant cat lion cat tiger penguin penguin emperor zebra I want to remove the line contain just cat and retain cat lion and cat tiger lines. There ...
0
votes
1answer
210 views

Replace pattern with line from file using awk/sed [duplicate]

I have a file first.html which contains the following code: <tr> <td class="headerValue">One</td> </tr> <tr> <td class="headerValue">Two</td> </tr> ...
2
votes
4answers
76 views

CSV - Only keep certain entries

I have a bunch of data inside "" and separated by , - Example below of two lines "stampthisandthat.com","GANDI ...
6
votes
5answers
7k views

How to get all lines between first and last occurrences of patterns?

How can I trim a file (well input stream) so that I only get the lines ranging from the first occurrence of pattern foo to the last occurrence of pattern bar? For instance consider the following ...
2
votes
4answers
332 views

Replace regex match with string containing match

I have a file with text as Afghanistan=+93 Albania=+355 Algeria=+213 American Samoa=+1 Andorra=+376 Angola=+244 It has all the country list and its dialing code. I want to replace: ...
1
vote
3answers
82 views

Match beginning of a line with something there?

I want to match the beginning of lines where the line contains something. Command in Perl/Sed 's/^.?/\section{}/g' but the problem here is that it replaces also the first character of sentences. ...