sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

2
votes
1answer
255 views

replace nth column of a csv file by the help of lookup table

I have a csv file (a.csv) as below (with thousands of lines): NMS_NE,,,,,,,,,,,,,,,,,,,,,,, D,NE Name,IP Address,Site Name,Product UPEI,Product Name,Product Model,NE Status,Serial Number,SW Major ...
-1
votes
1answer
37 views

Get the respective tablenames if it matches particular pattern

I need to list the table names if it finds particular pattern from the below file. CREATE TABLE IMS.ACTIVITY ( "ID" INTEGER NOT NULL , OWNING_TYPE FOR COLUMN OWNIN00001 VARCHAR(20) ...
1
vote
1answer
46 views

append few strings at the start and bottom of string coming from file read

I am working on a shell script which is getting data from another program and I am using that variable value to read the content from file and append few things inside those data at start and bottom: ...
-1
votes
1answer
1k views

How to fix 'command a expects \ followed by text' in my sed? [closed]

I have the following snippet in a sed statement: /pause$/ { N; /pause/d } but I'm getting: $ . sed_shorter_version_one_test_case_html_to_ruby.sh SAD/fullform_SAD sed: 19: " /pause$/ { N; /...
0
votes
1answer
168 views

How to use content address in sed on osx without error “expected context address”

I'm getting this error sed: 2: " /^[[:space:]]*pause$/, ...": expected context address with this line: /^[[:space:]]*pause$/,+1d which works on my debian Ubuntu system. The full file is ...
0
votes
3answers
121 views

How to remove <td> or </td> with sed?

I am successfully using: s/^[[:space:]]*<[/]*tr.*>// to change any lines starting with <tr> or </tr>, removing those tags. However when I try using s/^[[:space:]]*<[/]*td.*&...
6
votes
5answers
1k views

Remove backslash + newline sequences

We are having a file in Linux which contains one line per record, but problem comes when the line contains some new line characters. In this case, the backslash is appended at the end of line and the ...
4
votes
3answers
577 views

sed, convert single backslash to double backslash

I have a json string, which has a potpourri of doubly escaped/singly escaped newline chars. Json parser doesn't allow its string value to have single backslash escapes. I need to uniformly make all ...
0
votes
1answer
563 views

Getting RE error: repetition-operator operand invalid on osx sed

I'm copy a sed script form Ubuntu debian to osx but getting RE error: repetition-operator operand invalid What is wrong? $ . sed_shorter_version_user_extensions_to_ruby.sh sed: 22: " ### DELETE ...
3
votes
1answer
50 views

comparing 2 lists with additional data to be printed

I have a file lets say file1.txt ENSG00000223116 AL157931.1 ENSG00000233440 HMGA1P6 ENSG00000207157 RNY3P4 ENSG00000229483 LINC00362 ENSG00000252952 RNU6-58P I have another file say file2....
3
votes
3answers
2k views

Using grep/awk/sed to sort and combine 2 files

Ok I am running into a problem. I am taking a wifi log that has MAC addresses listed in it. I want to find out what MAC vendors/manufactors for the devices that are connected to my router. I have 2 ...
6
votes
2answers
11k views

Compare two file columns

I have long text files with space-delimited fields: cat file1.txt Id leng sal mon 25671 34343 56565 5565 44888 56565 45554 6868 23343 23423 26226 6224 77765 88688 87464 6848 66776 23343 63463 ...
7
votes
4answers
382 views

Matching and pasting to line

So I have 2 very big text files, consisting of lines like so: First: Robert:Dillain:Other:Other:Other Julian:Brude:Other:Other:Other Megan:Flikk:Other:Other:Other Samantha:Minot:Other:Other:Other ...
0
votes
4answers
103 views

Sed, remove all comments in a file [duplicate]

I have a sed command that removes comments in a file as, sed -i /^#/d /path/to/file This works but not when the comments are indented/have a preceding space. like #this is a good comment ...
7
votes
2answers
3k views

Join multiple sed commands in one script for processing CSV file

Having a CSV file like this: HEADER "first, column"|"second "some random quotes" column"|"third ol' column" FOOTER and looking for result like: HEADER first, column|second "some random quotes" ...
2
votes
1answer
163 views

dpkg: error: duplicate file trigger interest for filename `/usr/lib/gio/modules' and package `libglib2.0-0'

[Reposted from askubuntu.com] I can't install any software. I think this dates from an unexpected power loss a few days ago. When I try sudo apt-get install gparted I see Reading package lists... ...
0
votes
1answer
61 views

Replace a string with “string|string”

I have a pretty weird problem. I did some calculations for bioinformatics, but my downstream apps won't accept my header information in its current format. In order to circumvent this I modified my ...
0
votes
1answer
49 views

How to express variables efficiently for SED?

There are many ways to express arithmetic things: new and old ways. However, I would like to write this line efficiently and well sed -n '$START,$ENDp;$ENDq' < data.tex which output sed: 1: "$...
2
votes
4answers
131 views

How to add hyphens at the first empty line of a text file?

I'm trying to add three hyphens to the first empty line of a given text file, and am trying to use sed for this. My initial file (a MultiMarkdown file) looks something like this: title: "Ch. 1: ...
0
votes
1answer
39 views

How to SED with open boundaries? [duplicate]

I was thinking too complicated the slicing in the How to express variables efficiently for SED? I want to include (a,b), not [a,b] like this don_crissti's command: sed '/begin{document}/,/end{...
-1
votes
2answers
192 views

select portion of file by line interval [duplicate]

I want to practice using head, uniq and cut, for this data here. I know this thread but it focuses too much on cat and external programs. I would like a simpler answer. I want to take interval from ...
0
votes
2answers
38 views

How to get the sed command to only search for full words?

I'm fairly new to Linux and recently started experimenting with the command line. I decided to make a file and called it 'hi.txt'. The contents of the document was: This is hi.txt. hi hi hi I ...
1
vote
2answers
38 views

Reading a variable-sized line and splitting its content into separate lines with text

I have a file with the header (first line) with a variable length; and it comes as: "One"|"Two"|"Three" or "One"|"Two"|"Three"|...|"One Hundred" and want to get do a loop that transforms it ...
0
votes
1answer
97 views

Sed dynamic substitute, the unix way?

Got a bunch of files with domain qualified names in them, looking to replace them by their ip. Can I do that with one line of shell ( bash ) ? Typical input file ( nagiosconf ) : define host { ...
4
votes
4answers
641 views

Replace “/” with “\/”

I have a string /tmp/testing which I want to replace with \/tmp\/testing sed works fine as follows: echo testdir=/tmp/testing | sed 's/\//\\\//g' testdir=\/tmp\/testing However if I try and put ...
0
votes
3answers
122 views

Pulling out a certain value from a string via regex

I've see a bunch of examples, but I just can't seem to make this work. Can grep output only specified groupings that match? for example seems like it should work, but I get either errors or no output ...
1
vote
3answers
61 views

how to remove first occurrence of 0 in mm/dd a date using sed?

I want to remove first occurrence of 0 in mm/dd, e.g. I want to make 09/09/15 look like 9/9/15. I have tried using echo 09/10/15 | sed 's/^.//' but it gives output like 9/09/15. How can I do this?
4
votes
4answers
2k views

sed - Print lines matched by a pattern range if one line matches a condition [duplicate]

I have a multi line log entry format that I need to process. The log looks something like this: --START-- Device=B Data=asdfasdf Lorem=Ipsum --END-- --START-- Device=A Data=asdfasdf Lorem=Ipsum --...
0
votes
1answer
58 views

Fetch second and fourth column from a file

Problem Statement I am not able to get the values in second and fourth column. Tried grep -i "AUGC\|AACN\|AACY" output.csv | cut -d ',' 2,4 DATA success,1234,["madhu",{'first',[{'last',18}]},"I0"...
5
votes
8answers
744 views

awk or sed command to match regex at specific line, exit true if success, false otherwise

I need to determine if a file contains a certain regex at a certain line and to return true (exit 0) if found, and otherwise false. Maybe I'm overthinking this, but my attempts proved a tad unwieldy. ...
3
votes
5answers
770 views

Find text between tab (\t) as a delimiter

I thought this will be simple, but can't find out how to do it. Scenario I have a single .csv file with id_user,text,id_group columns where each column is delimited by tabs such like: "123456789" ...
0
votes
2answers
171 views

Use multiple commands in sed to insert text after string with special characters

I am having a little trouble trying to do a search and replace using sed in a config file. I want to use multiple commands using the -e option to do the following. Step 1: find the first occurrence ...
0
votes
2answers
33 views

Merge labels from lines with duplicate fields

I have a rather large text file(thousands of lines) in following format: 123.123 5 123.123 91 156.456 45 233.185 85 233.185 4 257.448 1 455.456 60 455.456 2 The first column are ...
1
vote
2answers
53 views

Split file in two on line before pattern

This is an almost identical question to split file into two parts, at a pattern but instead of having the lines up to XYZ in file1 in the example file.txt: ABC EFG XYZ HIJ KNL I want to have the ...
0
votes
1answer
21 views

sed - how to refer to the outer match from within a block?

For example: I have a piece of sed as follows: 36 sed $' 37 /^waitForVisible/ { 38 s/this/& and that/ 39 s/waitForVisible/expect/ 40 s/)$/).to be_visible\\\n/ 41 } with ...
0
votes
2answers
155 views

sed: Multiple -e or while loop?

I have a CSV file with tuples of values where I need to change occurrences of one into another in a different (large) file. Thus far, I have done a while read line [...] < foo.csv, essentially ...
0
votes
1answer
32 views

Print specific FULL column only by range

I have a file which have contents like this. Column1 Column2 Column3 Column4 USER1 25 Years xyz zyz 123 abc ...
1
vote
2answers
111 views

Change time format in a file

I have output of a sar commands and the time is not POSIX: Linux 2.6.32-431.29.2.el6.x86_64 (test.server.com) 2015-08-01 _x86_64_ (32 CPU) 12:00:01 AM CPU %usr %nice %sys %...
2
votes
1answer
78 views

Sed: replace text keeping digit occurrence

I have some text like: Blablabla <b>[intlink id="</b>2204<b>" type="page"] Blalala I want replace it for remove the </b> and <b>, keeping the id="number". So it should ...
0
votes
3answers
72 views

print last line of !! command

so I want to be able to print helloworld using the !! command. I tried creating a text file with the text helloworld on it. Then I say cat textFileName It prints helloworld. Then I say !! and it ...
0
votes
1answer
98 views

Need to capture part of file name into variable

I have bunch of CSV files named like so in a directory: s_m_u_supplemental_fields_account_9_4_2015_11_26_Error.csv s_m_u_supplemental_fields_account_9_4_2015_12_3_Error.csv ...
0
votes
1answer
49 views

Sed command with complex pattern and replacement

I have the following two strings saved as variables: str1='ds-cfg-global-aci: (targetcontrol="2.16.840.1.113730.3.4.2 || 2.16.840.1.113730.3.4.17 || 2.16.840.1.113730.3.4.19 || 1.3.6.1.4.1.4203.1.10....
0
votes
3answers
133 views

How to replace contents of each line depending on some formula?

I've been working with bash to get some data output to a multiline file with the data stored in this format for each row: text1 "text2" text3 integer1,integer2 A "space" separates the first four ...
1
vote
2answers
62 views

Remove last comma within a pattern

If I have the following text within a file (somefile.txt): CREATE TABLE "table_name" ( "id" int(11) unsigned NOT NULL, "some_field" varchar(10), ); CREATE TABLE "another_table" ( "id" int(...
8
votes
4answers
27k views

How to get only filename using sed

How can I get only the filename using sed? I've this out_file=$(echo $in_file|sed "s/\(.*\.\).*/\1mp4/g") But I get the path too /root/video.mp4, and I want only video.mp4.
0
votes
2answers
28 views

Delete multiple lines with text and curly braces

I have a multiple files that contains something like: this is a test1 { test 123 test 456 test 789 } this is a test2 { test 123 test 456 test 789 } this is a test3 { test 123 test 456 test 789 } ...
1
vote
1answer
57 views

Remove word after match

Consider the following line of text (taken from a MySQL insert): "description" varchar(255) COLLATE utf8_unicode_ci NOT NULL DEFAULT '', I want to return "description" varchar(255) NOT NULL ...
24
votes
6answers
82k views

How to parse JSON with shell scripting in Linux?

I have a JSON output from which I need to extract a few parameters in Linux. This is the JSON output: { "OwnerId": "121456789127", "ReservationId": "r-48465168", "Groups": [],...
2
votes
4answers
484 views

extract url between 2 strings in a file

I have a file in which each line is like this "372"^""^"2015-09-03 06:59:44.475"^"NEW"^"N/A"^""^0^"105592"^"https://example-url.com"^"example-domain < MEN'S ULTRA < UltraSeriesViewAll (18)"^"...
8
votes
8answers
1k views

Split text file into lines with fixed number of words

Related, but no satisfactory answers: How can I split a large text file into chunks of 500 words or so? I'm trying to take a text file (http://mattmahoney.net/dc/text8.zip) with > 10^7 words all in ...