sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

1
vote
1answer
256 views

Using sed output in another script or command

How can I use the output of sed in another script? For example (this doesn't work): sed -n "$COUNTER",1p /domains.csv | wget or sed -n "$COUNTER",1p /domains.csv > /myScript.sh As far as I ...
0
votes
3answers
101 views

linux + awk + file manipulation

I have the following file more file.txt 1 2 3 4 4 2 23 4 5 23 2 3 END_OF_LINE 1 3 7 4 2 23 4 6 7 4 2 1 5 How to print only the fields 4 and 5 and 6 until the line END_OF_LINE , and from ...
5
votes
3answers
777 views

Extracting fixed width records with no delimiter from a single line

I need to extract strings of text from a single file containing one very long line of text with no delimiters. Using the sample line below, these are the following known facts: ...
0
votes
1answer
40 views

rearranging from colum to row in a csv [duplicate]

I have a CSV file that presently has an output like this 1 2 3 4 5 what I need to do is reconfigure the output as follows 1 2 3 4 5 all the entry data on one line, any help would be greatly ...
1
vote
1answer
493 views

Problem with sed on a array containing strings containing spaces

I have a array looking like this: array=("(1 2 3) (123)" "2 31 (231)" "4 5 1 (451)" "(te)xt (1234)") This array is a example. It does not look like this but its structure is the same (the strings ...
1
vote
3answers
88 views

How to add zero (0) in blanks in a big file (example file is below)

Here is a example file where I need to replace blanks with zero A 2 2 2 2 2 B 2 2 2 2 2 C 2 2 2 2 2 D A E B F D R
4
votes
2answers
2k views

Basic sed command on large one-line file: couldn't re-allocate memory

I have a 250 MB text file, all in one line. In this file I want to replace a characters with b characters: sed -e "s/a/b/g" < one-line-250-mb.txt It fails with: sed: couldn't re-allocate ...
9
votes
4answers
174 views

Script to extract selected entries from a bibtex file

I have a large bibtex file with many entries where each entry has the general structure @ARTICLE{AuthorYear, item = {...}, item = {...}, item = {...}, etc } (in some cases ARTICLE might be a ...
4
votes
3answers
195 views

How to only output lines that are younger than 3 months?

cat a.txt 1387111124 ./asfjlasdf.txt 1348681215 ./akdfyxcv.txt So I need this: cat a.txt | SOMEMAGIC 1387111124 ./asfjlasdf.txt The output should only have the lines that are younger than 3 ...
2
votes
1answer
486 views

Oneliner to detect camelcase variables

I would like to insert some style tests as git hooks in order to stick with precise style rules. How can I come up with a oneliner that detects and prints arbitrary camelcase variables like the ...
7
votes
3answers
1k views

Print a line only if the next line does NOT contain a particular match

I am trying to search a log file for logged activities that did not complete. For example, I log a "Starting activity for ID 1234..." and if successful, the next line will be "Activity 1234 ...
2
votes
1answer
64 views

Matching two different lines that are at least N lines distant from each other

I would like to match files that contain try (in one line) and consecutive catch (in another line) at least N lines later (to find very long try-catches and try to get rid of them). I mean, the first ...
0
votes
2answers
315 views

bash script, printing multiple lines that matching to a specific patern

I have a log file and I'm making a script to do some actions. An action is to print a specific area of the log. Every block at the log starts with a specific time stamp and inside the block may have ...
5
votes
5answers
995 views

Pattern matching across multiple lines

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output? I was using something like the following: awk ...
2
votes
1answer
456 views

Getting data section with maximum element

I have some data stored in JSON-like format. { { value1: 14, value2: 12, value3: 1 }, { value1: 4, value3: -1 } } How can I get subsection ...
5
votes
2answers
126 views

Adding corresponding Numbers in a Range

I have a dataset with 30000 lines. I want to add all the corresponding values (i.e. Column 2) between the range 0-50 and then 50.1-100 and then 100.1-150 etc. (presented in column 1), and this ...
-1
votes
2answers
242 views

How can I force sed to continue parsing when it hits an “invalid” character?

I have a big playlist in .pls format (created with deadbeef playing software) which also contains some badly-encoded strings. Whether this is a bug in deadbeef or one of its plugins does not really ...
5
votes
4answers
1k views

sed behaves different on FreeBSD and on Linux?

I use both Linux and FreeBSD (specifically, I use Debian Linux and PC-BSD), and I found something weird about sed. I frequently need to convert "tab separated values" files to "comma separated ...
1
vote
2answers
189 views

Ensure /etc/export contains rule using bash

What's a working example of modifying /etc/export using bash? Example: I want to ensure the following rule is present and valid: /export/home 2400:123::/32(rw,no_root_squash) And it should: Add ...
2
votes
2answers
133 views

stripping out the last two [end] with sed?

I'm working on a script that improves the fluxbox 'menu', and my script has this line: sed -i 's/\[end]//g' ~/.fluxbox/menu;sed -i '/^\s*$/d' ~/.fluxbox/menu but this has the unfortunate effect of ...
3
votes
2answers
2k views

simple command to strip header and footer from a file

I want a simple little command to strip an XML-Header and Footer from a file: <?xml version="1.0" encoding="UTF-8"?> <conxml> <MsgPain001> ...
4
votes
2answers
73 views

Concatenating fields from lines with different numbers of fields

Given input like this: x y a b c t p q w w t a b c d p q r I'd like to concatenate fields from field 3 up to but not including the last one. If field 3 is the last field, I'd like to insert a ...
2
votes
2answers
258 views

sed - remove unmatched “}”, “)” or ']" (only for the line in question)

I have a file with a lot of lines Some of the lines end in unmatched braces, i.e. "}" This is only true for lines that contain ".to" How can I remove the unmatched braces with sed for ALL lines ...
1
vote
2answers
247 views

What does this regex mean?

I was working on the conversion of Unix newline to Windows newline. I tried to unix2dos but it gave me some binary error, so I looked it up and stumbled upon this regex sed 's/$'"/`echo \\\r`/" ...
3
votes
3answers
337 views

sed - how to (not) match unmatched brackets

How can I do my operation only on lines that close their brackets? For example I want to change ".is" lines and add a bracket but not for lines with unclosed brackets. so these are changed ...
1
vote
1answer
280 views

How can sed add text after existing indent?

I have a sed script that has: sed -i '/ *it.*do/! { /\.should/ { s/\.should/)\.to/ s/\(\S\)/expect(\1/ # ...others The last s adds expect( to the ...
0
votes
1answer
149 views

How can I pass in a parameter to sed?

To be more precise - how can I pass in a parameter to a program that then calls sed - how can I 'pass' that parameter along? I have a file 'source_code.sc'. I have a sed script "find_expect": #n ...
1
vote
2answers
61 views

Replace “-from” with “this” in the begining of line

I want to replace "-from" with "this" in the begining of line. This should happen when that line has 'R' at the end and line above it has "D" at the end. For example for the block shown below: ...
2
votes
3answers
237 views

How to delete a specific line and the following blank line using GNU sed?

I have the following text: Start-Date: 2013-11-11 07:43:49 Commandline: apt-get install --no-install-recommends catfish Start-Date: 2013-11-11 11:27:07 Start-Date: 2013-10-16 22:53:02 ...
0
votes
2answers
4k views

sed - how to exclude multiple patterns in the match

I have currently a set of seds that's working well except for one thing. The particular expression I'm having an issue with is: sed -i '/[^}.to]\.to[[:space:]]/ s/\(\S\)/expect(\1/' ...
0
votes
1answer
79 views

How to add parens around text

How can I use sed to replace text and also append to end of line for the match in question? I have a sed file which is doing many substitutions. One of them changes ~= to match( using /=\~/ ...
2
votes
3answers
203 views

sed - how to do several consecutive substitutions but process file only once?

If I am doing several substitutions which need to be consecutive, e.g. sed -i '/^[[:space:]]*browser.*\.should/s/browser/expect(browser/' t1_spec.rb sed -i ...
4
votes
3answers
294 views

how to emulate “replace a b b a” with a sed?

I am trying to emulate a replace command which swaps two characters, with a sed script, but the obvious choice: sed -e 's/a/b/g' -e 's/b/a/g' does no work. Can you point me how to do it correctly? ...
1
vote
1answer
82 views

sed - how to replace either space or underscore_word

Currently I have this sed: sed -i '/[^}.to]\.to[[:space:]]/ s/\(\S\)/expect(\1/' ../_spec_seded/"$file" which adds "expect(" at the start of lines that have ".to" (but not "{.to") How can I make ...
1
vote
1answer
322 views

grep block of lines

I want to grep every block of lines that has starting line with "CK \" at the end of line and ending line that has "D \" at the end of line, I can use sed/awk/grep etc. So far i have tried following ...
5
votes
1answer
1k views

Find all lines between a line matching a pattern and a line matching another pattern

i want to start grepping from the lines that has "CK" at the end of line and stop grepping when the line has "D" at the end. I tried grep "$CK" "$D" file..txt , but it didnt work. Input: kkkkkkkkkkk ...
4
votes
2answers
6k views

How to write text into the first line of a file?

I've been looking around sed command to add text into a file in a specific line. This works to add text after line 1: sed '1 a\ But I want to add BEFORE line1. It would be: sed '0 a\ but I get ...
5
votes
1answer
294 views

Variable in sed

I have a file that contains filenames with absolute path. In the middle of that path, is a variable directory, whenever I hard code the variable part, it works fine, but when I replace it with the ...
4
votes
6answers
3k views

Remove lines that do not start with a pattern from a given set of patterns

I have a file that contains data like this: report aaaaaaaa - .. -th bbbbbbbbb -to ccccccccc .. --. Question: I want to remove any line that does not start with following strings : report ...
2
votes
1answer
330 views

replace text after match

aaaaaaaaa bbbbbbbbb -th ccccccccc_RD \ DDDDDDDD FFFFFFFF -th GGGGGGG_RD \ using sed/grep/awk etc, In the above text, if a line ends with "RD \" , replace "-th" with "-to" at the ...
1
vote
1answer
369 views

Insert a space before the first comma in a csv file

I have a huge CSV file which contains 100000 rows. However, I need to do some processing and so for that I need to have a single space before the first comma. My current data is something like below. ...
3
votes
1answer
2k views

grep on line before the match plus the match

zzzzzzzzz aaaaaaaaa bbbbbbbbb & ccccccccc & ddddddddd hhhhhhhhh eeeeeeeee fffffffff & ggggggggg & in the above line, what i want is to grep/sed/awk (any method is fine) line that ...
3
votes
2answers
2k views

What is the best way to find a list of several strings within a large text file

The short, general question is: In Unix/Linux, what is the best way to find a list of several (about 150) strings within a large text file? I am asking this to all Unix/Linux experts as a general ...
-1
votes
1answer
355 views

Search low values in mainframe files

I am working with mainframe files and it seems that the low values (x'00') exists in the records in the file which throws off my import for fixed block. Does anyone know how to eliminate those ...
1
vote
2answers
457 views

iterate through files ('for file in…) and also include sub directories?

I am using sed to do a bunch of changes to files. I think the question is relevant for any operation that want to iterate through files that match a pattern and include subdirectories both in the ...
1
vote
1answer
193 views

best programming language for I/O intensive tasks [closed]

I need to process "big data" (nearly 1TB which can be partitioned into smaller chunks). Consider several programming languages (Java, Python, C++, Perl) and other scripting options (Perl, awk, sed, ...
4
votes
1answer
247 views

Matching Five Columns in two Files using Awk

I have two Input files. File1: s2/80 20 . A T 86 F=5;U=4 s2/20 10 . G T 90 F=5;U=4 s2/90 60 . C G 30 ...
3
votes
3answers
94 views

Using sed (?) to extract one instance of a string that is repeated in the original string

INPUT: <a href=FII/FOO./>FOO</a> OUTPUT: FOO Q: how can I replace the INPUT with the OUTPUT? The FII and "./>" is CONSTANT. The FOO could vary to any english letter and "-" and "." ...
4
votes
1answer
236 views

Reverse a line in sed?

echo 1234 | sed ' /\n/ !G s/\(.\)\(.*\n\)/&\2\1/ //D s/.// ' I'm unable to comprehend the above sed code. my understanding is: PATTERN SPACE=1234 first ...
1
vote
2answers
184 views

Replace a pattern which ends at newline with the same pattern without the newline

I have a javascript file with two new lines after a particular word "[FACT]", I want to replace it with one newline. I need to do it in bash script. Normally, I would have used sed but it turns out ...