Sed is a command-line stream editor for filtering and transforming text.
3
votes
2answers
573 views
simple command to strip header and footer from a file
I want a simple little command to strip an XML-Header and Footer from a file:
<?xml version="1.0" encoding="UTF-8"?>
<conxml>
<MsgPain001>
...
3
votes
1answer
503 views
Chinese characters instead of Latin being written to file
When I run sed like this and print to a console everything is fine:
sed '/Q/{
s/Q//g
r /Users/ericbrotto/Desktop/question.txt
}' Commision.txt
But when I do this and output t a file:
sed '/Q/{
...
3
votes
2answers
154 views
Why isn't this sed regex matching?
I've recently started learning sed. I did
$seq 10 | sed '/[^049]/d'
I was expecting
4
9
10
as output. But I got
4
9
Where am I making mistake in understanding this regex?
3
votes
2answers
889 views
sed or tr one-liner to delete all numeric digits
So, I have this textfile, and it consists of mostly alphanumeric characters. It's a standard document. But since I copied it and pasted it from a PDF, there are page numbers in there. I don't much ...
3
votes
3answers
75 views
sed + how to remove character/s that start or ended on each number
How do I remove the . character(s) that start in the beginning of each number or end on each number?
Remark – perl one liner also good alternative for sed.
Example input:
.23.12.44.5.
.233.3.3.3
...
3
votes
1answer
42 views
How to edit files in-place while performing operation on the values matching the searched pattern?
Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ?
Ideally a solution ...
3
votes
3answers
166 views
What does . match?
In working with regular expressions, I have been told that a dot character . will match everything.
Except for newlines \n.
Are there any other exceptions? What about the NUL character \0, or the ...
3
votes
1answer
500 views
Cleanly swap all occurences of two strings using sed
Suppose I have a file that contains multiple occurrences of both StringA and StringB. I want to replace all occurrences of StringA with StringB, and (simultaneously) all occurrences of StringB with ...
3
votes
3answers
433 views
Using sed to edit lines in a file with a variable
I need to edit lines in a file using sed. Now the problem is I am replacing a particular pattern with a combination of text and number. This number is a variable which keeps on incrementing for every ...
3
votes
2answers
3k views
sed one-liner to delete everything between a pair of brackets?
I am working with some text that is full of stuff between brackets [] that I don't want. Since I can delete the brackets myself, I don't need the one-liner to do that for me, but I do need a one-liner ...
3
votes
5answers
160 views
Print non-existent files from a pipe input
I have a directory of lots of .PDF and .JPG files.
There should be a .JPG file for each .PDF with the same name.
I'm trying to use a command to find .PDF files that doesn't have a .JPG file.
My ...
3
votes
2answers
80 views
Replace matched file path with contents of file at the path with sed
I want to write a short deployment script which enters the contents of javascript-files into the single html file. These js files are currently declared as:
<script ...
3
votes
1answer
549 views
Substituting the first occurrence of a pattern in a line, for all the lines in a file with sed
Is it possible to do it in one liner?
I have an output like this:
"First line" - Description
" Second line" - Description
"Third line" - Description
" Fourth line" - Description
This input ...
3
votes
1answer
168 views
Use sed to fix broken newlines
I have a folder full of files, each with many hundreds of lines. Many of these files have occasional broken newlines: when I open them in vim, many have ^M^L interspersed where newlines should be.
...
3
votes
2answers
511 views
Insert a multiline string into another string
I need to insert lines into an xml file :
Insert
<one>
</one>
into
<tags>
</tags>
To obtain
<tags>
<one>
</one>
</tags>
I tried this:
...
3
votes
2answers
388 views
sed inline editing fails for multiple sed statements
I want to append #SB# at the beginning of every line where it matches a given string
incron.sh
sed -i -e'/test1/s/^/#SB#/g' file1
sed -i -e'/test2/s/^/#SB#/g' file1
sed -i -e'/test4/s/^/#SB#/g' ...
3
votes
2answers
559 views
sed script to insert line after the last matching line in a file
I'd like to write a script in sed to insert line to a file after all similar lines in the file. So I need to locate the last match for a pattern and insert immediately after it. It's very unlikely ...
3
votes
4answers
597 views
Replacing text from a list of replacements. Added complication: backslashes
I have a file A that contains pairs of strings, one per line:
\old1 \new1
\old2 \new2
.....
I would like to iterate over file A, and for each line perform the replacement (e.g. "\old1 -> \new1") ...
3
votes
2answers
284 views
merging two column
input_file (columns are tab delimited, only column 4 contains spaces)
input
A B C D;<>X;<>Y 0
C D E F;<>Z;<>X 1
output
A B C D;<>X;<>Y;<>apple=0
C D ...
3
votes
2answers
223 views
Change string occurrences in file while respecting DOS new line sequences
I use Visual Studio for my C# development while using Cygwin for some of the tasks that require scripting. Recently I wanted to change all occurrences of string AAA to BBB in my project's files. I ...
3
votes
2answers
287 views
How to skip file in sed if it contains regex?
I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed:
find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+
As ...
3
votes
2answers
217 views
Extracting bigrams with sed
How do I split an entire text consisting of only letters into bigrams.
For example:
odd even
od -> de
dd -> ev
This is what I have so far but it does not produce the desired result.
[some ...
3
votes
2answers
1k views
sed: delete all occurrences of a string except the first one
I have a logfile with timestamps in it. Occasionally there are multiple timestamps in one line. Now I would like to remove all of the timestamps from a line but keep the first one.
I can do ...
3
votes
4answers
237 views
Remove empty configuration section
Files like ~/.config/vlc/vlcrc are 99% junk if you want to version control only the configuration options. I've got a script to remove the comments, but there's a ton of empty configuration sections ...
3
votes
4answers
110 views
Colored output in less when tailing a logfile
I am in dire need of a way to color my less output while reading a file that is constantly being appended.
The file in questions is a Resin servlet container log.
My current "implementation" of the ...
3
votes
2answers
473 views
simple script rotating table
I'm trying to write a simple script, that just rotates csv table.
I mean I have some file:
head1;head2;head3
field11;field12;field13
and so on.
All I want, just to make my file
...
3
votes
1answer
205 views
Relative line offsets in awk (/pattern/+1 or /pattern/-1)
When I want to print lines between two patterns, excluding the lines with those patterns, I can do it in ex using +1 and -1 after the pattern. That is awesome. Is this possible in awk? Right now, I ...
2
votes
5answers
280 views
Can sed remove 'double' newline characters?
I have a document with a lot of empty lines.
How can I remove them when there are 2 or more together.
I tried sed "s/\n\n//" file but it didn't work. No error.
2
votes
4answers
166 views
Output some sections of a text file [closed]
I have input like the following:
>gi|481443199|gb|MH447771.1|
49 0 0 0 1
50 0 0 1 0
>gi|987078004|gb|RTYJ01067061.1|
412 0 0 1 ...
2
votes
3answers
1k views
Extracting column from comma separated text
I have a long comma-separated delimited file with 20K lines. Here's a sample:
"","id","number1","number2","number3","number4","number5","number6","number7"
...
2
votes
4answers
399 views
shorten long lines in a log file
I keep a logfile of a program's output. The trouble is sometimes errors occur dumping very long lines of 7bit ascii data(encoded binary) I don't care to keep. These lines can be 200KB+ long before ...
2
votes
3answers
54 views
How to sed a range of lines?
I have this:
sed -i '/^$/d' temp_spec.rb
which is stripping blank lines and works well. How can I make it only do this for lines 5-999 (or ideally 5 to end-of-file).
I tried:
sed -n5,999 -i ...
2
votes
3answers
637 views
Delete files if contents do not match pattern
There are hundreds of .html files in a directory. I need to delete files that don't contain word bluecar in their contents.
I thought sed with rm would do that, but I don't know how to combine them.
2
votes
2answers
463 views
sed, grep, or tr command that returns only Latin characters from a UTF-8 file
I am working with the text of the 300 tang poems, which unfortunately is a single file containing both Chinese and English. Since I am interested in 'extracting' the English, I hope to use sed, ...
2
votes
3answers
258 views
sed one-liner to delete any line that begins with a digit
Pretend I am working with this file.
The beginning of the first visible line is obscured, so ignore that. 43 is the beginning of the next line, and that line would be deleted.
The next line (which ...
2
votes
2answers
2k views
Show only text between 2 matching pattern
I'm going to submit form using cURL, where some of the contents is come from other file, selected using sed
If param1 is line matching pattern from other file using sed, below command will works ...
2
votes
2answers
1k views
Strip trailing whitespace from files
The answer to removing trailing whitespace with sed has most of the answer, but I want
sed -i 's/[ \t]*$//' $1
to be able to take arbitrary number of file arguments as a shell script on the command ...
2
votes
4answers
2k views
How can I find and replace with a new line?
I have a CSV delimited by commas and I want to delimit it by newlines instead.
Input:
a, b, c
Output:
a
b
c
I've written Java parsers that do this stuff, but couldn't this be done with vim or ...
2
votes
3answers
2k views
Extracting tokens from a line of text
Using bash scripting and grep/awk/sed, how can I split a line matching a known pattern with a single character delimiter into an array, e.g. convert token1;token2;token3;token4 into a[0] = token1 … ...
2
votes
3answers
94 views
Easy way to parse syslog date format
I'd like to write a script that analyses the last week's syslog.
But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial.
Is there a way ...
2
votes
3answers
142 views
Use find + sed + cp to find files and copy them to a directory with a different name
I'm trying to copy a bunch of files named folder.jpg into a folder. The problem is because all the files are named the same thing, I need to rename them in the process. I know I can probably do it ...
2
votes
5answers
74 views
Generate “item tag” pairs from “item,item,…|tag” lists [closed]
The characteristics of the input data:
Each line consists of two fields, separated by a pipe '|', where
the first field is a comma-separated list of items, and
the second field is a tag.
Input
...
2
votes
3answers
411 views
Average rows with same first column
Given a file with two columns:
Id ht
510 69
510 67
510 65
5l0 62
510 59
601 29
601 26
601 21
601 20
I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
2
votes
3answers
146 views
Facing problem with Regex Inside Sed Command
Suppose I run the following commands:
export STR="abcdef.ghijkl.mnopqr.stuvwy.log"
echo $STR | sed 's/\.[^.]*$//'
I am getting the following result:
abcdef.ghijkl.mnopqr.stuvwy
Please help me ...
2
votes
2answers
260 views
sed: get 2 lines from a single one
I have a list of filenames as id-datetime.txt, one each line, where id is always the same and datetime is ordered.
I need first and last datetime so having a sed_script in a variable $script that ...
2
votes
3answers
177 views
Bash commands/script to remove a line from CSV with duplicate in column
I have a lot CSV files that I have combined. However, there are duplicates, but the entire line is not duplicated. I do have a column that I want to use as the criteria to search for a duplicate. ...
2
votes
4answers
92 views
How do I correct both too many and too few space characters?
I have files that are edited by people other than me. They contain lines that look like this:
<E> This is text </E>
I use some Bash shell scripts do a bunch of text replacement ...
2
votes
2answers
250 views
Detect pattern repetition in a file
I want to make sure a file follows the following pattern:
... ... ... foo ... foo_KO ... ...
... bar ... ... ... ... bar_KO ...
The file is extremely long, so, in other words, I want to make sure ...
2
votes
2answers
364 views
Adding 2 lines in text file stanza
I have a text file divided in stanzas.
Each stanza starts with a [keyword] and ends with a blank line. Something like this:
[keyword1]
param1 = value1
param2 = value2
param3 = value3
[keyword2]
...
2
votes
4answers
2k views
Deleting Lines Containing a Specified String in Different Files
Suppose there are two files in web/ named foo.php and bar.php. The 1st line of foo.php is "sdajgeSTRINGdsad" and the 10th line of bar.php is "gdfhu98324STRING". The task is to first locate these two ...
