Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

4
votes
2answers
1k views

delete first line of file only if blank using sed

I have a solution for this in awk: awk '{if (NR==1 && NF==0) next};1' somefile but was unable to find one that worked in sed. E.g., sed -i.bak '/^$/{1,1d;}' somefile ended up deleting ...
4
votes
3answers
322 views

How to parse a file to extract 3 digits numbers kept in a “group number”

I am a beginner, and trying to write a shell script to parse a text file that is extracted from a standardization pdf file. I would like for each test group (identified by Group 0, Group 1... to get ...
4
votes
3answers
6k views

Delete range of lines above pattern with sed (or awk)

I have the following code that will remove lines with the pattern banana and 2 lines after it: sed '/banana/I,+2 d' file So far, so good! But I need it to remove 2 lines before banana, but I can't ...
4
votes
2answers
71 views

Cut a string- Catch: There is file with same name as string in the same path

Input /in/archive/ABC/20140111_ABC_018_PART_001.dat Desired Output: 20140111_ABC_018_PART_001.dat Command used: cut -d/ -f4 "/1a/ftproot/archive/ABC/20140111_PR1_018_SALESDOC_ITEM_001.dat" ...
4
votes
3answers
375 views

using sed to replace pattern with hash values

I want to search the file and replace specific pattern with its hash (SHA1) values. For example, let file.txt has the following content: one S56G one two three four five V67X six and I want to ...
4
votes
1answer
303 views

Unix - How to append text to every line of a file except for the header line/first line

I have a file say abc.txt with following data (real data has many records,say around 200) Sno | Name 1 | Jack 2 | Jill 3 | June Now how do I append text to make my file look like this, ...
4
votes
1answer
178 views

grep: display filename once, then display context with line numbers

Our source code has error codes scattered throughout. Finding them is easy with grep, but I'd like a bash function find_code that I can execute (eg. find_code ####) which will provide output along ...
4
votes
3answers
151 views

Check for, and add, missing timestamps to individual lines in a file

I have a file in the following format: ABCD 01206001022T01YA022T01YA022T07SO 09:20:38 ABCD 01206001022ACION 09:24:40 ABCD 04006001021S01UK 09:24:42 ABCD 7878696621321312 23213213213213 ABCD ...
4
votes
1answer
738 views

Removing leading zeros from DATE

I made an alias of the date command to display date in the following format: 2013.06.14.12.10.02 using this command: alias date = date +"%Y.%m.%d.%H.%M.%S" Everything works great, except i ...
4
votes
3answers
4k views

How to get only filename using sed

How can I get only the filename using sed? I've this out_file=$(echo $in_file|sed "s/\(.*\.\).*/\1mp4/g") But I get the path too /root/video.mp4, and I want only video.mp4.
4
votes
3answers
559 views

Print a line only if the next line does NOT contain a particular match

I am trying to search a log file for logged activities that did not complete. For example, I log a "Starting activity for ID 1234..." and if successful, the next line will be "Activity 1234 ...
4
votes
3answers
238 views

sed one-liner to replace word-medial capitals

I used OCR to turn some scans into plaintext, but unfortunately the letters 'fi' which are commonly joined in some fonts, got read in as capital W's. Now I need to replace all the W's with 'fi', and ...
4
votes
1answer
97 views

Matching Five Columns in two Files using Awk

I have two Input files. File1: s2/80 20 . A T 86 F=5;U=4 s2/20 10 . G T 90 F=5;U=4 s2/90 60 . C G 30 ...
4
votes
2answers
356 views

Passing parsed output of sed to find (in this direction)

Well, I think you can find dozens of questions on this platform how to pipe find output to sed, but I haven't found anything for the reverse direction so far. What I want to do is modify my input, and ...
4
votes
5answers
1k views

extract fields from “tail -f” of a syslog stream

Example line from syslog file: Aug 1 10:25:50 10.10.10.1 id=firewall sn=XXXX time="2012-08-01 14:35:18 UTC" fw=x.x.x.x pri=6 c=1024 m=537 msg="Connection Closed" f=11 n=195273698 ...
4
votes
1answer
222 views

substituting third occurrence of a pattern using sed

Here is a sed command. I want to substitute third occurrence of 00 to 11 echo "0000000000000000" | sed -e 's/00/11/3' The output is: 0000110000000000 But I thought output would be: ...
4
votes
2answers
465 views

Replace one line with STDIN in a Makefile

One Makefile target is supposed to be in charge of: grabbing the output of $ perl Markdown.pl src/index.md (markdown to html) using that string to replace one line, something like CONTENT ...
4
votes
2answers
665 views

Moving average on a log file with awk or other unix utilities?

Scenario: I have a log file that has a few number of entry "classes", like this: R0 dx=0.00500 rb=0.00000 sn=1 3145.88 2.59 0.08 se=21315 id=16190 R0 dx=0.00300 rb=-1.00000 sn=1 3150.40 2.38 0.05 ...
4
votes
1answer
888 views

Cleanly swap all occurences of two strings using sed

Suppose I have a file that contains multiple occurrences of both StringA and StringB. I want to replace all occurrences of StringA with StringB, and (simultaneously) all occurrences of StringB with ...
4
votes
4answers
149 views

Bash Scripting : Printing column data in the same row

I have written a bash script to calculate the size of a PostgreSQL database and print the output along with the date when the script was executed in a text file. The script code is as follows: ...
4
votes
1answer
87 views

Reverse a line in sed?

echo 1234 | sed ' /\n/ !G s/\(.\)\(.*\n\)/&\2\1/ //D s/.// ' I'm unable to comprehend the above sed code. my understanding is: PATTERN SPACE=1234 first ...
4
votes
1answer
59 views

Is strace a suitable tool for finding out where a program wrote to the filesystem?

I'd like to determine where a certain app (as an example, surf) writes to the filesystem in order to make sure it leaves no traces. Without a dedicated sandboxing system this probably isn't easy, but ...
4
votes
3answers
400 views

Multi-line replace

I spent ages trying to figure this out. As it happens, I don't need to do it any more; I found another way. But for the sake of my sanity, I'd like to know the answer anyway. It's quite easy to use ...
4
votes
2answers
336 views

Out of memory while using sed with multiline expressions on giant file

I am currently trying to remove all newlines that are not preceded by a closing parenthesis, so I came up with this expression: sed -r -i -e ":a;N;$!ba;s/([^\)])\n/\1/g;d" reallyBigFile.log It does ...
4
votes
3answers
539 views

How to remove all white spaces just between brackets [] using bash? [duplicate]

Replace text between brackets Input testing on Linux [Remove white space] testing on Linux Output testing on Linux [Removewhitespace] testing on Linux So, how can we just remove all the ...
4
votes
3answers
325 views

Want to search for two consecutive lines with at least n characters

I would like to use sed to search for the first part of a text file with consecutive lines with at least n non-space characters. I would like to print from the first of those lines to the end of the ...
4
votes
5answers
181 views

Multi-line grep beween two tags, only show the last match

I'm having a program that intermingles data segments into log output: log message log message ----BEGIN INLINE DATA---- data data data -----END INLINE DATA----- log messge I'm looking for something ...
4
votes
3answers
533 views

Delete the matching line and several more from a file

I have a text file called file_a.txt. My first command is grep -A 12 ".production =" file_a.txt The output is a few block. Each block of string contains 13 rows I specifically want to delete all ...
4
votes
3answers
450 views

How to skip file in sed if it contains regex?

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed: find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+ As ...
4
votes
1answer
372 views

Are there zero-width assertions in sed?

Is it possible to make lookahead or lookbehind zero-width assertions in sed? I want to emulate Perl's (?=) and family. My sed is not GNU sed version 4.0.
4
votes
1answer
242 views

sed : insert sth before subsequent lines that begin the same, but are not the same

I have a LaTeX file with one glossary entry per line: ... \newglossaryentry{ajahn}{name=Ajahn,description={\textit{(Thai)} From the Pali \textit{achariya}, a Buddhist monk's preceptor: `teacher'; ...
3
votes
4answers
4k views

removing the first and the last character of every line from command line

I am trying to remove the first and the last characters of everyline in a text file and save the resulting truncated version in a new file. Does anyone have an idea about how to do that efficiently ...
3
votes
4answers
209 views

How to Prefix a column values with an apostrophe ( ' )?

I have a CSV file with multiple columns and 1000's of records, I need to prefix all the values of one of the columns (lets say 2nd column) with an apostrophe ' except in the first line or header line, ...
3
votes
3answers
196 views

How to perform an action only on the first line?

sed 's/.\(.*\)/\1/' myfile Say myfile contains: abc def ghi I want to remove the first character only from the first line but the above removes it from all the line.
3
votes
2answers
209 views

What is this sed command doing?

Can someone explain me what this sed command is trying to achieve? sed 's/^[[:space:]]*//;s/[[:space:]]*$// I understand that it is searching for whitespace characters from the beginning of the ...
3
votes
4answers
129 views

Split the input file with line breaks

I am trying to insert line breaks on a file based on the same names which happens to be the second field in my file. So, my input file is like below. 17412193|name1|organization ...
3
votes
4answers
136 views

Rearrange text file by prepending data rows with header row

I have a file whose content looks like this: 2009 150 0 0 0 0000 75.316 0.0400390625 0.00007 0.00000 0.8980 76.216 0.0400390625 0.00007 1.00000 0.9046 77.217 0.0400390625 ...
3
votes
3answers
113 views

sed syntax to remove 2 characters only when it is box and space

I have a sed expression to remove the first 2 characters from all the lines in a file. sed 's/^..//' raw.txt > converted.txt However, the problem is some lines are large and get appended to the ...
3
votes
2answers
174 views

sed: match two lines

I have a text file, where records are split on two lines. I would like to put them on one line. The records look something like this: 2013-07-29 apt-get install git 2013-07-29 apt-get install rsync ...
3
votes
4answers
575 views

Replacing pattern after nth match is found on each line?

I have a file containing lines: india;austria;japan;chile china;US;nigeria;mexico;russia I want to replace all the occurences of semicolon on each line with e.g. ;NEW;, but starting from the 2nd ...
3
votes
5answers
662 views

Remove lines based on pattern by keeping first n lines

I need to remove lines from a text file based on pattern but I need to keep the first n lines of that pattern. Input % 1 % 2 % 3 % 4 % 5 text1 text2 text3 output %1 %2 text1 text2 text3 I used ...
3
votes
2answers
215 views

Extract lines containing PAT1 but not PAT2 with sed

I want to extract the lines containing PAT1 but not PAT2 from a file. For example, with PAT1='dog', PAT2='cat', and the following input: 1 cat chicken 2 bird dog apple 3 dog orange cat 4 cat juice ...
3
votes
2answers
133 views

Join: Two files - but only append last two columns

Given the files: 1.txt 1, abc, 123, 456, 789 2, lmn, 123, 456, 789 3, pqr, 123, 456, 789 2.txt 1, abc, 123, 000, 000 3, lmn, 123, 000, 000 9, opq, 123, 000, 000 OUTPUT.txt ID, NAME, X, ...
3
votes
3answers
114 views

Easily get a particular column from output without sed or awk

Is there a quicker way of getting a couple of column of values than futzing with sed and awk? For instance, if I have the output of ls -hal / and I want to just get the file and directory names and ...
3
votes
3answers
951 views

Delete XML node containing certain element

I want to remove all Placemarks from a KML file that contain the element <tessellate>. The following block should be wholly removed: <Placemark> ...
3
votes
4answers
3k views

How to get all lines between first and last occurrences of patterns?

How can I trim a file (well input stream) so that I only get the lines ranging from the first occurrence of pattern foo to the last occurrence of pattern bar? For instance consider the following ...
3
votes
4answers
319 views

Piping paths with different types of quotes for slash substitution

I would like to use sed to convert a path with backslashes to the same path with forward slashes: E.g. I would like to pipe \\path\to\file\ and obtain /path/to/file None of the following commands ...
3
votes
4answers
640 views

Strip spaces after single capital letters with sed

I am writing a bash script to automatically generate some other files, and I have to format some strings a certain way. Specifically, the last problem I'm having is formatting a string that has ...
3
votes
2answers
3k views

Show only text between 2 matching pattern

I'm going to submit form using cURL, where some of the contents is come from other file, selected using sed If param1 is line matching pattern from other file using sed, below command will works ...
3
votes
2answers
589 views

How do remove identical lines in one file from another, using sed?

I have two files, one being a superset of the other. I want to remove the identical lines in the smaller files from the larger file. One possible complication is that the lines contain backslashes. ...