Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

0
votes
1answer
315 views

globbing, sed, or awk html files

I wanted to create a script that I could check .xhtml files for images and add alt tags if needed. While searching I found regular expressions and did man on glob but I'm unsure on where or what to ...
0
votes
3answers
1k views

Split and print

I have tab-demlimted text file with 50,000lines. I want to split the first column and print as shown in output.txt Input.txt rt|371443144|mb|MN556661.1| 2200443 A rt|371443344|mb|MN556645.1| ...
2
votes
2answers
1k views

extract last match from logfile till end

Using sed, grep, awk, or ruby, what's a simple way to find the last match of a string in a larger log file, and send every line following it (inclusive) to another file? Example: grep -A222 ...
1
vote
1answer
635 views

Linux + replace STRING/WORD in file according to rule

My target is to replace any word/string/Any characters in file with new word/string according to the following rules: If numeric characters on the left side of word/string then we not replace the ...
2
votes
4answers
1k views

How to rename files with sed and csv

I'm new with Linux, sed, and awk but I don't mind challenging myself on new ideas. That being said, I understand the purpose and how to use rename and sed for a common event such as adding a $date or ...
3
votes
3answers
538 views

Script matching literal pattern over multiple lines?

I have a multi-line string in the variable $PAT. $PAT must be searched for within the file $FILE. If $PAT is in $FILE, it needs to print the file with $PAT removed. If $PAT is not found, then print ...
0
votes
4answers
514 views

Sed regex - include original matching

INPUT: dsfgsdf8gfsd 2011.06.26. v iudsfg98sdfg sosdufgsdfg 2011.06.27. h 8xdofguiosdfg jdasfhasd89fa 2011.06.28. k ydsfgsdgsdg dsfgdsfzfszgh 2011.06.29. sze ds9fgisdfgsdfg asdfasdfasddf 2011.06.30. ...
1
vote
1answer
197 views

Display words in color

I am looking for a tool which takes a file in input and a word to search. It should display the file with color the words if it corresponds to the search. Like grep --colors but displays all the ...
3
votes
2answers
717 views

Colorizing tail output with sed

How would I go about colorizing the output of tail with sed? echo "`tput setaf 1`foo`tput op`" works as expected echo "foo" | sed -e 's/(foo)/`tput setaf 1`\0`tput op`/g' however, does not. What am ...
16
votes
4answers
23k views

How to manipulate a CSV file with sed or awk?

How can I do the following to a CSV file using sed or awk? Delete a column Duplicate a column Move a column I have a big table with over 200 rows, and I'm not that familiar with sed.
2
votes
4answers
8k views

Remove newline from unix variable

I have a variable whose value is found using sql query. I want to remove the new line charcater from that variable since I want to concatenate this variable with the other. Below is the code: ...
3
votes
5answers
192 views

grep device name and look for next value :

I have this output from a find command: abc,10.11.13.14,def,1.2.3.4,geh,6.7.54.23 where abc,def and geh are device names and could be of any length and others are IP address belong to devices. ...
7
votes
10answers
2k views

What's a good way to filter a text file to remove empty lines?

I have a .csv file (on a mac) that has a bunch of empty lines, e.g.: "1", "2", "lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum ...
1
vote
2answers
450 views

How can I find matches with sed (or similar) for configuration parameters?

I'm pretty good at using php's preg_match (and similar) commands, and I'm also pretty good with regular expressions, but I don't do very well with sed. I have two shell scripts I'm working on and I'd ...
1
vote
2answers
469 views

Deleting specific lines and duplicates from a 11gb wordlist text file

I have an 11gb wordlist file which is already sorted as each word is on its own line. I need to remove duplicates and lines starting from 077. I guess I need to run sed and sort -u together but I ...
0
votes
1answer
98 views

Sort Column with matching string

I have tab-demlimted file, I want to sort them only for red_ball as follows input.txt id tagid name HH556 EN76 red_ball JH678 EN45 green_ball MH678 CV67 red_ball TY896 ...
1
vote
4answers
8k views

How can I find and replace with a new line?

I have a CSV delimited by commas and I want to delimit it by newlines instead. Input: a, b, c Output: a b c I've written Java parsers that do this stuff, but couldn't this be done with vim or ...
3
votes
2answers
211 views

Replace matched file path with contents of file at the path with sed

I want to write a short deployment script which enters the contents of javascript-files into the single html file. These js files are currently declared as: <script ...
3
votes
2answers
281 views

Extract lines containing PAT1 but not PAT2 with sed

I want to extract the lines containing PAT1 but not PAT2 from a file. For example, with PAT1='dog', PAT2='cat', and the following input: 1 cat chicken 2 bird dog apple 3 dog orange cat 4 cat juice ...
2
votes
4answers
1k views

Replace all but a set of characters in a file with newline

how to delete all characters in file except numbers and "." , each word (numbers/dot) should be in new line in file see example2 the solution can be with sed or awk or ksh syntax remark - ...
5
votes
2answers
3k views

Removing Color Codes From Output [duplicate]

Possible Duplicate: Removing control chars (including console codes / colours) from script output I'm working on a script to work alongside a program that I'm writing. What i'm trying to do ...
0
votes
2answers
297 views

Easiest way to find/replace in a file using a list?

I have a file, foo.txt, and a regexp I want to find in that file. Each time I find the regexp, I want to take a line from another file, bar.txt, and substitute it in for the regexp match I found in ...
0
votes
1answer
547 views

Remove a block of lines between two patterns [duplicate]

Possible Duplicate: Show only text between 2 matching pattern In a bash script using sed how can I remove a block of lines beginning with -pattern a- and ending with -pattern b- where the ...
2
votes
1answer
4k views

sed special character replace not working in shell script

I want to write a script 'test.sh' that will take a user's input and replace all special characters with a '\' + the character. My script: #!/bin/bash echo 'input='"$1" arg=`echo "$1" | sed ...
0
votes
3answers
76 views

Fixing header and print

I have header starting with '>' and I want fix the header by keeping first word and removing other shown as in output.txt and print it input.txt >AGAJ01065549.1 ...
6
votes
2answers
3k views

Replacing missing value blank space with zero

I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt ...
1
vote
4answers
780 views

Replace text between brackets

I'm using awk '{ gsub(/BAR|WIBBLE/, "FOO"); print }' to replace text in data like: SOMETHING [BAR, WIBBLE] SOMETHING [BAR] This gives the desired result of: SOMETHING [FOO, FOO] SOMETHING [FOO] ...
2
votes
3answers
597 views

Regex/pattern matching problem to extract data

What is the best way to remove distances in the following lines and output cdv format. Can this be done reasonably easily on the command line? I'd like to avoid having to write a full-blow script. ...
1
vote
1answer
223 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...
1
vote
5answers
223 views

frequency of occurence

I have a tab-delimited text file, that is around 3K lines long. I want to calculate frequency of data appearing in it, in 10,000 ranges. Input.txt Price 500 1500 10001 15000 17000 ...
3
votes
2answers
887 views

merging two column

input_file (columns are tab delimited, only column 4 contains spaces) input A B C D;<>X;<>Y 0 C D E F;<>Z;<>X 1 output A B C D;<>X;<>Y;<>apple=0 C D ...
7
votes
4answers
4k views

sed on OSX insert at a certain line

So I've been using 'sed' on linux for a while, but have had a bit of difficulty trying to use it on OSX since 'POSIX sed' and 'GNU sed' have so many little differences. Currently I'm struggling with ...
5
votes
5answers
672 views

Sed: extending a string of numbers to a certain number of digits by padding with zeros

I'm currently trying to modify an echoed string of alphanumeric characters by piping it to sed. The area I'm interested in modifying is a block of numbers, which I would like to make 8 characters long ...
1
vote
3answers
2k views

How to rename files using sed?

This is an sed expression that was created by a contributor here. More specifically by @craig-sanders. $FileToMove is set to "2012-09-01 Home insurance quote_zz20.pdf" # use sed to extract folder ...
2
votes
4answers
197 views

Output some sections of a text file [closed]

I have input like the following: >gi|481443199|gb|MH447771.1| 49 0 0 0 1 50 0 0 1 0 >gi|987078004|gb|RTYJ01067061.1| 412 0 0 1 ...
2
votes
4answers
892 views

Change currency format to generic number using sed

I am working on a large csv file with all coma separated entries. The entire document is composed of plain numbers, except for a few columns reflecting currencies that are formatted as text. For ...
4
votes
3answers
822 views

Delete the matching line and several more from a file

I have a text file called file_a.txt. My first command is grep -A 12 ".production =" file_a.txt The output is a few block. Each block of string contains 13 rows I specifically want to delete all ...
1
vote
2answers
193 views

Complicated variable content failed to get into sed

The first line is variable ann hold quite complicated value. The reason I use the second line is because I don't want to change anything in the third line col1content. To change replacement, I just ...
3
votes
3answers
461 views

Sed processed file displays differently in vi vs cat

I can't tell if sed is mucking my file up. In vi or less it displays properly, but cat and more insert other characters. why are they showing up differently I am on a redhat linux system with a ...
2
votes
5answers
82 views

Generate “item tag” pairs from “item,item,…|tag” lists [closed]

The characteristics of the input data: Each line consists of two fields, separated by a pipe '|', where the first field is a comma-separated list of items, and the second field is a tag. Input ...
3
votes
1answer
329 views

reading input in sed

i'm new to shell scripting. I'm learning little by little. I'm more familiar with php which is working toward my disadvantage right now. Anyway, i'm writing a shell script and i read in input with ...
1
vote
3answers
4k views

bash merge 2 lists removing duplicated

I have an embedded linux system using Busybox (OpenWRT) - so commands are limited. I have two files that look like: first file aaaaaa bbbbbb cccccc mmmmmm nnnnnn second file mmmmmm nnnnnn ...
3
votes
6answers
215 views

Extracting help message from script itself

I want that my help message be extracted from the script itself. #!/bin/bash # # foo - do things # Author: John Doe <jhon@doe> # ---------------------------------------------- # SYNOPSIS # ...
3
votes
3answers
2k views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 5l0 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
3
votes
2answers
860 views

Substitute pattern within a file with the content of other file

I have a text file (devel.xml). I added the word REPLACETHIS to it in order to replace this string with the content within a different file (temp.txt). The closest thing I have is this: sed -i -e ...
4
votes
2answers
516 views

Passing parsed output of sed to find (in this direction)

Well, I think you can find dozens of questions on this platform how to pipe find output to sed, but I haven't found anything for the reverse direction so far. What I want to do is modify my input, and ...
6
votes
5answers
1k views

resolve all ip addresses in command output using standard command line tools

I have several log files that contain a bunch of ip addresses. I would love to be able to pipe the data through a program that would match and resolve ip addresses. I.E. cat /var/log/somelogfile | ...
2
votes
1answer
1k views

Inserting a file into another file using sed

I want to insert the contents of file1 into file2 after a matching PATTERN. I want to do it only after the first occurrence of the PATTERN. I would like to know the modification I need to make to ...
2
votes
3answers
4k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
2
votes
2answers
2k views

Replace the shortest match of a string pattern

I have this string: update mytable set mycol=myvalue where mycol=yourvalue; I need to convert it to: insert into mytemp select * from mytable where mycol=youvalue; I can accomplish it like this ...