sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

4
votes
3answers
55 views

Inserting a text between 2 patterns

I have a .xml file which contains the below <controller-host></controller-host> <controller-port></controller-port> <tier-name></tier-name> <node-name></...
2
votes
3answers
510 views

Shellscript to remove particular tags in HTML file

Can anyone please guide me with writing Shell Script command for following scenario. I have an html file with all tags in a single line like this, <html><head></head><body>&...
3
votes
3answers
91 views

How can I remove lines under a sub heading in a file using linux commands

I have a file as below . In which I want to remove lines under a particular sub heading alone. I tried using some sed and awk commands but I couldn't get it. Can any one help to crack this with some ...
0
votes
3answers
48 views

sed - How do I match this pattern?

I'm fairly new to using sed and I am having trouble finding out how to remove some of the first line of text from each block. Here is my original data: "s220823vaps2512": { "hostname": "...
1
vote
2answers
732 views

How can I print the second to last line of many files into one file? [duplicate]

I have many CSV files in one directory which have various lengths. I'd like to put the second to last line of each file into one file. I tried something like tail -2 * | head -1 > file.txt, then ...
0
votes
1answer
74 views

\1 - what does this sed action mean?

I am using this sed command to extract a json value for a key, say MyKey.. sedPattern='s/^.*"myKey":"\([^"]*\)".*$/\1/' For a json string like, {"myKey" : 1, "yourKey": 2} It returns what i need,...
4
votes
3answers
4k views

How to grep lines between start and end pattern? [duplicate]

I have a file which is having following content: zdk aaa b12 cdn dke kdn Input1: aaa and cdn Output 1: aaa b12 cdn Input 2: zdk and dke Output 2: zdk aaa b12 cdn dke I could use below ...
0
votes
2answers
152 views

Prepend lines with file prefix using sed or awk

How do I change a sample input file so as to get something like the output file below? Input file: /tmp/logs 10/13/15 00:19:13 templogs1 10/13/15 00:19:13 dummylogs 10/13/15 01:19:12 test 10/13/15 ...
0
votes
2answers
96 views

sed command to comment out specific block of code containing search string [closed]

I have a listener.ora file for Oracle in which I need to comment out a block of lines specified by a search string, e.g., >(DESCRIPTION= >>(ADDRESS_LIST= >>>(ADDRESS=(PROTOCOL=tcp)(...
1
vote
1answer
49 views

stream editing tools: output what's scraped

I was just wondering if it was at all possible when using different stream editing and text processing/manipulation tools; such as grep, cut, sed, awk, perl etc. Are there ways to output (in the form ...
0
votes
0answers
25 views

In sed, what's the best way to apply blocks of commands to multiple addresses?

If I have a block of sed commands I want to apply to multiple addresses that aren't an address range, what is the best way to do this without repeating the block separately for each address? The only ...
0
votes
2answers
2k views

Shell Script to remove double quotes within the column value

I have a input text file with 10 columns, while processing this file, in one of the middle column, I am getting this type of data.I required column value to be as below: Input Column value: "This is ...
4
votes
2answers
168 views

bash command that outputs the result of a previous pipe

I've got a text file outputted from a WebSpider. The Spider extracts all sentences from a given list of URLs. What I need to do is then process this file and find all lines that contain more than 65 ...
2
votes
4answers
155 views

Only leave longest line when first column is duplicate

INPUT: user@server:~/bar/foobar$ SAT=$(date -dsaturday +%Y-%m-%d); SUN=$(date -dsunday +%Y-%m-%d) user@server:~/bar/foobar$ awk 'BEGIN {FS="'^'"} {print $1"\t"$3"\t"$6}' STs.csv STt.csv | egrep -w "$...
4
votes
3answers
197 views

Remove long string from files

Files on my server have been corrupted. I want to remove a 13000 character string from all PHP files containing it. The string looks like: ?php if(!isset($GLOBALS["\x61\156\x75\156\x61"])) { $ua=...
0
votes
2answers
42 views

need to replace end of column with a character if it does not already exist

I am trying to use awk but can use anything else that will work lets say i have a file with these lines ewhueh wnlknlweknlnw/ qwejmdpdhhd 47578 enhkine enoekjohnkjiof elkjehbjuwekjuk 88989 ehnjkine ...
0
votes
3answers
97 views

sed command not replacing a text with newline character

The Unix sed command doesn't recognize a backslash escaped n \n as a newline character. For instance, the command sed "s!,!\n!g" is replacing commas in the input text with the letter n and not a ...
0
votes
1answer
198 views

SED find and replace element in filename with incremental values

A sample filename is Sample-0.0.1-SNAPSHOT.jar (result of a maven build in jenkins). My goal is to run a shell script that executes on failure and what that script does it adds incremental numbers to ...
0
votes
1answer
41 views

Can you replace displayed text with something else in command window?

I am trying to manage user accounts. They are basically just numbers, but I would rather see user last names on the screen. Is it possible to automatically replace displayed text with something else? ...
6
votes
2answers
477 views

Delete multiple lines in a file

Usually I find my answers in the previously answered questions, but not this time (or I don't understand the answers and can't modify them to do what I want them to do), so I'm asking my first ...
1
vote
1answer
59 views

sed - simple regular expression replacement not replacing

I'm trying to use sed to remove any subdomain prefix from a list of hostnames, but it's not replacing anything when I use a regular expression. Here's the simplified example that's not working: echo ...
4
votes
6answers
172 views

Make longer strings from substrings keeping new line between each string

I have a file that contains strings as: GTACGACGGAGTGTTATAAGATGGGAAATCGGATACCAGATGAAATTGTGGATCGGTGCAAAA GTCGGCAGATATCGTTGAAGTCATAGGTGATTATGTTCAATTAAAGAAGCAAGGCCGAAACTAC ...
3
votes
3answers
118 views

Passing variables to the sed command,

We have a file containing 6 columns. FILEDESCRIPTOR . DESCR00001 FILEDESCRIPTOR . DESCRIPTIVENAME FILEDESCRIPTOR . LSTUPDNAM FILEDESCRIPTOR . LASTUPDATENAME FILEDESCRIPTOR . LOCAT00001 ...
0
votes
1answer
48 views

Search for a pattern in a file and move it to the new files one by one

I am trying to search for a particular pattern in a file and move it to a new files one by one. I am a noob and I tried sed -n 's/CREATE PROCEDURE/,/END ;/p' File It is again printing the same file. ...
0
votes
1answer
26 views

Using xargs to generate links

I have a file that looks like this: abc,123,x xabc,345,xy cabc,6234,ab abgc,12,yo jabc,9991,to I need to generate a list of links: http://www.mywebsite.com?param1=abc&param2=123&param3=x ...
1
vote
4answers
244 views

Trying to grep url from html source in .txt file using sed

i have been able to use the code below to grep lists of urls from html source before, but for some reason it's not working for this specific example. grep: grep -1 box-download shareit1.txt|sed 's/&...
1
vote
2answers
185 views

Printing all lines of a file with multiple columns in a shell script

I have a file with 4 columns with the below format: name;user_id;dob;date_of_joining I want to print all the variables in below output via shell script The employee "$name" is having "$user_id" ...
2
votes
2answers
57 views

Extract content between tags

I have a file with data as below I want to extract the lines and write to another files. The o/p should be I want to do this in a unix shell script.Can someone hellp me on this. I know this can be ...
1
vote
4answers
125 views

Add a space or tab before the last character in each line

I have a file with a list of values that end with a letter. Is there an easy way to add a space or tab before the last character of each line to separate the number from the letter? 44A 354T ...
12
votes
6answers
6k views

How to delete the last column of a file in Linux

I want to delete the last column of a txt file, while I do not know what the column number is. How could I do this? Example: Input: 1223 1234 1323 ... 2222 123 1233 1234 1233 ... 3444 125 0000 5553 ...
3
votes
3answers
176 views

How do I create uniform columns from text containing both tabs and spaces?

In the simplest form, let's say I have an auto-generated file called file.txt. The contents of file.txt appears as follows: Source Destination Maximum To Maximum ...
1
vote
2answers
61 views

How do I stream edit progress-like output?

I'm having trouble editing some progress output. In this case it's Postgres' pg_basebackup, but it's similar to other progress output like curl and wget. Take for example the output of this: ...
0
votes
3answers
44 views

how to add a row name column into a file in linux?

I Have a huge file (50000 rows) looks like : input.txt: 123315334262625363534 265343426272282827262 224343536625242536363 635352627273635373732 363353533637383838327 635342422325337474524 I want ...
1
vote
3answers
239 views

Help with awk / sed shell script

I have to make a script using the info in the following table (fake info) AnimalNumber,DOB,Gender,Breed,Date-moved-in IE161289240602,04/02/2010,M,AAX,20/07/2011, IE141424490333,13/01/2009,M,LMX,21/09/...
6
votes
4answers
448 views

Find 2nd Occurrence of string from the end of file

I have a log file that looks like this, Another thousand lines above this I 10/03/15 12:04AM 42 [Important] 4th to last I 10/03/15 04:31AM 42 (534642712) [1974,2106,258605080,0,0,32817,30711] I 10/...
2
votes
1answer
431 views

How can I append an incremental count to every line of a text file?

I have a large document with over 500000 lines like this car milk dog And I want to count every line and add the number in the end of line and the final document to look like this car1 milk1 dog1 ...
1
vote
4answers
152 views

Find the smallest numbers in the second column corresponding to index values in first column

I have some very large tables and I need to extract specific rows. I am illustrating the task using a simple example. Say, I have weighed a number of apples, bananas and oranges. I need to extract the ...
0
votes
2answers
422 views

Quick way to delete a line from an extremely large file [duplicate]

I have a 67GB .csv file (I know... I know...) I need to remove the 4,125,878th line from the file as it is corrupt. My CSV parsing tools won't ignore or skip it. I could use sed to do something ...
0
votes
1answer
53 views

how to extract some rows in a data file by comparing with another data file? (merging) [duplicate]

I have 2 different file : first file: id1 19501112 id2 19831207 1d4 20001212 id6 20000101 id10 20061213 id11 20081212 id12 20100109 id13 20101218 id14 20101212 id15 20111111 id16 20150919 id20 ...
4
votes
5answers
760 views

How to split a file based on information within a column in linux

I would like to separate those rows where the value in second column is smaller than a specific value. example input: id1 19501112 id2 19831207 id3 20001212 id6 20000101 id10 20061213 id11 20081212 ...
1
vote
2answers
59 views

How to make + work in sed's regexps on FreeBSD?

Consider the following typescript: $ freebsd-version 10.0-RELEASE-p5 $ echo ' found' | sed -n '/[[:blank:]]\+/p' $ echo ' found' | grep '[[:blank:]]\+' found When I do this on Arch Linux with GNU ...
0
votes
1answer
33 views

Increase a number using ssh

Suppose 3 hosts called host1 ,host2, host3 I want to replace the string 5801 on file /etc/FAKEFILE using +1 for every host,with sed(but perl,awk also ok) using ssh. Must return 5802 on host1 5803 on ...
1
vote
2answers
40 views

extract part of the text from all matching lines

I am running a for loop against set of servers to find the cert expiry. Output of the cert command will be below for each server: line 1 line 2 line 3 Valid From: Tuesday, 29 September 2015 10:40:13 ...
1
vote
2answers
26 views

Duplication of each line while changing some value within each copy of a line differently in linux

I Would like to have each line in a file repeated a two number of times while in in first copy of line any value of 3 be changed into 2 and any value of 4 be changed into 1, and in second copy of each ...
2
votes
4answers
148 views

Grepping pieces of a string to form a new string

I am parsing some log files and have grepped out the errors. Each line looks something like this: CreateOrder_hostname1.domain.com_201509291530_tee.log:2015-09-29 15:42:06,715:ERROR :Thread-...
18
votes
1answer
1k views

How does the command sed '1!G;h;$!d' reverse the contents of a file?

My question is related to the sed-specific solution given in this answer for this question of reverse grepping. The sed/grep solution that I am unable to decipher is the following one: sed '1!G;h;$!...
0
votes
2answers
84 views

select and move the contents to another file

I have a file with the below contents. All I need to do is filter the ALTER script that has UNIQUE keyword in it and move it to a different file. I tried using sed -n '/ALTER TABLE/,/UNIQUE("ID");/p' ...
2
votes
2answers
106 views

Cleaning PHP exploits from infected files using sed

Recently my server has been a target for a series php exploit attacks and using ClamAV I was able to identify many infected files. The catch here is that there are legitimate files that are infected ...
0
votes
0answers
38 views

How to restore sbatch array job output file names to their original names

In order to submit array jobs using Slurm, I've renamed hundreds of files with incremental numbers using the following command: COUNT=1 for file in ./*; do mv "$file" "${COUNT}" let COUNT++ ...
1
vote
2answers
191 views

Find and replace a string if particular pattern is found in a line

All I need to do is, search for CCSID in this file, wherever it finds CCSID, the CHAR in that line should be replaced with NCHAR and VARCHAR in that line should be replaced with NVARCHAR2. I tried ...