Sed is a command-line stream editor for filtering and transforming text.

learn more… | top users | synonyms

4
votes
1answer
28 views

Reindexing a large CSV file

I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed). I have a large CSV file (200+ GB) with rows that ...
0
votes
0answers
41 views

what is hexa dump and how to handle in unix?

I am trying to extract some portion of text from the file by using awk command, The file is separated by comma. awk -F, '{print $1 FS $3 FS $6}' sample.csv But always I am getting wrong output , ...
2
votes
2answers
53 views

Add leading zeros until all lines are the same length and subsequently insert a character every three digits using sed

The aim is to add leading zeros until all lines are the same length and subsequently insert a character every three digits using sed. Input 12345,1s4c3v6s3nh6 123456789,9h5vgbdx34dc 12,7h4f45dcvbgh ...
1
vote
3answers
73 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
-3
votes
0answers
23 views

How to extract strings from a given input file and store them in two different files in a column wise manner

Given below is the input file pattern: *********************************************************** Search URL: ...
1
vote
0answers
17 views

Create files from templates, substitutions in middle of other words

Is there a way to convince m4 to replace a macro in the middle of a word? I would like this file, day.m4: define(Sat,Cat)dnl Saturday and this command: m4 day.m4 to produce this output: ...
1
vote
3answers
35 views

Replace word instances with its count attached to it

I have a file which contains word 'name' multiple times ( 250 times ). I need to replace every instance of 'name' with its count attached to it as name_1, name_2, name_3....name_250. Can I do this ...
4
votes
6answers
129 views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
0
votes
4answers
60 views

Use SED or AWK to move rows to columns

I have a report that is generated in rows and I need to move it to columns. The number of rows per record will always be the same. This will be in a BASH script on Linux, so I would like to use ...
1
vote
3answers
40 views

sed match pattern N times

I want to split a file into chunks with 2 words each. $cat tmp word1 word2 word3 word4 word5 word6 word7 $sed -e 's/word. word. /&\n/g' tmp word1 word2 word3 word4 word5 word6 word7 $sed -e ...
2
votes
2answers
93 views

Sed — replace a character in a matched line in place?

In a file containing lines like this one: # lorem ipsum blah variable I would like to remove # (comment) character in the same line that contains a specific string, in place. Is sed good for this? ...
1
vote
1answer
15 views

Sed / awk script for a non-standard apache access log ( Oracle Weblogic )

I have hit a wall with my limited sed scripting abilities, and i wonder if any of you guys could help me out. I have a non-standard apache access log with the following format: #Version: 1.0 ...
3
votes
1answer
52 views

Extract csv files from path to multiple files in bash

I have a files multiple file path for backup purpose ,now i want to filter it based on ".csv " fromat. I have tried this. filesall=(/home/abc/allfiles/*) files=$((${filesall[@]}|grep -F ".csv")) ...
6
votes
2answers
87 views

How come sed is so fast?

I thought about whether this question is suitable for SE or not, I hope you agree it is. Some time ago I asked on SE how to find text in files and leave the file with only the matching lines that ...
2
votes
2answers
31 views

Change path in sql script with sed

I'm trying to change the path in a oracle sql script from a bash script. read -e -p "Enter the ORACLE SID: " -i "orcl" ORACLE_SID_VALUE sed -i "DEFINE tsdir = '/u02/app/oracle/oradata/orcl';/c\DEFINE ...
2
votes
2answers
32 views

Use awk to find first occurrence

I have a log file that gets updated when my script runs. The script will insert "Start of script" text when it starts and "End of script" text when it finishes running. I am trying to capture the ...
2
votes
0answers
38 views

How can I make sed not append a newline character? [duplicate]

How can I make sed not append a newline character to an input stream that was missing one? If I can't, what's the quickest way to remove a newline character from a variable? A code sample was ...
0
votes
3answers
36 views

How to Print certain Fields if a condition is met

I have a big Input file as shown below. If "PRIMER_LEFT_NUM_RETURNED" is not equal to 0 then print the corresponding SEQUENCE_ID, SEQUENCE_TEMPLATE,PRIMER_LEFT_NUM_RETURNED, PRIMER_RIGHT_NUM_RETURNED, ...
5
votes
5answers
83 views

Replace data at specific positions in txt file using data from another file

I have a text file in the below format: $data This is the experimental data good data This is good file datafile 1 4324 3673 6.2e+11 7687 67576 2 3565 8768 8760 5780 8778 "This is line ...
0
votes
4answers
46 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
3
votes
3answers
105 views

Replacing string in all files found by grep. Can't get it to work

I'm trying to do what title says. So I find all needed files with grep: grep --include=\*.{php,ini,conf,sh} -ril -P "'([\d\w\-\_\.]+)(@domain.com)'" '/var/www_data/somepath/' Now I assume it's ...
3
votes
4answers
42 views

Need to remove - (Minus) sign at the end in number from many columns and insert it on columns removed

I have a huge csv file which has - (Minus) sign at the end of numbers. Now I need to move these - minus signs to begin of the numbers, i.e I need to remove the last character if it has '-' sign and ...
4
votes
4answers
140 views

extract lines according to a particular column

I have a CSV file like CK,ck XYZ,xyz,xyzs ABC,abc,abcs PQR,pqr,pqrs LMN,lmn, IJK,ijk, I have to check if something is written in column 3 except whitespace, then display the whole line.the output ...
2
votes
7answers
74 views

How to replace the third character in number to other character

IPS parameter include diff IP address as: IPS="192.9.200.12-14 172.17.200.12-89 12.21.1.10-25 127.0.0.1-127" I want to replace the third “.” In the IP address number to “,” How to achieve this by ...
1
vote
1answer
24 views

how to change a complete line with sed c option

The sed command below is supposed to change the country for Sergio Oliva to Cuba from USA, but as you can see in the output that it's still USA. The book I'm using uses this example to illustrate ...
1
vote
2answers
39 views

Using sed with herestring (<<<) and read -a

I use sed with <<< and read to assign all words in a string to variables. What I do is: read -a A0 <<< $(sed '2q;d' /proc/stat) Hence, sed reads the second line of the file file ...
4
votes
3answers
598 views

Replace “o” with “0” in large wordlist and save the original word?

I have a large text file with words. Every word is on a single line (typical wordlist). I want to replace all characters "o" with the number "0" in every word, and the new formed word to be pasted on ...
2
votes
4answers
242 views

Only cat from specific line X (with a pattern) to other specific line Y (with a pattern)

A little extended problem from "cat line x to line y on a huge file": I have a huge file (2-3 GB). I'd like to cat/print only from the line having "foo:" to the line having "goo:". Assume that "foo:" ...
1
vote
1answer
23 views

Cannot redirect output from sed

I'm piping output from clock through sed to remove leading zeroes from numbers. It looks like this: clock -sf 'S%A, %B %d. %I:%M %P' | sed 's/\b0\+\([0-9]\+\)/\1/g' That works fine and produces the ...
5
votes
4answers
329 views

How to add a carriage return before every newline?

I have a file that only uses \n for new lines, but I need it to have \r\n for each new line. How can I do this? For example, I solved it in Vim using :%s/\n/\r\n/g, but I would like to use a script ...
0
votes
0answers
52 views

Can sed and awk take in multiple files?

grep can search in multiple files, including those files recursively under a specified directory. This makes grep able to find which file contains a pattern. Can awk and sed also? It seems that they ...
2
votes
2answers
41 views

Sed find and replace with slashes

I am having trouble using sed find/replace with slashes in the find parameter. Sample contents of test.sh str=$(sed -n '/^user:/p' /tmp/shadowtest) # find "user:" line in /etc/shadowtest ...
2
votes
2answers
31 views

Live replacement, while viewing log using LESS

It's very convenient to monitor logs using less command - you see what is going with your system as it happening. The bad thing is that my logs are optimized for grep tool: each atomic action is ...
1
vote
1answer
27 views

awk: Columns are not printed when concatenating strings is passed as command string

I've found some anomaly when writing a script. The following examples works as expected: $ echo 123 | awk '{print $1 456}' 123456 $ sh -c "echo 123 | awk '{print $1}'" 123 But the following ...
0
votes
1answer
45 views

Unix sed interpreter | replace a specified character between two words but only replace if the character has a succeeding specified character also

previously, i posted a question on how to replace a specified character between two words and it has been answered perfectly. It works, but to minimize the effort, i would like to add something on my ...
1
vote
3answers
128 views

Replace specified character between two strings?

i have something to accomplish. i need to replace all occurrence of & inside or between <ex> </ex> to #. actual example below: a & b & c <ex> a & b & c ...
2
votes
5answers
146 views

Remove multiple lines after a pattern match but only after pattern match

I am trying to remove all instances of a pattern match from a file if it matches a pattern. If there is a match, the (complete) line with the matching pattern and the next line get removed. The next ...
0
votes
1answer
44 views

Is there a way to create a new file and insert text into this file all with bash? [closed]

I need to change and create several configuration files for some kiosks at work, is there a way to do this with a single bash script? I've heard about SED but the documentation is confusing, could ...
1
vote
3answers
30 views

Concatenate Sequences under a Heading

I have a file like File1. With a heading for example ">Contig1" and I want to concatenate all the sequences under the respective heading. File1: >Contig1 ACCCCATACATCCAAA NNNNNNNNNNCATACA ...
0
votes
5answers
107 views

print only rows that has latest non empty cell that starts with the word “Failed” on a CSV file in Linux [closed]

I want to print the table only if non-empty latest column that starts with the word "Failed"on a CSV file in Linux.. My input - Name,Mon,Tue,Wed,Thu,Fri,Mon Linux,Successfull Fullbackup; Failed ...
1
vote
1answer
119 views

Pattern replacement

Please can anyone explain to me in brief the syntax of this command. Explanation in terms of how actually this command is written significance of each and every character used. I know the output of ...
2
votes
4answers
91 views

sed load substitute from file

I'm attempting to replace a pattern in a file with the complete contents of another file using sed. Currently I am using: sed "s/PATTERN/`cat replacement.txt`/g" "openfile.txt" > "output.txt" ...
3
votes
1answer
46 views

Using sed to replace only the first two regex catches with different strings

So I'm trying to change two instances of a regex pattern to two different strings with sed like: sed -E "s/\\t([A-Za-z]|[-])+$/\\t`get_addr.py -1`/" /path/to/file.ext sed -E ...
3
votes
3answers
48 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
8
votes
7answers
361 views

How can I “merge” patterns in a single line?

I am doing grep and sed and I get 2 lines of a file that I am interested in. How can I get these lines in a single line ending with the new line character? Now I am getting: pattern1 pattern2 ...
0
votes
1answer
48 views

Add text to end of every line

I have tried a lot of commands, like this: sed -i '' "s/$/<@string>/" txt.txt > txt2.txt This makes txt2.txt empty, why?!? sed 's/$/<@string>/' txt.txt > txt2.txt This does ...
-1
votes
1answer
22 views

How do I change the Old Directory path to New Directory path using SED command

I have the old path it contains the some files. /home/ubuntu/Desktop/test-dir Now I want to change the test-dir to new-test-dir directory. /home/ubuntu/Desktop/new-test-dir How do I change ...
1
vote
2answers
32 views

How can I replace ^I into tab spaces in vim editor for .java file?

When I open my .java file in vim, I could see a couple of lines prefixed with one / more ^I characters. It looks like tabs in Eclipse that has got converted into ^I. I would like to replace a single ...
3
votes
3answers
122 views

Parse multiple sections of data into seperate files

I have a concatenated log file with multiple logs inside that I'm trying to parse out into individual log files (I will later rename them to the date/time of each). Each log is separated by "--- LOG ...
3
votes
4answers
79 views

How to remove text matching specific patterns from a file

I want to compare two files to check for differences between them, fileA and fileB. fileA is like a template file and fileB is the file which I want to compare to it. Anytime I find a difference I ...