2022 Developer Survey is open! Take survey.

Questions tagged [sed]

sed is a command-line stream editor for filtering and transforming text.

Filter by
Sorted by
Tagged with
1 vote
3 answers
129 views

uniq - how to ignore IP address?

I have the following: cat /var/log/example.log | cut -d ' ' -f 3,4,5,6 | sort -u | uniq The result has several thousand lines but I would like to cut this down to show the actual activity and to do ...
user avatar
1 vote
2 answers
82 views

Remove duplicates of specific line keeping only the first appearance of each without touching other unspecified duplicates

I'm trying to edit a text file containing several duplicates. The goal is to keep only the first match of a string and remove the rest duplicate lines of the same string. In the example file * Title 1 ...
user avatar
1 vote
5 answers
69 views

include a cent-sensitive character that does not interfere with the delimiter of a csv file

I have a CSV file that I need to include a separator to highlight the cents in that column that doesn't interfere with the ',' delimiter Below is the model of how my current file is: 99999999,2021-01-...
user avatar
0 votes
1 answer
148 views

Match multiple patterns in text file, copy and modify the matched line

I want to match multiple patterns in text files on the same line, copy and modify the matched line. I want to duplicate every line that contains either the string "businessServices" or "...
user avatar
0 votes
1 answer
557 views

Download and sort list of all videos from a youtube channel with youtube-dl

I want to download a list of all videos of a youtube channel and sort it by duration. Here is what I tried with youtube-dl resp. yt-dlp: yt-dlp --skip-download "url_to_channel" --get-...
user avatar
  • 16.7k
0 votes
3 answers
105 views

Best tool to use to extract command output

I want to programatically process the results of nmap output but cannot work out how to take the output and extract just the details of the protocol or port tables shown in the two outputs below. I am ...
user avatar
  • 239
0 votes
1 answer
89 views

Concatenate Columns and Create a Composite Key from a Text File in Shell Script

I have been trying to implement a Feature in my Shell Script but somehow stuck in implementation, need help please. Sample File - f1.txt col1|col2|col3|rev|spt A1|54|tyre|56.89|45.23 B1|54|pole|11.89|...
user avatar
1 vote
2 answers
46 views

Insert charater at left side of matching pattern using sed?

Here is the data file as input. 1 2 2 979 data raw 32j 23 32323 Output: /* 1 /* 2 /* 2 /* 979 data raw /* 32j /* 23 /* 32323 How I can get the desired OUTPUT using sed? Regex for matching pattern is ...
user avatar
-1 votes
1 answer
66 views

awk - is it possible to define my custom new line character/word?

AWK reads and procesess input data line by line. My data has it's own defination of line. So is it possible to configure AWK to consider a char or word i define as end of line. Just like with -F ...
user avatar
  • 554
-1 votes
3 answers
76 views

How can I change lowercases to '-' in the even and odd lines using sed piping?

I want to convert the lowercase sequences (a,t,c,g) into '-' using the Unix command. But it's not saving the file in place. Rather it's showing errors and removing '-i' outputs in the terminal but ...
user avatar
1 vote
3 answers
140 views

Match multiple patterns in text files on the same line, copy line to next line and modify string

I want to match multiple patterns in text files (in different directories) on the same line, copy the matched line to next line and modify the copied line. I want to duplicate every line that contains ...
user avatar
0 votes
2 answers
43 views

Sed an XML balise

Please, I have this xml balise inside my test.xml file <ingressAnnotations>nginx.ingress.kubernetes.io/server-snippet: | location @custom_503 { return 503 &quot;&lt;html&gt; &...
user avatar
  • 657
0 votes
4 answers
142 views

Problems with replacing XML tag contents using sed

I have the following XML piece: <value id="1" creatorId="0" creationTime="1639487132" expirationTime="1639573532">+380554446363</value> <value id=...
user avatar
0 votes
2 answers
43 views

How can I replace a version number captured by regex with another one using SED?

I am trying with the following sed command: sed 's/shared:core:([0-9]{1,4}(\.[0-9a-z]{1,6}){1,5})/shared:core:1.1.1/' test.txt to replace the content of test.txt shared:core:0.0.2 I expect it to ...
user avatar
  • 456
-1 votes
6 answers
96 views

delete string ending with a pattern for each column

I would like to delete all entries with (L=6), (L=7).....(L=12) occurring in a file randomly the following format; my input file: TRINITY_DN1910_c0_g1_i13 GO:0005975(L=2) GO:0006022(L=4) GO:0006026(...
user avatar
-1 votes
3 answers
74 views

Changing the place operator in a csv file

I've been editing a CSV file so I can import it into postgres. At this point I want to change the operator from column 5 to the left side of the column when the value is negative"-". When it ...
user avatar
0 votes
0 answers
58 views

Keep every process in 1 execution block in CSV handling

I asked some questions about how to handle a csv file and this result was numerous processes. Every month I get several TXT files in a Google Drive directory. I need to process, unite and load this ...
user avatar
-2 votes
3 answers
104 views

change date format in csv

I have the following csv 9999999,012021,I,0099999999,000000000099999999+,000000000000000000-,000000000000000000- 9999999,012021,I,0099999999,000000000099999999+,000000000000000000-,000000000000000000- ...
user avatar
2 votes
2 answers
122 views

Split year from date column and create new column with year with awk or sed

I have a dataset in csv format like this (6000 rows): Store,Date,Weekly_Sales,Holiday_Flag,Temperature,Fuel_Price,CPI,Unemployment 1,05-02-2010,1643690.9,0,42.31,2.572,211.0963582,8.106 1,12-02-2010,...
user avatar
  • 25
0 votes
4 answers
65 views

Want to merge 2 lines in one and avoid the correct line using end pattern ABC in unix [closed]

I have file like below: 07-10-2021-ABC-hello-203455 07-10-2021-Aer-hello-2034 55 07-10-2021-Aad-hello-20345 5 07-10-2021-dafC-hello-203 455 I want output as below: 07-10-2021-ABC-hello-203455 07-10-...
user avatar
0 votes
1 answer
47 views

Replace the keyword with the whole content of another file [duplicate]

I write a bash script to replace the keyword with the whole content of another file. Here is the command I use: sed -i "s/CONTENT/$(cat $pathB)/" $pathA A file Here is my introduction: ...
user avatar
  • 21
1 vote
2 answers
103 views

Understanding how GNU sed range addresses work

I am reading about range addresses in GNU sed. But I don't understand how does that exactly work, I have tried to run sed --debug but the output is too verbose to understand. Let's assume that I have ...
user avatar
3 votes
4 answers
318 views

Using GNU sed to extract header names between two pattern lines possibly separated by blank lines

I have the following file: ------ Introduction ---------- Optio eum enim ut. Et quia molestias eos. Doloribus laborum quia quae. Magnam cupiditate quis consectetur. ----- Chapter1: Foo ----- Odit ...
user avatar
3 votes
9 answers
663 views

Convert line with words enclosed by single quotes and separated by comma to a column

I have this file: 'dog', 'cat', 'horse', 'turtle' I want to convert the line to: dog cat horse turtle if I use just the comma as separator and command tr I can do: tr -s ',' '\n' < file with the ...
user avatar
0 votes
0 answers
55 views

Using sed to replace text with spaces

I am trying to make a script to change the /etc/pam.d/common-password file to require a minimum password length by changing the line password [success=1 default=ignore] pam_unix.so obscure ...
user avatar
1 vote
2 answers
134 views

Why `sed` with multiple `-e`ends up with error on macOS Monterey? [duplicate]

On macOS Monterey, why this: $ sed -i -e 's/<azure\/core\/internal\//</g' -e 's/<azure\/core\//</g' -e 's/<azure\/iot\/internal\//</g' -e 's/<azure\/iot\//</g' . … ends up ...
user avatar
0 votes
3 answers
96 views

Multiline grep or sed

I have a very large text file with multiple sections. These sections include section headers. I would like to extract the section headers and print them with grep or sed. The file looks like this: =...
user avatar
2 votes
1 answer
112 views

Removing lines with empty fields in CSV by sed

I have a CSV file with two columns. The first is ID (int) and the second is text. All lines have the ID column. I want to remove files where the second column is empty (zero-length or space only). I ...
user avatar
  • 1,663
0 votes
1 answer
30 views

sed does not replace #DRECONTENT [duplicate]

Hi Here is a snippet of what I am trying to replace in file. #DRECONTENT 1gjHRjT9VYIBd8NdLeKOYq1wvJjdsGtf36OmiB9CfreBQRQSz3aPqDZdZZaB92m8YTNVqCRL2d45/GLcrwCyQ93RkpJiQCMe6SfVmHEAcOoUU/...
user avatar
1 vote
0 answers
11 views

Unable to replace Special character using Sed Command [duplicate]

I have to replace word Check with protected/note/run.json from book.txt file but I am receiving error. sed -i 's/Check/protected/note/run.json/g' book.txt Error: sed: -e expression #1, char 24: ...
user avatar
0 votes
2 answers
128 views

How to Read every word from a file and Replace it with the substitute word (if found) from another file

I need to read every full word from each line of a file and replace every occurrence of that word (if found anywhere in the file) using sed or awk with the word from another file. Contents of fileA....
user avatar
  • 517
7 votes
2 answers
503 views

What is this sed doing with the script: `$a\`?

I've come upon this sed command and I cannot figure out what it is doing. I understand that it is changing a file in place using -i, that it is using a script -e and that the script is $a\, but what ...
user avatar
2 votes
2 answers
90 views

How to get number in the first line of a file

My input files are file_1.txt, file_2.txt, file_3.txt and so on. These files contains data such as $ head file_*.txt ==> file_1.txt <== ----- Reset Loop 1 ------- Test #1 data Test #2 data ...
user avatar
  • 55
0 votes
1 answer
74 views

Sed + regex is not matching and replacing occurence [duplicate]

I am trying to remove the numbers at the start of the line with the following command: sed -i 's/^\d*\t//' sea_news_2020_corpus.txt The line looks as follows: 809940 The sea will be moderate in the ...
user avatar
0 votes
0 answers
48 views

How do you use sed to remove up to a certain key in a very large json file?

I'm trying to use sed to remove up to a certain element defined by KEYNAME in a very large json file. sed 's/^.* KEYNAME/KEYNAME' metadata_unique_all.json Tried that but it doesn't seem to work. ...
user avatar
  • 99
2 votes
2 answers
149 views

How to get filename if the data is available

My input files are file_1.txt, file_2.txt, file_3.txt and so on. These files contains data such as $ head log_file_reset_*.txt ==> file_1.txt <== Test #1 data Test #2 data Test #3 Test #4 ...
user avatar
  • 55
0 votes
4 answers
80 views

Replacing lines in one text file with line from another text file

I have a HLS playlist file looking like this: #EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:12 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:12.500000, playlist0.ts #EXTINF:8.333333, playlist1.ts #EXTINF:12.500000,...
user avatar
  • 113
0 votes
1 answer
67 views

sed remove timezone from date string of ics file

I have very ics files that are malformed. Here is one example: BEGIN:VCALENDAR PRODID:-//cals.sig9.net//ptHolidays VERSION:2.0 BEGIN:VEVENT ORGANIZER;CN=Feriados Portugueses:MAILTO:nuno@sig9.net ...
user avatar
-2 votes
4 answers
144 views

Get sequence number behind data

My input file : {empty line} Test #1 data {empty line} Test #2 data {empty line} Test #3 {empty line} Test #4 data {empty line} Test #5 Expected output: 1 2 4 How to get the sequence number (after ...
user avatar
  • 55
0 votes
2 answers
274 views

Replace content of a specific column in a CSV with sed

I have several CSV log files and want to replace the status codes stored in the 8th column of these log files with a description The log files looks like this: ip,date,time,zone,cik,accession,...
user avatar
1 vote
1 answer
98 views

Shell process substring

I know I can replace a substring with sed and integrate command evaluation with $(...). But what if I want the replacement to be calculated from the string to be replaced? Example I have a formatted ...
user avatar
  • 395
0 votes
2 answers
99 views

sed command to replace pattern including newline pattern with blank

I have a yaml file which include text: - name: flexvol-driver-host hostPath: type: DirectoryOrCreate path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec ...
user avatar
  • 13
2 votes
3 answers
375 views

Extract values between two patterns

I am using Ubuntu and I want to extract the values between two patterns and the required string is not in the file. My data is as shown below: [{"rows":[{"_uuid":["uuid",&...
user avatar
-3 votes
2 answers
109 views

awk or sed to format variable durations

Tried a lot of awk and sed commands, but cant get the file to format properly. The primary issue is with handling of extra comma in the middle of a column that exits sometimes and doesnt exist other ...
user avatar
  • 95
0 votes
3 answers
124 views

sed word boundary not working with colon character

I have the following file > cat tmp "aaa" "aaa:xyz" The desired output is "bbb" "aaa:xyz" I'm using the following sed command sed "s/\baaa\b/bbb/"...
user avatar
  • 23
0 votes
2 answers
154 views

how to comment out many lines in a file with sed

in a file I have the following lines foo bar xxx yyy zzz I need to comment out the lines xxx, yyy, zzz. So I run the following command: sed -e '/xxx/ s/^#*/#/' -i file sed -e '/yyy/ s/^#*/#/' -i file ...
user avatar
  • 781
0 votes
1 answer
35 views

Only perform sed operations on lines starting with a particular string [closed]

I have the following file format Received from +11231231234 at 2021-10-10T19:56:50-07:00: This is a message that contains words like from, at, etc. Sent to +11231231234 at 2021-10-11T06:50:57+00:00: ...
user avatar
  • 53
1 vote
2 answers
77 views

Bash Script to replace attachment paths in file with full path

I have a mix of filenames and relative paths in a markdown file that I want to refactor so that the full relative path to ~/app/ is inserted for interoperability between markdown readers. For my files ...
user avatar
0 votes
3 answers
52 views

enumerate all matches in file with sed or perl

I have next file: :~$ cat test <Set Id="16"> <Ver="44" Sniff="no" B2vpnId="" Collision="no"> </Set> ...
user avatar
1 vote
4 answers
165 views

Awk/sed to separate last column and add quotes around middle column?

I've got a file called test.txt and after some manipulations it looks like this: Metabolism Global and overview maps 01100 Metabolic pathways (1689) 01110 Biosynthesis of secondary metabolites (677) ...
user avatar
  • 146

1
3 4
5
6 7
135