Questions tagged [sed]
sed is a command-line stream editor for filtering and transforming text.
6,740
questions
0
votes
3
answers
44
views
sed, awk to copy paste pattern1 line into following pattern2 lines
I'm looking to copy -city- line to end of "all" following -business- lines
This:
-country-france
-city-Paris, Marseille, Lyon, Toulouse, Nice, Nantes
-business-TotalEnergies
-business-AXA
-...
1
vote
2
answers
59
views
Regexp to fix GPS data
I'm looking to fix GPS data in text file on Linux
Incorrect string model: 49:29:37N 005:28:39E
2or3digits:2or3digits:2or3digits(N|S) 2or3digits:2or3digits:2or3digits(E|W)
Correct string model: 49:29:...
- 13
3
votes
7
answers
149
views
How to join lines in a file
I would like to ask a question about joining lines in a file on Linux.
This is a part of the file:
"64f94e8e-4bd2-4721-95e7-7151cfbf3eb6"|"fanel"|""|""|"&...
- 39
-2
votes
3
answers
98
views
How can I remove a character {only found between curly brackets}, and take the brackets along for the ride?
For example, if I'm removing only spaces between brackets, the title of this post would be:
How can I remove a character onlyfoundbetweenbrackets, and take the brackets along for the ride?
I've only ...
- 19
1
vote
5
answers
83
views
How do I replace all the spaces in just a substring of certain lines in a file?
I have 3000 Markdown files named "journal/*.md", some of which contains file links like this:
blah blah text [Label here](file:///path/to file/may contain/spaces)
I want to change all such ...
- 111
2
votes
2
answers
280
views
Replacing // with /* */ using sed command [duplicate]
How do you replace // with /* */ in sed command?
I have tried:
sed -e "s/////* *//g" <<< filename
but this results in bad flag in substitute command: '/'
Anyone know how to fix ...
- 21
0
votes
2
answers
51
views
Delete a block including nested braces
I have the following block in an NGINX config file
server {
listen 80;
listen [::]:80;
server_name _;
root /web/default/wwwroot/;
# Load ...
- 101
3
votes
1
answer
96
views
How do I remove anagram duplicates from a list?
I have a tab-delineated .csv with strings and descriptions. A bit like this:
strap tight
elba Island
foo bar
able Cain
parts buy
able Cain
First, I'm sorting the file by length of ...
- 31
3
votes
1
answer
47
views
How to get the output file format(with defined spaces) as same as input file format when passing through loops [closed]
I need to get the output file same as the input file format after passing through the while loop.
eg:
cat samp.csv
a b c d
cat samp.csv | while IFS= read -r line
do
echo $...
0
votes
2
answers
66
views
Reformatting a list of elements using bash/sed
I have a CSV file with a list like this:
URL,Domain,anchor
https://example1.com,Example1,Category1
I need to reformat this into HTML, like this:
<li><a href="https://example1.com" ...
-1
votes
3
answers
57
views
How to use cut to trim the first x variable spaces [closed]
I have a file that is like this:
67 lol
143 hi
21 test
1 ciao
5 lo
I want to remove the spaces.
67 lol
143 hi
21 test
1 ciao
5 ...
- 221
1
vote
2
answers
192
views
How to "give priority" to certain pattern instead of others in sed?
I have this sed filter:
/.*[1-9][0-9][0-9] .*/{
s/.*\([1-9][0-9][0-9] .*\)/\1/
}
/.*[1-9][0-9] .*/{
s/.*\([1-9][0-9] .*\)/\1/
}
/.*[0-9] .*/{ # but this is always preferred/...
- 221
-2
votes
4
answers
112
views
Add two new columns in tab delimited file
I have to add two new columns at the end of the tab-delimited file with specific entries. My file looks like this
chr2 88861474 88861574 IGKJ2
chr2 88861170 88861270 IGKJ3
chr2 ...
- 89
2
votes
4
answers
94
views
How to append string to uppercase letters in bash
Here's some sample code:
{"key_code":"a"},
{"key_code":"B"},
{"key_code":"c"},
{"key_code":"D"}
Here's what I'm trying ...
- 19
0
votes
1
answer
94
views
How can I match any character BUT a string with POSIX Extended Regex (for sed)?
I am dealing with a string:
Input File
----
some**wor*ds**with**asterisks
By telling sed to match any character that is not an asterisk:
echo "some**words** **" | sed -E 's/\*\*([^*]+)\*\*/...
-3
votes
2
answers
53
views
merge lines startimg from pattern to next pattern excluding second patten
Input:
@@string1
string2,
word
@@string3
string4,
word
@@string5, word
Desired output:
@@string1string2,word
@@string3,string4,word
@@string5, word
Tried
sed -n '/^@@/{:a;N;/^@@/!ba};/^@@/s/\n//g;p'
...
- 65
0
votes
0
answers
24
views
If I complement the address, I have the previous command append in sed
I have this file:
$ cat ciao
ciao
ciao
ciii
ciaooo
ocisdjoo
kdvkks
kdafjdjf
akfk
and
$ sed ciao -n -e "1,3p"
ciao
ciao
ciii
$ sed ciao -n -e "1,3!p"
sed ciao -n -e "1,3ps -...
- 221
0
votes
2
answers
83
views
Need help to convert csv file format using Unix commands
Input File :
"Run Date/Time: 2022-02-09 12:47",,,GOOD_MORNING_WORLD
"File Processed: AB-FILE2.20220209.110516",,,GOOD_MORNING_WORLD
AB1234,5," PQR2",...
1
vote
6
answers
107
views
Extract URLs from inside img html tag across multiple files
I have a folder with lots of .html files, the files have <p>..</p> html tags and <img...> html tags across all of them
I am trying to extract ONLY the URLs found inside the <img......
- 45
2
votes
2
answers
265
views
How to format this IP list?
I have a comma-separated list in this rather simple format:
IPrangestart,IPrangeend,int number,date (delimited by slashes),Name (is dirty, contains spaces, dots, dashes, quotemarks etc)
The first ...
- 637
0
votes
5
answers
489
views
Remove all characters till the first letter ([a-zA-Z]) in a string
I want to remove all characters till the first letter [a-zA-Z] in a string. For example:
#121Abc --> Abc
%Ab#c --> Ab#c
Ab#c --> Ab#c
Abc --> Abc
1Abc --> Abc
1 21Abc --> Abc
1^1 ...
- 1,364
0
votes
1
answer
42
views
Select line matching pattern followed by substitution using `sed`
Here's what I'm trying to replicate:
$ cat << EOF | { var=$(grep 'foo=');\
if (( $? == 0 )); then echo "$var"\
| sed -E 's/foo=(.+)/\1/'; else exit 1; fi; }
foo
foo=bar
EOF
bar
...
- 481
0
votes
1
answer
70
views
How do you remove all punctuation using the sed command?
I'm trying to remove all punctuation from a text file using the sed command, but I don't quite know how to.
-4
votes
1
answer
54
views
How to remove newline in a column and merge it by using sed
How to remove newline in a column and merge it by using sed.
xlsx2csv rawdata.xlsx -n 'CONSOLIDATION MAIN' -d '|' | sed '1d-6d' |grep -v ^REGION > RAWDATA.txt
0
votes
2
answers
61
views
Replace either of `"(.+)"` or `{(.+)}` by `\1`
I can do
{(.+)}-->\1, and
"(.+)"-->\1, or
({.+}|".+")-->\1.
I would I do {(.+)}|"(.+)"-->\1?
$ echo "\"x\"" | sed -E 's/"(.+)"/...
- 481
1
vote
2
answers
67
views
how to create a csh script that can identify the correct USB disk where I have saved my virtual machines
I'm running Freebsd 13R and as shell I'm using csh. I would like to create a script,that looks the presence of the word "r1w1e2" or "NM13N4CZ" or "3e4d31334e34435a" when ...
- 371
1
vote
1
answer
104
views
grep with many patterns from file and show which pattern matched which file without rereading files
I have n separate non fixed greps against m files. I only need to know if there is at least 1 match in each file, but I need this for each pattern. I currently make n separate greps so I can merge ...
- 47
1
vote
3
answers
160
views
Split .txt file into 3 files based on text in line using shell script
I have a requirement where I'm getting a .txt file in output from some tool and I want to split that into three.
Example text from file:
First line
Second line
23456
45677
45678
Third line
90909
...
- 31
1
vote
1
answer
63
views
Extract linux (admin) users report for remote hosts [closed]
I am trying to find the best way to extract a report of Linux users and also which one of them are admin. The targets are Debian (8,9) and Ubuntu (18,20). The final report would be very long, so I ...
- 19
0
votes
1
answer
70
views
Delete last occurrence of a variable using sed / bash script
I have multiple files written as */*/*/*/ ... */*.h or .hpp. I need to get the directories of these files, instead of the files themselves. Is there any way, using sed, I could do something like 'find ...
- 23
1
vote
0
answers
38
views
SQL output piped to `kubectl exec` crashes, if not replacing TABS with SPACES
The context:
I'm generating SQL from a backup dump, inside a subshell, to then pipe it to mysql for execution:
(
echo "SET FOREIGN_KEY_CHECKS=0;";
tar --to-stdout -xvjf archive.tar;
echo &...
- 203
2
votes
3
answers
76
views
Using sed to parse % score
I am trying to parse code coverage for my github ci action, I have everything working but can't parse the coverage % result. Please help me parse the % score for code coverage, I can't get the regex ...
-1
votes
1
answer
120
views
How to grep the difference between two files into a file
Saying that I have two files: a.txt and b.txt.
The content of a.txt:
mosfet23
cap098
inductors 98
metal96
The content of b.txt:
test 23 cap098
drc45 metal96
lvs cap098
Of course I can use vimdiff to ...
-1
votes
4
answers
83
views
Easy way to output a file and change the implicit newlines ($) in the file into literal (\n)
I have a file with a number of invisible Newlines. In Vim it's shown as $ (can be seen with :set list). I had a hard time to convert them to an output which is represented as a one line string with ...
1
vote
1
answer
86
views
Parse pylint result to extract score?
I am building a github CI using action and I was able to run pylint and write a message to PR with the result. However, I want to parse the exact score so I can fail the action if it's bellow a ...
-1
votes
3
answers
70
views
Trim column X leaving a specific number of chatacters if column Y and column Z are not equal to NULL
I am trying to find a way to use awk/sed to filter a csv to truncate the value in a specific column such as column 1 to retain the last 8 characters and replace the beginning with a wildcard, only if ...
- 11
-1
votes
2
answers
99
views
awk/sed filter example for column manipulation based on data in other columns
I am trying to find an example of how I may effeciently use sed or awk to manipulate csv data in a specific column (col 1 for instance) if the data in col 2 AND col 5 are not equal to "null"....
- 11
0
votes
0
answers
45
views
Prevent xxd from adding newline to the output
On unix if I do
printf "aY.S'Hk([;jYJ}8eJ5)Wsd58/x}s]Pne3V-6:t@" | xxd -p -c 256
I get
61592e5327486b285b3b6a594a7d38654a352957736435382f787d735d50
6e6533562d363a7440
However on mac os (...
- 101
0
votes
1
answer
79
views
replacing value in xml file using sed value from variable missing ""s
hi can some one please help me here
i am trying to replace value in tomcat server.xml file username using sed command
<Resource username="user1" password="xxxxxxxx"/>
below ...
- 3
0
votes
3
answers
32
views
This sed syntax won't remove blank lines
I have this input file:
...
https://long-url/something?value1=999&something=xyz?value2=555
https://long-url/something?value1=534&something=xyz?value2=2312
https://long-url/something?value1=...
- 1,580
-3
votes
1
answer
23
views
How to use sed to delete this string </p>\n</body>\n</html>"
How to delete this string using sed?
</p>\n</body>\n</html>"
0
votes
2
answers
75
views
Complete file line by line reading dates
I have a text file containing 2 columns. The first one has DATES (DD/MM/YYYY) and the second one, numbers. It looks like this:
15/01/1945 105.0
16/01/1945 4.2
17/01/1945 3.0
31/01/1945 12.0
01/02/...
- 1
0
votes
3
answers
111
views
Remove all base64 blocks from a file
I am in the process of trying to upload around 20 years worth of Usenet archives to archive.org but my first batch got reject because some of the archives contained trojans that are encoded in base64. ...
- 201
-1
votes
3
answers
53
views
Trim pathname in CSV file
I am struggling over a filter where I am trying to trim data in a specific column of a CSV after the 3rd or nth occurrence of the character \.
My data looks something like this:
data,data,c:\path1\...
- 11
0
votes
3
answers
77
views
Search and Remove Lines of Code from all files
I need help and need to search for and remove a 3 line bit of code from all files in a directory.
Code is:
if ( file_exists( plugin_dir_path( __FILE__ ) . '/.' . basename( plugin_dir_path( __FILE__ ) )...
-2
votes
1
answer
104
views
Select first n words of each line of a file
Using sed or awk, I want to select the first n words of each line of a file.
I have a file with three lines, say
one two three four
one two three four
one two three four
and I want the output file to ...
- 1
0
votes
2
answers
38
views
How to modify all lines in a file
I have a file called index.js with more than 200 lines :
AcademicCapIcon.svelte
AdjustmentsIcon.svelte
AnnotationIcon.svelte
ArchiveIcon.svelte
...
I'd like to change this to the following:
import ...
- 599
1
vote
1
answer
46
views
Advanced sed command in pyenv installation instructions
Pyenv repository, has the following sed command for bash:
sed -Ei -e '/^([^#]|$)/ {a \
export PYENV_ROOT="$HOME/.pyenv"
a \
export PATH="$PYENV_ROOT/bin:$PATH"
a \
' -e ':a' -e '$!{...
- 123
0
votes
1
answer
47
views
Does the regex engine have an associated executable?
I'm working with a tool that uses a whitelist to allow certain commands to be executed. The whitelist works using file paths like so:
command1=/bin/echo
command2=/bin/sed
...
The problem is commands ...
- 48
-1
votes
2
answers
49
views
Replace empty spaces with TAB except in last column
I have a file that looks like this.
How can I replace each empty space with a TAB except in the last column that starts with ID?
chr1 13513 14763 Medtr1g004950 . + JCVI gene . ID=Medtr1g004950;Note=...