New answers tagged

0 votes

split line according to matching pattern in bash

This is a task that needs a text processing language (such as awk or perl), not a shell script. $ cat vars2csv.pl #!/usr/bin/perl use strict; # %vars is a Hash-of-Arrays (HoH) where the primary ...
user avatar
  • 68.3k
0 votes

How to split a single line of fields delimited by commas into multiple lines with a specific number of fields in each line?

sed 's/,/\n/2;P;D' m=3 sed "s/,/\\n/$m;P;D"
user avatar
  • 5,398
0 votes

How to split a single line of fields delimited by commas into multiple lines with a specific number of fields in each line?

$ awk -v n=2 -F',' '{for (i=1;i<=NF;i++) printf "%s%s", $i, (i%n ? FS : ORS)}' file Hello,Hi Hullo,Hammers Based,Random $ awk -v n=3 -F',' '{for (i=1;i<=NF;i++) printf "%s%s"...
user avatar
  • 22.5k
0 votes

How to split a single line of fields delimited by commas into multiple lines with a specific number of fields in each line?

Using perl: $ echo 'Hello,Hi,Hullo,Hammers,Based,Random' | perl -F, -le ' BEGIN { $n = shift }; for ($i=0; $i < @F; $i += $n) { print join(",", @F[$i .. ($i + $n ...
user avatar
  • 68.3k
0 votes

Grep line containing a word OR has length in a specific range?

Using Raku (formerly known as Perl_6) raku -ne '.put if (.contains("CSCO")) | (.chars <= 15);' OR raku -ne '.put if .contains("CSCO") or .chars <= 15;' Sample Input: ...
user avatar
  • 1,232
0 votes

How to split a single line of fields delimited by commas into multiple lines with a specific number of fields in each line?

Using sed $ sed -E 's/([^,]*,[^,]*),/\1\ /g' input_file Hello,Hi Hullo,Hammers Based,Random $ sed -E 's/(([^,]*,){2}[^,]*),/\1\ /g' input_file Hello,Hi,Hullo Hammers,Based,Random
user avatar
  • 1,139
0 votes

Bash function for 'Deep Replace' in file contents, file name, directory name, allowing for whitespace

Your function were almost conform to your requirements, just missing a lot of quotes around variables and substitutions, that's why it did not resist to space in filenames. variables : "$1" ...
user avatar
0 votes

Grep line containing a word OR has length in a specific range?

I assume CSCO is on the 1st filed. The awk method: awk '$1 == "CSCO"; length <= 15 { print }' INPUT # length is a built-in awk function which returns the length of a line
user avatar
1 vote

Replacing strange characters in a sed command

From the sed -n l listing it is clear that you have a file with many characters 174 (in decimal or 256 in octal) and [character 175] (in decimal), or 257 (in octal). Listed as \256 and \257 and that ...
user avatar
6 votes
Accepted

Replacing strange characters in a sed command

Sed can also be used as a script (easier to devel): create a file "nb2txt" with #!/usr/bin/sed -Ef s/®[^¯]*¯//g s/-{20,}//g s/\.{20,}//g and: $ chmod 755 nb2txt $ nb2txt file.nb
user avatar
  • 10.8k
1 vote

Bash function for 'Deep Replace' in file contents, file name, directory name, allowing for whitespace

As in comments, I've determined that a pure bash solution was not optimal for my use case. I've settled for a command that depends on Node.js / npm (since Node is fairly ubiquitous on environments I ...
user avatar
  • 121
3 votes

Replacing strange characters in a sed command

Your regular expression uses \| (alternate pattern in GNU sed, literal bar in most other sed implementations) and \+ (one or more occurrences in GNU sed, literal + in most other sed implementations). ...
user avatar
  • 11.7k
5 votes
Accepted

Grep line containing a word OR has length in a specific range?

With grep -E for Extended regexps you can use alternation (|). $ grep -E 'CSCO|^.{0,15}$' file 1598427@931 CSCO 220715C00090000 ohlc=0,0,0,0 vol=0 oi=739 nbbo=0@0/4@1056 nbbo2=0@0/4@121 CSCO ...
user avatar
  • 68.3k
5 votes

Grep line containing a word OR has length in a specific range?

grep -e CSCO -e '^.\{0,15\}$' filename Here you give grep two patterns to look for. The first is "CSCO": so it'll match any lines with that. The second looks for the start of the line ^ ...
user avatar
  • 7,347
0 votes

Finding the line number of first occurrence of a text in bash script

Using Raku (formerly known as Perl_6) raku -ne 'state $i; ++$i; say "c starts line $i" and last if m/^c/;' OR raku -ne 'state $i; ++$i; say "c starts line $i" and last if (....
user avatar
  • 1,232
0 votes

Capturing section of first line and output all lines matching pattern

Here are examples of scripts that you can customize as needed: Script search1 Script task: Use the 6-th field of first row and search for occurrence in all lines. #!/bin/bash if [ ! -f "$1" ]...
user avatar
  • 111
0 votes

How to make grep for a regex that appear multiple times in a line

Using Perl in sed mode (-p), (-l) shall remove LF (\n) from every record read and also append to every record printed. perl -plse ' $_ = "@{[/\[\K\d{1,3}(?:\.\d{1,3}){3}(?=])/g]}" || "...
user avatar
  • 5,398
0 votes

Finding the line number of first occurrence of a text in bash script

grep can print the line number of a match with -n or --line-number so you can use that. $ cat sample.txt | grep '^c' --line-number 3:cddefefef // this is the line that I need its line number 10:cat // ...
user avatar
  • 215
1 vote

awk/sed find indexes of the first and the last capital letter in a string

POSIX awk with field separator as an uppercase regex. LC_ALL=C \ awk -F '[A-Z]' ' NF>2{ print length("x"$1), length("x"$NF) }' file Perl has index & rindex builtins to ...
user avatar
  • 5,398
1 vote

awk/sed find indexes of the first and the last capital letter in a string

Another perl approach: perl -Mopen=locale -lne ' print 1+length$`, " ", 1+length$'\'' if /\p{Lu}(.*\p{Lu})?/' \p{Lu} matches an upper case Letter (such as ABCÀÁÂÃÄÅАБВГᏢᏣᏤᏥ𝓐𝓑𝓒ⰗⰘⰙⱠⱢⱣⱤⱧⱩ....
user avatar
2 votes

awk/sed find indexes of the first and the last capital letter in a string

Using Raku (formerly known as Perl_6) Sample Input: --AbbbAnde--- abksjiRNNBBKUGFLYFYLF -ankNUGUYUBUIGCafrg-- BNKJUGFVULNK-Kew--- nouppercaseletters oneUppercaseletter Skips lines with 0 or 1 ...
user avatar
  • 1,232
4 votes

How to make grep for a regex that appear multiple times in a line

With perl: perl -lne ' if (@ips = /\[(\d{1,3}(?:\.\d{1,3}){3})\]/g) { print join ",", @ips; } else { print "n.a."; }' Or using the regexp for dotted-quad-decimal ...
user avatar
1 vote

How to make grep for a regex that appear multiple times in a line

Executable awk file filter.awk: #! /usr/bin/awk -f { ret = "" line = $0 while (match(line, /\[([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}\]/) > 0) { if (ret != "&...
user avatar
7 votes

Finding the line number of first occurrence of a text in bash script

With POSIX sed, you suppress normal output with the -n option, then for the line starting with c (pattern ^c), print the line number with = and quit: sed -n '/^c/{=;q;}' With GNU sed, you can use the ...
user avatar
  • 11.7k
3 votes

How to make grep for a regex that appear multiple times in a line

Using GNU awk for the FPAT: awk -v FPAT='\\[([0-9]{1,3}[.]){3}[0-9]{1,3}\\]' -v OFS=, ' { $1=$1; print (gsub(/[][]/, "")?$0:"N/A") }' <infile >output or with any POSIX ...
user avatar
  • 38.4k
6 votes

Finding the line number of first occurrence of a text in bash script

Several solutions exist with AWK awk '/^c/ { print NR; exit}' "${varFileLog}" /^c/: matches the line starting with c print NR: prints the record (line) number exit : does not continue ...
user avatar
  • 764
1 vote

awk/sed find indexes of the first and the last capital letter in a string

{ pad = s = match($0, /[A-Z]/); clone = substr($0, RSTART + 1) while (match(clone, /[A-Z]/)) { clone = substr(clone, RSTART + 1) pad += RSTART } print $0, ": &...
user avatar
4 votes
Accepted

Finding the line number of first occurrence of a text in bash script

You need to tell grep about your “that should be in the start of a line” constraint, by anchoring the match to the start of a line with ^: trimLineNum=$(grep -m1 -n -- '^c' "${varFileLog}") ...
user avatar
1 vote

replace text in specific field with variables from loop

Another awk solution, which assumes that the lines from src will be used exactly once each and in order. This allows us to only keep track of the next line from src until it has been used, and then ...
user avatar
  • 271k
1 vote

substitute 2 substrings from a single line in a file into 2 different lines in a different file from command line

As a rule of thumb, using grep along with sed is almost always superfluous, because sed can filter by itself. In your case, you can use the p flag to the s command to print on a match only, then ...
user avatar
  • 11.7k
2 votes
Accepted

substitute 2 substrings from a single line in a file into 2 different lines in a different file from command line

I can get it down to these two lines: values="$(sed -n -E 's/^.*fixed_stringA\s*=\s*([0-9]+)\s*,\s*fixed_stringB\s*=\s*([0-9]+).*$/\1,\2/p' first_file)" sed -i -E -e "s/unknown1/${...
user avatar
  • 7,347
4 votes

awk/sed find indexes of the first and the last capital letter in a string

$ awk ' match($0,/[[:upper:]](.*[[:upper:]])?/) { print $0, RSTART, length()-(RSTART+RLENGTH-2) } ' file xyzAb 4 2 --AbbbAnde--- 3 7 abksjiRNNBBKUGFLYFYLF 7 1 -ankNUGUYUBUIGCafrg-- 5 7 ...
user avatar
  • 22.5k
3 votes
Accepted

replace text in specific field with variables from loop

awk 'BEGIN{ FS=OFS="|" } NR==FNR { id[$1, $2]=$3; next } { $4=( ($1, $4) in id? id[$1, $4]: $4) } 1' src dest FS: Field Seperator OFS: Output Field Seperator NR==FNR: An ...
user avatar
  • 38.4k
4 votes
Accepted

awk/sed find indexes of the first and the last capital letter in a string

awk ' { start = match($0, /[A-Z]/) end = match($0, /[A-Z][^A-Z]*$/) print (start ? start : "NaN"), (end ? length() - end + 1 : "NaN") }' infile
user avatar
  • 38.4k
3 votes

awk/sed find indexes of the first and the last capital letter in a string

It is easy to get the length of the leading or trailing part with AWK. Add 1 to get the index as shown in the question. echo '--AbbbAnde--- abksjiRNNBBKUGFLYFYLF -ankNUGUYUBUIGCafrg-- BNKJUGFVULNK-Kew-...
user avatar
  • 5,433
0 votes

How to replace only the Nth occurrence of a pattern in a file?

Use sed with -z option to define the occurrence you want to replace. sed -i -z "s/is/us/3" pom.xml your.file
user avatar
  • 101
0 votes

Deleting all text after a specific string for multiple text files in a directory

Using Raku (formerly known as Perl_6) Sample Input (erroneous JSON): ~$ cat file.json {"analysis":{"score":3},"sample":{"completed":"2022-01-27T21:22:21Z&...
user avatar
  • 1,232
2 votes

Find regex occurrances on block device (line length buffer issue)

In a comment above, @terdon gave the key insight about reducing the search space first. By using the extended grep pattern syntax to reduce the maximum line length given to the perl (PCRE) grep ...
user avatar
  • 1,108
1 vote

Replace values in fifth column

Using Raku (formerly known as Perl_6) raku -ne 'given .words -> $w {put "$w.[0..3] ", (S/ (\d+) /user{$0.substr(0,1)}/ with $w.[4]) };' OR raku -ne 'put .[0..3], " ", (S/ (\d+) ...
user avatar
  • 1,232
3 votes

Replace values in fifth column

Okay, unfortunally, sed doesn't know that 10 is John. So if you don't want to write another substitution for each user, you might want to use a second file users.txt with a translation table like this:...
user avatar
  • 11.7k
2 votes

Replace values in fifth column

awk 'gsub(/0/, "", $5) { print $1, $2, $3, $4, "user" $5 }' INPUT gsub removes number "0" in the 5th column. Then awk prints column 1-4, and combines word "user&...
user avatar
2 votes

Replace values in fifth column

Two more awk solutions. $ awk '{$5="user"substr($5,1,1)}1' file Name 1:10:34 date short_id user1 Name 1:10:45 date short_id user1 Name 1:20:54 date short_id user2 Name 1:30:43 date short_id ...
user avatar
  • 20.2k
1 vote

Replace values in fifth column

Just use awk: $ awk '!($NF in users){ users[$NF]="user"(++cnt) } { $NF=users[$NF] } 1' file Name 1:10:34 date short_id user1 Name 1:10:45 date short_id user1 Name 1:20:54 date short_id user2 ...
user avatar
  • 22.5k
1 vote

Replace values in fifth column

Using sed sed 's/\(.*\) \(.\).*/\1 user\2/' input_file Name 1:10:34 date short_id user1 Name 1:10:45 date short_id user1 Name 1:20:54 date short_id user2 Name 1:30:43 date short_id user3 Name 1:40:43 ...
user avatar
  • 1,139
1 vote

Replace values in fifth column

If all you want to change is the last column you don't need the g modifier for global and you will want to add the $ anchor to limit it to the last column. I.e.: sed 's/\b10$/user1/' That is assuming ...
user avatar
  • 2,289
0 votes

Search and replace entire line after 1st match using SED?

If you are using GNU sed, you can use the -z option to process the whole file in one buffer and use the number flags to the s command to substitute the nth occurence: sed -zE 's/(^|\n)M104[^\n]*/\...
user avatar
  • 11.7k
0 votes

Search and replace entire line after 1st match using SED?

Your (failed) attempts imply that your sed version allows the 0 for the start of a range (not all seds do). If so, try sed -rn '0,/^M104/{p;b;}; 0,/(^M104).*/ s//\1 S300/; p' file M104 S200 M107 M104 ...
user avatar
  • 8,571
2 votes

Search and replace entire line after 1st match using SED?

If you have Perl, I'd do something like this: $ perl -pe 'if (/M104/) { $i++; if($i == 2) { $_ = "M104 S999\n" } } ' test.txt M104 S200 M107 M104 S999 M110 M104 S275 I.e. if /M104/ matches, ...
user avatar
  • 117k
5 votes

Deleting all text after a specific string for multiple text files in a directory

Using jq (available for most Unix-like systems), you could extract the first JSON object from a file and discard the rest using jq -n 'input' file >newfile This produces "pretty-printed" ...
user avatar
  • 271k
1 vote

Remove all characters from string, except specific multibyte range

Using Raku (formerly known as Perl_6) Sample Input (OP's text saved as a one-line file): ~$ cat peace.txt Peace be upon you. السلام عليكم. שלום עליכם. Paz sobre vosotros. Extract Arabic script: ~$ ...
user avatar
  • 1,232

Top 50 recent answers are included