8
We use the awk utility to employ flip flop logic to toggle from on state to off as shown:
$ awk -v str="==" '
$0""==str{f=!f;next};f
' testfile
Last test
Using Posixly sed constructs we can implement the flip flop logic:
sed -ne '
/^==$/{
x; # access state info from hold
s/^$/0/; # initialize state
y/01/10/; #...
5
Using GNU datamash:
$ datamash -t ' ' -s -g 1 collapse 2 <file
A 1,2,9
ABC 12,6
B 4,3
C 5
Options:
-t '_' use the space character as field delimiter
-s sort the input before grouping
-g 1 group on the first field
collapse 2 collapse values of the second field into comma-separated list
5
$ awk '{ object[$1]= (object[$1]==""?"":object[$1] ",") $2 }
END { for(obj in object) print obj, object[obj] }' infile
A 1,2,9
B 4,3
C 5
ABC 12,6
A bit more efficient (use of the memory; matter for a huge file that cannot fit into the memory), i.e not buffering the file partially into the memory that awk command alone does ...
5
In awk, you could collect the line numbers to an array and read through the file once, printing the lines that are mentioned in the array:
#!/bin/sh
awk -v lines="$*" 'BEGIN { split(lines, a, "[, ]");
for (i in a) b[a[i]] = 1;}
NR in b {print $1, $2}' < data.txt
The split() splits the ...
3
I'd use perl:
$ perl -0777 -ne 'print $3 while /^((\S)\2+\n)(.*?)^\1/smg' < your-file
This is a test
Another test
Last test
Or pcregrep:
$ pcregrep -Mo3 '(?s)^((\S)\2+\n)(.*?)\n?^\1' < your-file
This is a test
Another test
Last test
If it's only about returning what's between fixed delimiters:
$ pcregrep -Mo1 '(?s)^==\n(.*?)\n?^==$' < your-file
...
3
The last line of your script is not identical to the one that works on the command line. There is no reason to add those two backslashes.
s/(\[0-9]+\.[0-9]{2}\)[0-9]+/\1/
should be
s/([0-9]+\.[0-9]{2})[0-9]+/\1/
(also, you don't need to escape the semi-colon, in your first s/// command, but this is not an issue, as it is just ignored)
3
A bit awkward, but I think it gets the job done.
The csvkit Python package contains command line tools for working with CSV files. Among these is csvcut, which works like the standard cut command, but which is CSV aware, which means that it allows for fields that may contain embedded field delimiters and newlines, as long as they are properly quoted.
With ...
3
tl;dr
$ sed '/^==*$/,//{//!p};d' testfile
This is a test
Another test
Last test
On first look, a simple range could print all pairs (no loops needed):
$ sed -n '/^=/,//p' testfile
===
This is a test
===
====
Another test
====
==
Last test
==
That prints every line between a line that starts with = and the next repeated regex (\\).
That could be improved to ...
3
Seems like your question wasn't clear and you got various types of solution. To get only content surrounded by == with GNU sed (syntax/feature might vary with other implementations):
$ sed -n '/^==$/,//{//!p}' testfile
Last test
sed has a way to select a range of lines by specifying two addresses separated by a comma. To avoid repeating the regexp you can ...
2
Using awk in place of sed:
newdata='This is the new data'
newdata=$newdata awk -F '\n' -v OFS='\n' -v RS= -v ORS='\n\n' \
'$1 == "# SECTION ONE" { $(NF+1) = ENVIRON["newdata"] }; 1' file
or
newdata='This is the new data'
newdata=$newdata awk '
BEGIN { FS = OFS = "\n"; RS = ""; ORS = "\n\n" }
$1 ...
2
Perl
perl -MList::MoreUtils=all -F'\t' -lane '
print if all { /^comp-/ } @F[0,1];
' file
sed GNU version
$ sed -n '/^comp-.*\tcomp-/p' file
2
awk -F'\t' '$1 ~/^comp-/ && $2 ~/^comp-/' infile
same but pass the pattern from a parameter:
awk -F'\t' -v pat='comp-' '$1 ~"^" pat && $2 ~"^" pat' infile
or compare as string match and still pass from a parameter:
awk -F'\t' -v str='comp-' 'index($1, str)==1 && index($2, str)==1' infile
see also How do I find ...
2
command:
awk '{a[++i]=$0}/==/{for(x=NR-1;x<NR;x++)print a[x]}' filename|sed '/^$/d'
output
This is a test
Another test
Last test
2
Perl to the rescue!
perl -C -pe 's/[\x{80}-\x{ffffffff}]/0/g'
-p reads the input line by line, running the code for each and printing it after processing
-C turns on UTF-8
2
Using any awk in any shell on every Unix box:
$ awk -F, -v OFS='|' 'NF>7{$5=$5",\n"; sub(/\n./,"")} 1' file
Name,Age,Country,ID,Address,Category,DOB
John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995
$5=$5",\n" adds a , and a newline (which can't otherwise be present in the current record since records are ...
2
With awk you can try:
awk -F, -v OFS='|' 'NR==1{print $0} NR>1{$5 = $5 FS $6; print $1,$2,$3,$4,$5,$7,$8}' file
Name,Age,Country,ID,Address,Category,DOB
John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995
2
With bash, one can do this, which handles an arbitrary number of commas in the address field:
# function to join strings with a separator
join() {
local IFS=$1
shift
printf '%s\n' "$*"
}
# process the file
{
IFS=, read -ra header
join '|' "${header[@]}"
f=${#header[@]} # expected num of ...
2
You haven't really explained how we can know which commas to keep and which to change. Based on the single example line you have given us, it might be enough to just replace all , that don't come after a space with a |:
$ sed -E 's/,(\S)/\|\1/g' file
Name|Age|Country|ID|Address|Category|DOB
John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995
...
1
On a POSIX compliant system, and in a locale that uses the UTF-8 charset you can do:
tr -c '\0-\177' '[0*]' < file
However note that the GNU implementation of tr at least is not compliant in that regard as it doesn't support multibyte characters yet.
On GNU systems, you can use GNU awk for that though where, in recent versions ranges are based on code ...
1
This solved the issue
ssh $q "sed -i '/apache$/a sshd: 192.168.2.2 : allow' /etc/hosts.allow" &>> log.txt
1
$ perl -F, -lne 'if ($#F == 7) {$F[4] .= ",$F[5]"; splice @F,5,1};
print join("|",@F);' input.csv
Name|Age|Country|ID|Address|Category|DOB
John Doe|19|England|3653|Manchester, England|Main Worker|20-05-1995
This perl one-liner checks to see how many comma-separated fields are in each input line. If there are 8 fields, ...
1
#!/bin/bash
perl -le '
for (@ARGV) {
# separate command line args into filename(s) and line-number(s)
# line-numbers can be space and/or comma separated.
if (-e $_) { push @files, $_ } else { push @lines, split /,/};
};
@ARGV = @files;
$re = join("|",@lines);
while(<>) {
print join("\t",(split)[0..1]) if ($. =~ m/^($re)$/)...
1
The following bash script builds a sed script that will modify and display the lines by the numbers provided on the command line:
#!/bin/bash
sed_script=()
for lineno do
sed_script+=( -e "$lineno ba" )
done
sed "${sed_script[@]}" \
-e 'd' -e ':a' -e 's/[[:blank:]]*[^[:blank:]]*$//' <data.txt
or, for /bin/sh:
#!/bin/...
1
Put the below lines in a text file and name it as get.sh. then make it executable .
#!/bin/sh
## this is GNU sed
sed -En "
$(printf '%sbp\n' "$@" "d;")
:p;s/\S+/&\n/2;P
" data.txt
Now invoke the script as shown:
chmod +x ./get.sh
./get.sh 1 3 5
1
Using GNU sed we can do as follows:
sed -e '/# SECTION ONE/,/^$/s/^$/__NEW-DATA__\n/' file
Range operator in awk can be used to get the desired output
awk '
/# SECTION ONE/,!NF{
if (!NF) print "__NEW-DATA__"
}1
' file
This method uses POSIXly sed constructs
sed -e '
/# SECTION ONE/!b
:a
n;/^$/!ba
G;s/^/__NEW-DATA__/
' file
In ...
1
I wasn't able to do this with sed because the use of commas as the delimiter as well as the comma in the Name field not being a delimiter made it too complicated so I used awk with the gsub option:
awk -F ',' 'NR==1; NR>1{gsub("\.0","",$7);print} ' OFS="," file
This is somewhat messy given the circumstances but it sets a ...
1
Adding a fake root node around your broken XML is easy enough, modifying the modified XML would be a breeze with xmlstarlet, and deleting the added root node (and the <?xml version="1.0"?> that xmlstarlet adds) once you've modified it is not too hard either:
{ echo '<root>'; cat file.xml; echo '</root>'; } |
xmlstarlet ed -u '//...
1
Assuming you are on GNU sed version of the sed editor.
Try to null the path d data in both fileold and filenew files before diffing. So you could do something along these lines:
sed -i '
/<path d=/c\
<path d=/>
' -- "$fileold" "$filenew";
Or if you must need to make sure that the characters in between the quotes are only ...
1
You need to protect the backslashes from the shell on the command line either by using quotes OR double them up.
myscript.sh
'my_server\/my_repo\/my_image'
..... Or
myscript.sh my_server\\/my_repo\\/my_image
1
GNU sed in extended regex mode -E we use the range operator , to select the right block and within that block use the regex of the s/regex/repl/ to zero in on the right line to make the edits.
sed -Ee '
/\[ req_distinguished_name ]/,/^$/s/^(\s*commonName\s*=).*/\1__NEWVALUE__/
' file
Only top voted, non community-wiki answers of a minimum length are eligible
