I want to know how to use grep
in order to display all lines that begin and end with the same character.
POSIXly:
pattern='\(.\).*\1
.'
grep -x -- "$pattern" file
It won't work if line starts or ends with invalid byte character, if you want to cover that case, you can add LC_ALL=C
, although LC_ALL=C
works with single-byte character data only.
perl6
seems to be the best tool, if you have it in your box:
$ printf '\ue7\u301 blah \u107\u327\n121\n1\n123\n' |
perl6 -ne '.say if m/^(.).*$0$/ || /^.$/'
ḉ blah ḉ
121
1
Although it still chokes on invalid characters.
Note that perl6
will alter your text by turning it to NFC
form:
$ printf '\u0044\u0323\u0307\n' |
perl6 -pe '' |
perl -CI -ne 'printf "U+%04x\n", ord for split //'
U+1e0c
U+0307
U+000a
$ printf '\u0044\u0323\u0307\n' |
perl -pe '' |
perl -CI -ne 'printf "U+%04x\n", ord for split //'
U+0044
U+0323
U+0307
U+000a
Internally, perl6
stores string in NFG
form (stand for Normalization Form Grapheme
), which is perl6
invented way to deal with un-precomposed graphemes properly:
$ printf '\u0044\u0323\u0307\n' | perl6 -ne '.chars.say'
1
$ printf '\u0044\u0323\u0307\n' | perl6 -ne '.codes.say'
2
-
2Perl's handling of Unicode text is nothing short of exemplary, to the point that many "simple" tasks in Perl are practically impossible to implement using other tools, at least with the same level of correctness. – Dietrich Epp Sep 21 '16 at 19:25
-
1It should be noted that
perl6
will alter the text though (turn it to NFC (Normalization form "composed")). – Stéphane Chazelas Sep 21 '16 at 23:06 -
@StéphaneChazelas: Yes, fair point. Also note that string in
perl6
is store inNFG
form (G
forGrapheme
), which isperl6
way to deal with un-precomposed graphemes properly. – cuonglm Sep 22 '16 at 3:56
Not grep but awk:
awk -F "" 'NF && $1 == $NF'
These special cases are handled:
- it does not print empty lines
- it always prints 1-character lines
An empty FS splits the record into one character per field in gawk
, mawk
and busybox
awk
(bytes, not characters for the latter two), but is not standard and doesn't work in the implementations of awk
derived from the original one by A, W and K like on BSDs and commercial Unices. More portable but more to type:
awk '/./ && substr($0,1,1) == substr($0,length)'
-
1Note that
FS
as empty string is not standard, and won't work in someawk
implementation. – cuonglm Sep 21 '16 at 9:15 -
2Alternative that avoids splitting and is fully portable (even to the maximally awful Solaris 'old' awk)
awk 'length&&substr($0,1,1)==substr($0,length)'
(note default argument oflength
is$0
, and default action is{print $0}
) – dave_thompson_085 Sep 21 '16 at 11:03 -
@dave_thompson_085: thx, I'm only using your default action hint to have the shortest command. – rudimeier Sep 21 '16 at 11:23
-
Firne. One minor correction; my test for Solaris old awk was mistaken (I accidentally had xpg4 on) but this method does work in
nawk
which is almost as bad :-) – dave_thompson_085 Sep 21 '16 at 12:09
grep -xe '\(.\).*\1' -e .
Example:
$ printf '%s\n' il y était cet été | grep -xe '\(.\).*\1' -e .
y
été
-x
is for exact match (match on the whole line). \1
being a back-reference to the character captured in \(.\)
. We add a -e .
to take care of the special case of a line containing one single character.
It assumes the input contains valid text in the current locale.
The match is on character, not byte (those é in UTF-8 are the two bytes 0xc3 0xa9 for instance), nor graphem cluster (it wouldn't work if those é were written in their decomposed form with e
followed by the U+0301 combining acute accent for instance).
To work on graphem clusters, with a grep
that supports -P
for PCRE:
$ printf 'e\u0301te\u0301\n' | grep -xPe '(\X).*\1|\X'
été
That assumes the decomposition is the same for the two clusters, for instance a ḉ
expressed as c
U+0301
U+0327
would not match one expressed as c
U+0327
U+0301
or ć
(U+0107
) U+0327
or ç
(U+00E7
) U+0301
or ḉ (U+1E09
). For that, you'd need to do the check on a normalized form:
$ printf '\ue7\u301 blah \u107\u327\n' |
perl -MUnicode::Normalize -C -ne '
print if /^\X$/ || NFC($_) =~ /^(\X).*\1$/'
ḉ blah ḉ
-
1If you have
perl6
, thenperl6 -ne '.say if m/^(.).*$0$/ || /^.$/'
should do all the works for you. – cuonglm Sep 21 '16 at 12:50
Quick python2 alternative:
python -c 'import sys;[sys.stdout.write(l) for l in sys.stdin if len(l)>1 and l.rstrip("\n").endswith(l[0])]' < input.txt
Example:
$ python -c 'import sys;[sys.stdout.write(l) for l in sys.stdin if len(l)>1 and l.rstrip("\n").endswith(l[0])]' < input.txt | cat -A
nathan$
ookie $
a line a$
-
It's failed if line contains trailing or leading spaces, example ` 121`. – cuonglm Sep 21 '16 at 10:33
-
@cuonglm that's true. But was trailing or leading whitespace a requirement ? This does the job asked - check if the leading and last character are the same. Whitespace is still an ascii character, no ? – Sergiy Kolodyazhnyy Sep 21 '16 at 10:36
-
@cuonglm yours failed with trailing and leading space too, by the way :) – Sergiy Kolodyazhnyy Sep 21 '16 at 10:37
-
Your code removes leading and trailing white spaces, so it changes the input line. Also it gives an error for empty lines. – rudimeier Sep 21 '16 at 10:49
-