For example:


The output should be:

44788      # 7 is not repeated
66h1122    # letter h is not repeated 
PPDd88     # letters D or d are not repeated

Delete the line that all its characters are repeated at least two times in the same line.

If there was any character not repeated in the same line so print that line only.

  • 3
    Is this homework – eyoung100 Sep 28 '18 at 5:50
  • Is this some school exercise? – RalfFriedl Sep 28 '18 at 5:52
  • interesting problem, but I can't think of a way to solve it using regex alone with grep (even if PCRE is available).. also, please add what you've tried to solve this – Sundeep Sep 28 '18 at 6:27
  • Dear all I am new in this world of command line so I am trying my best to learn this asap so and searching studying and doing some exercises starting with the thoughts in my mind and studying all the details related to the answer because this is the best way for me to learn as i think ...Thanks – Ahmed Sep 28 '18 at 6:33
  • 1
    @Ahmed please see unix.stackexchange.com/tags/grep/info for some learning resources.. it is good that you want to learn, but on this forum you are expected to show what you've tried yourself.. that means going through tutorials, man pages etc, trying some code and then ask here when the code you tried didn't work – Sundeep Sep 28 '18 at 7:15

With perl:

perl -ne 'my %count;
          $count{$_}++ for /./g;
          print if grep {$_ == 1} values %count'

With sed:

sed '

We split the pattern space into two lines. Characters that are duplicated are moved to the second line in a loop. We print the record if at the end, the first line is not empty.

| improve this answer | |
sed -e '


  • Place a marker \n at the beginning of the pattern ,which travels to the right during the process.
  • We setup an infinite loop and provide for two exits inside the loop.
  • One, if during the process the whole string ins emptied leaving only the marker, we exit knowing that this string comprised all duplicate stuff.
  • Two, if during the process we find that the first element in the string is non-repeated. Implies, at least one non-duplicate located. Hence this line needs printing. We pull out the original which we had stored away in the hold area prior to beginning the process.
  • The alternative is that the first element has at least a duplicate somewhere in the string which we go about in t-loop totally purging that element's presence from the string.

Another way is usingPerl as shown:

perl -lne '
    my $s = $_;
    s/\Q$1\E//g while $s =~ /(.)(?=.*?\1)/g;
    print $s if /./;

Here we make a copy of the line, and continuously look for duplicated elements and keep stripping away these from the original line. After the while loop ends, we print the copy provided something remains in the line (meaning, the non-duplicated stuff was left behind).

| improve this answer | |
  • your command is ok with my question but if i want to delete the lines that all their characters are consecutive duplicates as 1122233 (Deleted) but not 1222313 (not deleted) – Ahmed Oct 17 '18 at 12:12

perl one-liner: removes all pairs of characters, print the line if there are characters left over.

perl -lne '($copy = $_) =~ s/(.)\1//g; print if $copy' file

As you say, the above is wrong: it will incorrectly print "56667775" because that answer only looks at pairs of characters. Look to Stéphane's answer for correctness.

| improve this answer | |
  • If i want to print only the lines that have all their characters are repeated on the same question above what would syntax for that command be? – Ahmed Sep 28 '18 at 15:54
  • I don't understand your question. If you want the "opposite" output, only lines with all repeated characters, change if to unless – glenn jackman Sep 28 '18 at 18:10
  • Jackman your command is not right with the queation cos it works only with the pairs of characters if there was a character repeated 3 times it will not work with it it will not be deleted. – Ahmed Sep 29 '18 at 13:55
  • yes, you are quite right. – glenn jackman Sep 29 '18 at 14:57

There is a command that extract only non-repeated words in a list:

$ printf '%s\n' one one two | uniq -u

You could divide every character in a word on each line and use uniq:

$ echo "1122e4455" | grep -o . | sort | uniq -u

All you have to do is loop for all the words to test and if the command above doesn't have any output, print the tested line.

| improve this answer | |

Adaption of the solution to your recent problem:

awk '
        {split ("", N)                                          # delete N array
         L = 0                                                  # reset boolean L used for print decision
         for (i=1; i<=length; i++)  N[substr($0, i, 1)]++       # calculate count of characters
         for (n in N) if (N[n] < 2)     {L = 1                  # for non-duplicate chars: set print decision
                                         break                  # and quit the for loop
L                                                               # print if non-duplicate chars exist
' file
| improve this answer | |
  • RudiC your commant is not working with the space character it doesnt recognize the space if it was found in a line. – Ahmed Sep 28 '18 at 10:48
  • 2
    You didn't specify having more than one field in your input file. Change $1 to $0 (done that in above answer). PLEASE be way more precise in future questions, and develop some creativity of your own. – RudiC Sep 28 '18 at 11:45
  • RudiC thank you for your answer now its ok but when i said CHARACTER in my question that is covering all the characters including the space and all other special symbols so no need to mention them one by one – Ahmed Sep 28 '18 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.