Explaining is hard I want to start with example. Please see the Bold text.

I have very long list like the following list :


I want to find all Duplicate lines that included same strings after /film/, but I don’t want to search the line to the end, the search should look for duplicate strings just like 10 or 20 characters after /film/

  • The strings after /film/ are different.
  • The method should search a list with like 5 thousand line and find all duplicate line and maybe after founding delete, cut, copy or mark One or Two of them.

Thanks in advance

  • Here's a start: cat long-list |grep -Eo '/film/.{15}' |sort |uniq --repeated. If this isn't what you want, please explain in more detail.
    – bitinerant
    Apr 4 '20 at 10:32
  • Are the * significant, especially between T*he.Wi and **The.Wi ? Would terminating near *.number be reliable? Apr 4 '20 at 10:39


sed -n 's/^/@/' > newfile


sed -ni 's/^/@/'

Both commands are inserting an @ at the beginning every libe of your last command.


nice, command output for

cat long-list |grep -Eo '/film/.{15}' |sort |uniq --repeated

is a list that contain all duplicate names. now how can i mark all duplicate line with something like a @ in start line?

  • You copied my comment into an answer and presented it as your own. Also, don't put questions in the 'answer' section. Ask a new question.
    – bitinerant
    Apr 5 '20 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.