I am appending a command output to a text file. But if I do it again it will have the same text twice in the text file. Is there a way with for example sed that if a word already exist don't add a new word, but just replace the old one?

What I am doing is appending the output of a ls command to a text file. So sometimes there are new files that have to be added to the text file, but then the files that were already there are now twice in the text file if you append again the ls output.

I hope you understand what i mean.

I am using AWS linux AMI


I used

ls client_certs/ >> message.txt

I didn't use the sed command yet. If I do that ls command twice then I get double names for example:

  • After first ls client_certs/ >> message.txt:
    1. 2. 3.
  • After second ls client_certs/ >> messages.txt:
    1. 2. 3. 1. 2. 3

What I want is:

  • If I do the first time ls client_certs/ >> message.txt:
    1. 2. 3.
  • If i do the second time ls client_certs/ >> message.txt :
    1. 2. 3.
  • If there is a new file in client_certs/ (for example 4.) and I want to do ls client_certs/ >> message.txt it should show like this:
    1. 2. 3. 4.
  • 2
    Hi, please provide us some input and expected output examples, and what you have tried so far. – Francesco Lucianò May 28 at 11:55
  • Please, edit your question to add these details – Francesco Lucianò May 28 at 13:30
  • 2
    Why don't you use > instead of >>? Do you also want to keep in the record files that were deleted or moved away? – Quasímodo May 28 at 15:15
  • No if a file is removed in client_certs then I also want it gone in the text file – Danny May 29 at 10:22
  • 1
    Then why are you appending (>>) as opposed to overwriting (>)? What is the information originally present in message.txt that you want be preserved. – Stéphane Chazelas May 29 at 10:51
touch message.txt # the file must exist even if it is empty
ls client_certs | grep -vxFf message.txt >>message.txt
  • -xF checks for whole literal (no regex) lines
  • -f message.txt checks all lines in the file message.txt
  • -v inverts the matches i.e. instead of printing everything in the file it prints everything which is not in the file

delete old entries

The most simple solution is counting the entries in the directory and the file. If the numbers differ then delete and recreate the file:

touch message.txt # the file must exist even if it is empty
ls client_certs | grep -vxFf message.txt >>message.txt
# update the file, then compare
# save some time by trusting mtime
if [ client_certs -nt message.txt ] && [ $(wc -l message.txt) -ne $(ls -U client_certs | wc -l) ]; then
    : >message.txt # set file size to 0
    ls client_certs | grep -vxFf message.txt >>message.txt

Of course, the question arises why the file should not just be deleted and recreated every time.

| improve this answer | |
  • Would you mind adding an explanation on how this performs the desired task (your answer popped up in my "Low quality post" review queue, which I find somewhat sad ... ;) ) – AdminBee May 29 at 8:11
  • @AdminBee Wouldn't that be a feature if the answers of "trusted users" got whitelisted... – Hauke Laging May 29 at 8:38
  • Yeah, that' true ;) – AdminBee May 29 at 8:45
  • Thanks for your Answer. This helped me alot, is there also a way for example if a file got deleted out of the client_certs file that it also can be deleted out of the text file? I changed the command a little bit because I only want files with .ovpn extension in the text file: ls client_certs | grep '.ovpn' | grep -vxFf message.txt >>message.txt I hope there is a way that if a file gets deleted out of client_certs it will also be deleted out of the text file. Again thanks for your answer. – Danny May 29 at 10:18
  • @Danny What do you mean by "deleted"? It would be the easiest solution to just recreate the file after a mismatch has been detected. – Hauke Laging Jun 2 at 0:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.