department.txt contains column as ID, Department I am using below to change the order to Department,ID enter image description here

But if i further update the command to below enter image description here

Then department.txt becomes empty, without any output. If i use some other file instead of the same file which i am reading, then it works.

I understand i am reading and updating the same file, but as per my understanding is | should take care of it, as internally it must be storing the the output somewhere in its memory and dumping it in the file which i have asked. Isnt it ? Can someone throw some insight on how it works ?

Note: I know there are many similar questions, but none of them have really answered how internally | works, which is the fundamental question for which i need an answer.

marked as duplicate by muru, Kusalananda linux Apr 7 '18 at 6:49

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 5
    Please post plain text, not hard-to-read screenshots. – Barmar Apr 7 '18 at 6:37
  • You're not updating the same file. Input is department.txt, output is departmet.txt (no n). – Barmar Apr 7 '18 at 6:39
  • Thats a typo. I am updating the same file. – LoveWithMaths Apr 7 '18 at 6:43
  • 1
    I'm just pointing out that your question doesn't really show the problem you're having. Post the real command you executed. – Barmar Apr 7 '18 at 6:50

No, piping does not take care of it. The pipe doesn't run the two commands sequentially and store the output of the first one in memory. The two programs run at the same time, and they're connected using a special I/O device called a pipe. But before it starts the programs, the shell sets up all the I/O redirections, which means it opens the output file and truncates it.

If you have GNU awk, you can use its inplace option to overwrite the input file.

gawk -i inplace -F'\.' -v OFS=. '{print $2, $1}' department.txt
  • Good to hear alternatives, but first i need to clarify my understanding on the fundamentals itself. So | will execute the program parallely, but how does it synchronise execution ? Still unclear about why it gets truncated ? – LoveWithMaths Apr 7 '18 at 6:52
  • 1
    The shell does redirection. It truncated department.txt before anything else happened. – Barmar Apr 7 '18 at 7:01
  • 1
    > is not a command, there's nothing sequential about it. In order to redirect the output of a command, the shell has to open the file (truncating it), connect the process's stdout to that file descriptor, then start the program running. – Barmar Apr 7 '18 at 7:52
  • 1
    Just remember, output isn't saved in any temporary memory. If the process is writing to an output file, it has to be opened and truncated before the process starts writing. – Barmar Apr 7 '18 at 7:55
  • 1
    BTW, <( is a pipe in disguse. – Barmar Apr 7 '18 at 7:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.