I have been trying to pipe this command

tail -n +$line $FILENAME

to this command

parallel -a $FILENAME -k --block $sub --pipe-part "cut -d ' ' -f1" | sponge $FILENAME

And the resulting command is as follows:

tail -n +$line $FILENAME | parallel -k --block $sub --pipe-part --eta --progress "$cut -d ' ' -f1 | sponge $FILENAME

I also tried the same thing with --pipe instead of --pipe-part

Why is this command emptying my file completely? How does piping work with the parallel command?

  • 1
    Because the input must be seek()-able when you use --pipepart. – Satō Katsura May 19 '17 at 18:42
  • 1
    What error message do you get, or what happens incorrectly? – Jeff Schaller May 19 '17 at 18:43
  • @JeffSchaller the file gets emptied – HashWizard May 19 '17 at 18:44
  • 1
    is this running in a script? If so you must set job control using set -m – Valentin Bajrami May 19 '17 at 18:48
  • 1
    From the manual: --pipepart has a few limitations: The file must be a normal file or a block device (technically it must be seekable) and must be given using -a or ::::. The file cannot be a pipe or a fifo as they are not seekable. – Satō Katsura May 19 '17 at 19:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.