0

i try run docker with entrypoint.sh, but it doesn't exequite .sh line:

echo `ls -d /input/sub-*/ | sed -e 's/.*sub-\(.*\)\//\1/' | split -l 8 - participants_`

I wrote it in dockerfile:

ENTRYPOINT ["bash", "-c", "source /code/entrypoint.sh | ts '[%Y-%m-%d %H:%M:%S]' &>> /output/stderr.log"]

Why echo doesn't execute inside docker run [my_image]

Here is full entrypoint.sh code:

#! /bin/bash

alias time='/usr/bin/time -f "%C --- CPU:\t%E real,\t%U user,\t%S 
sys\t%P\tMem:\t%KkiB avg.,\t%MkiB max.\tExit:\t%x"'

echo `ls -d /input/sub-*/ | sed -e 's/.*sub-\(.*\)\//\1/' | split -l 8 - 
participants_`

while read input_bids_path
do      
    participants_id=$(basename $input_bids_path)
    LD_LIBRARY_PATH=/usr/lib/fsl/5.0:$LD_LIBRARY_PATH
    time fmriprep /input /output participant --fs-license-file 
/opt/freesurfer/license.txt --fs-no-reconall --use-aroma --ignore fieldmaps 
--n_cpus 12 --force-bbr --participant_label $(cat $participants_id) -w 
/output
#   rm -r /input/$participants_id
done < <(find /input -name "*participants_*" -type f)
echo  `rm -r /input/$participants_id`
wait `jobs -p` && echo __ok__ || echo __err__
8
  • Why do you use echo on command substitution? That rm will only remove the last /input/$participants_id as it's after the loop. Aliases are not expanded in shell scripts by default (and I don't see why you would want to use an alias here anyway). Did you try to run this outside of Docker to debug it? Did you run the individual commands to see that they executed in the way that you expected them to?
    – Kusalananda
    Mar 4, 2019 at 20:36
  • @Kusalananda, yes I ran it without docker in shell, everything is working fine. It’s strange that he’ll command doesn’t execute inside of it. Mar 4, 2019 at 20:46
  • You have't mentioned what command you are referring to. You said echo, but there's more than one of those.
    – Kusalananda
    Mar 4, 2019 at 20:49
  • @Kusalananda echo ls -d /input/sub-*/ | sed -e 's/.*sub-\(.*\)\//\1/' | split -l 8 - participants_ Mar 4, 2019 at 20:50
  • The command substitution does not produce anything for echo to output. There is nothing to echo. This is also the case for the second echo. It is unclear what you want these echo calls to actually do.
    – Kusalananda
    Mar 4, 2019 at 20:52

1 Answer 1

0

There's a number of things in this script that are wrong or that can be improved.

The main issue in the question seems to be why the two calls to echo does not produce any output.

When you use a command substitution, like in

echo `rm file`

or the equivalent

echo $(rm file)

then the echo will get the output of the command within the backticks or inside the $(...). Neither of your command substitutions produces any output. Both commands that you use within backticks modify files, but again, neither produces output to their standard output stream (what would ordinarily be displayed in the terminal). This means that both calls to echo will produce no output either, apart from an empty line each.

In general, echo $(...) is an anti-pattern, meaning you can do the same thing in a much better way.

If you do want to output the result of some pipeline pipeline, then instead of writing

echo $(pipeline)

you would simply say

pipeline

The output of pipeline would be displayed, as the output of commands are usually displayed in the terminal.

In the code below, I have inserted a couple of printf statements that will output the relevant "progress information" in the script(s).


Here's a modified version of the script which is totally untested (as I don't have access to the tools that you use, or the input files), but it should mimic what your script is doing, including creating those intermediate files (these are not needed, and I'll show how to get rid of them later).

#!/bin/bash

export LD_LIBRARY_PATH="/usr/lib/fsl/5.0:$LD_LIBRARY_PATH"

timefmt="%C --- CPU:\t%E real,\t%U user,\t%S sys\t%P\tMem:\t%KkiB avg.,\t%MkiB max.\tExit:\t%x"

for dirpath in /input/sub-*/; do
    name=$(basename "$dirpath")
    id=${name#sub-}
    printf '%s\n' "$id"
    printf 'Found ID: %s\n' "$id" >&2
done | split -l 8 - participants_

for participants_id in participants_*; do
    ids=( $(<"$participants_id") )

    printf 'Processing ID: %s\n' "${ids[@]}" >&2

    /usr/bin/time -f "$timefmt" \
    fmriprep /input /output participant \
        --fs-license-file /opt/freesurfer/license.txt \
        --fs-no-reconall --use-aroma \
        --ignore fieldmaps --n_cpus 12 --force-bbr \
        --participant_label "${ids[@]}" \
        -w /output

    rm -f "$participants_id"
done

Fixes:

  1. The time command does not need to be an alias just due to having a long option argument to its -f option. Aliases aren't expanded in scripts anyway. I simply save the argument in a string and use that when invoking time.

  2. Your loop added to LD_LIBRARY_PATH in each iteration. This was not needed.

  3. The getting of the IDs from the directory names is better done in a proper loop. This loop will disappear later when we use an array to store the IDs instead.

  4. Rather than using find to locate the intermediate files, we just use them with a simple filename globbing pattern. We know they're right there and what their names are anyway.

  5. The intermediate file just processed is deleted inside the loop.

  6. The code is made readable through use of line continuations.

  7. The call to wait was removed. There are no background tasks to wait for.

The following variation stores the IDs in an array, all_ids instead of in temporary files:

#!/bin/bash

export LD_LIBRARY_PATH="/usr/lib/fsl/5.0:$LD_LIBRARY_PATH"

timefmt="%C --- CPU:\t%E real,\t%U user,\t%S sys\t%P\tMem:\t%KkiB avg.,\t%MkiB max.\tExit:\t%x"

all_ids=( /input/sub-*/ )
all_ids=( "${all_ids[@]#/input/sub-}" ) # remove "/input/sub-" from each item
all_ids=( "${all_ids[@]%/}" )           # remove the trailing "/" from each item

printf 'Found ID: %s\n' "${all_ids[@]}" >&2

n=0
ids=( "${all_ids[@]:0:8}" ) # pick out the first eight IDs

# Loop until the first ID in the ids array is empty
while [ -n "${ids[0]}" ] ; do
    printf 'Processing ID: %s\n' "${ids[@]}" >&2

    /usr/bin/time -f "$timefmt" \
    fmriprep /input /output participant \
        --fs-license-file /opt/freesurfer/license.txt \
        --fs-no-reconall --use-aroma \
        --ignore fieldmaps --n_cpus 12 --force-bbr \
        --participant_label "${ids[@]}" \
        -w /output

    n=$(( n + 1 ))
    ids=( "${all_ids[@]:n*8:8}" ) # pick out the next eight IDs
done
7
  • first of all - wow, thanks of extended answer. Seconldy, does your script,does you code interpretation allow to iterate throught unknown number of n elements generated with all_ids? Mar 5, 2019 at 12:04
  • @Relyativist I believe it should cope with a variable number of IDs, no matter if these are a multiple of 8 or not. As I said, I haven't tested this, so I'm not 100% certain that it works as it should, but you should at least be able to use it to get something working at your end. In particular, I am no Docker user.
    – Kusalananda
    Mar 5, 2019 at 12:15
  • as i said it doesn't work like that inside container: root@9bca6f96475d:/tmp# all_ids=( /input/sub-*/ ) root@9bca6f96475d:/tmp# all_ids=( "${ids[@]#/input/sub-}" ) root@9bca6f96475d:/tmp# all_ids=( "${ids[@]%/}" ) root@9bca6f96475d:/tmp# echo $all_ids *nothing* Mar 5, 2019 at 15:28
  • @Relyativist I had typos in my code. Now corrected. Thanks for pointing it out!
    – Kusalananda
    Mar 5, 2019 at 15:34
  • thanks, could you please give an idea how to check if subject in output already exists, and not to include into all_ids array? thanks Mar 5, 2019 at 16:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .