5

In my directory I have two files with space, foo bar and another file. I also have two files without space, file1 and file2.

The following script works:

for f in foo\ bar another\ file; do file "$f"; done

This script also works:

for f in 'foo bar' 'another file'; do file "$f"; done

But the following script doesn't work:

files="foo\ bar another\ file"
for f in $files; do file "$f"; done

Not even this script works:

files="'foo bar' 'another file'"
for f in $files; do file "$f"; done

But, if the files do not contain space, the script works:

files="file1 file2"
for f in $files; do file "$f"; done

Thanks!

Edit

Code snippet of my script:

while getopts "i:a:c:d:f:g:h" arg; do
  case $arg in
    i) files=$OPTARG;;
    # ...
  esac
done

for f in $files; do file "$f"; done

With files without spaces, my script works. But I would like to run the script passing files with spaces as argument in one of these ways:

./script.sh -i "foo\ bar another\ file"
./script.sh -i foo\ bar another\ file
./script.sh -i "'foo bar' 'another file'"
./script.sh -i 'foo bar' 'another file'
7
  • 1
    You may want to try using an array. This should explain how to use a bash array with elements that have spaces in them and how to iterate over the array: stackoverflow.com/questions/9084257/…
    – Natolio
    Apr 8, 2021 at 16:16
  • 3
    Please edit your question and clarify how files will be populated. Will it be all arguments? Some? A command line option? Also, this is almost certainly a dupe of Why does my shell script choke on whitespace or other special characters?. Does it answer your question?
    – terdon
    Apr 8, 2021 at 16:26
  • Your last code snipped: It makes no sense to use getopts in this script if you only have a single option that signals "use these files". The script could just loop over "$@" to process the files.
    – Kusalananda
    Apr 8, 2021 at 17:14
  • 1
    @PedroSiqueira Is the -i option the only one that takes what amounts to multiple values?
    – Kusalananda
    Apr 8, 2021 at 17:38

4 Answers 4

9

If you're using bash you can use an array for this

#!/bin/bash
files=('foo bar' 'another file' file1 'file2')
for f in "${files[@]}"; do file -- "$f"; done

Quoting is required for file names containing whitespace; it's optional (but I'd recommend it) for plain file names. If the list of files comes from the current directory you can use wildcards as you'd expect, e.g. files=(*f*) to match any file or directory with f in its name. (But then you could probably just use for f in *f*; do...done and avoid the array entirely.) The -- marker for file tells it that any subsequent parameter is a filename - even if it starts with a dash.

Read more with man bash (search for Arrays).

0
5

For your command line parsing, arrange with the pathname operands to always be the last ones on the command line:

./myscript -a -b -c -- 'foo bar' 'another file' file[12]

The parsing of the options would look something like

while getopts abc opt; do
     case $opt in
         a) a_opt=true ;;
         b) b_opt=true ;;
         c) c_opt=true ;;
         *) echo error >&2; exit 1
    esac
done

shift "$(( OPTIND - 1 ))"

for pathname do
    # process pathname operand "$pathname" here
done

The shift will make sure to shift off the handled options so that the pathname operands are the only things left in the list of positional parameters.

If that's not possible, allow the -i option to be specified multiple times and collect the given arguments in an array each time you come across it in the loop:

pathnames=()

while getopts abci: opt; do
     case $opt in
         a) a_opt=true ;;
         b) b_opt=true ;;
         c) c_opt=true ;;
         i) pathnames+=( "$OPTARG" ) ;;
         *) echo error >&2; exit 1
    esac
done

shift "$(( OPTIND - 1 ))"

for pathname in "${pathnames[@]}"; do
    # process pathname argument "$pathname" here
done

This would be called as

./myscript -a -b -c -i 'foo bar' -i 'another file' -i file1 -i file2
2
  • It worked! Thanks! But what if I want to pass a wildcard like ./script -a -b -c -i "*.mp4", and process all files ending ".mp4"? Apr 8, 2021 at 17:59
  • 2
    @PedroSiqueira I would avoid this if possible. You would have to find a safe way to expand that globbing pattern to a list of pathnames, and allow for it to also not be a pattern (or, if you have a separate option for that). One could possibly plug it into find, but that would be fiddly. You could definitely ask a separate question about that, because it's most definitely non-trivial to do safely.
    – Kusalananda
    Apr 8, 2021 at 18:04
3

There's a bit of a difference between the four script invocations you posted.

./script.sh -i "'foo bar' 'another file'"
./script.sh -i "foo\ bar another\ file"

The above two both pass to the script -i as the first argument, and a single string as the second. In the first one, that's 'foo bar' 'another file', and in the second, it's foo\ bar another\ file. In the shell language, both are valid ways to present the two strings (or filenames) foo bar and another file. But the quote and backslash processing only applies when the strings are on a raw command line, not when they're inside a variable, as they end up in the string.

./script.sh -i foo\ bar another\ file
./script.sh -i 'foo bar' 'another file'

On the other hand, these two pass a total of three arguments: -i, foo bar, and another file.

The difference is somewhat important in that it's much easier to deal safely with distinct arguments. You just need to keep them intact, and don't have to process the quotes and escapes embedded within.

Also, importantly, running something like script ./*.txt will pass the filenames as distinct arguments.

E.g. this would just call file on both files if called as script 'foo bar' another\ file:

#!/bin/sh
for f in "$@"; do
    file "$f"
done

But you have the getopts there, too. And with the filenames as distinct arguments, only the first would appear as the argument to -i. Here, there's basically two common options.

Either have the user use the -i option repeatedly, collecting the filenames to an array, so:

#!/bin/bash
files=()
while getopts "i:" arg; do
  case $arg in
    i) files+=("$OPTARG");;
  esac
done

for f in "${files[@]}"; do file "$f"; done

and run as script -i "foo bar" -i "another file". (Running script -i file1 file2 would have file2 ignored.) Similarly you could add another array to collect filenames given through another option.

Or, have the option set the "mode" the script works in, and take the filenames as a list distinct from the options. getopts leaves all the arguments intact, you'll just have to drop the ones it processed with shift. So:

#!/bin/bash

while getopts "ie" arg; do
  case $arg in
    i) mode=i;;
    e) mode=e;;
  esac
done
shift $(($OPTIND - 1))

if [ "$mode" = i ]; then
    for f in "$@"; do file "$f"; done
elif [ "$mode" = e ]; then
    echo "do something else with the files"
else
    echo "error: invalid mode" >&2
fi

and then run it as script -i "foo bar" "another file".

I'm assuming here that you are also doing something else with getopts other than taking the -i in, since otherwise you could just drop the flag entirely. :) But what other options you have, somewhat affects what the most sensibly (or customary) solution is.

Also if your loop only calls file on the files, you could just run file "${files[@]}" or file "$@" and skip the loop.


However, if you want to be able to this:

script -i foo bar -e doo daa

and have the script to do one thing for the files foo and bar, and another thing for doo and daa, then that's a bit of a different issue. It can be done, sure, but getopts might not be the tool for that.

See also:

And of course:

for the issues with trying to deal with multiple distinct arbitrary strings (filenames) within a single variable.

-1

Bash is not too good at dealing with variables with space in them.

you have to use the "rename" functionality to help you with this a bit like below:

#use below if you are on a debian like system

sudo apt-get install rename

#use below if you are on a red hat like system

sudo yum install install rename

#cd into the directory where your files are

cd your_target_directory

#rename all the files/variables containing the space character #with that step no more problems with the space

rename 's/ /_/g' *

#proceed without issue from here...enjoy

1
  • 4
    Note that spaces is just one of several problematic characters that are valid in filenames. Others includes tabs, newlines, characters that are part of globbing patterns (such as * and ? etc.), control characters, emojis and other non-latin characters (visible and invisible). It's easier to write code that correctly handles all valid filenames, than to rename files that broken code can't handle.
    – Kusalananda
    Apr 8, 2021 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.