2

I have the misfortune of dealing with filenames that contain spaces. I want to concatenate files of which filenames contain spaces. I also want to sort the filenames numerically. Obviously the following fails:

cat $(ls *.sql | sort -n)

since foo bar.sql is passed as two arguments to cat. What is the usual approach here?

  • 1
    The usual approach is to avoid parsing the output of ls. Use find instead. find can use a NUL byte as the delimiter, and both sort and xargs can accept that delimiter, so that approach can even handle filenames that contain \n. – PM 2Ring Nov 17 '14 at 10:47
  • Note that that code would also fail (not do what you want) if filenames contained wildcard characters (*, ?, [), tab or newline, started with - or some of the *.sql files were of type directory (in which case ls lists their content). – Stéphane Chazelas Nov 17 '14 at 11:23
7

No need for ls here. It's the shell that lists the directory content by expanding the *.sql glob.

On a GNU or FreeBSD system:

printf '%s\0' *.sql | sort -nz | xargs -r0 cat --

(using \0 instead of \n together with -z/-0 makes sure it also works with file names containing newline characters).

Note that the numeric sorting with -n assumes the number is at the start of the filename.

Or if you have zsh:

cat ./*.sql(.n)

(The n glob qualifier is to enable numeric sorting (also works when the number is not at the start provided all file names have the same prefix (like file12.sql, file2.sql). I added . as well to only include regular files. Add D if you also want hidden files like .foo.sql).

1

You can do cat $(ls -1) if you override IFS. IFS is the shell variable that tells BASH which characters to use as a deliminator. The default value for IFS is space, tab, newline. If you change IFS to just newline then you can do cat $(ls -1).

There is nothing wrong with the other answers but this might be a more direct answer to how to deal with spaces in general and this answer introduces the IFS variable which most are not familiar with.

#!/bin/bash
dir=$(mktemp -d)
for x in $(seq 10); do
    echo $x > "$dir/$(date) $x.txt"
done

pushd $dir
ls -1 $dir

# Set IFS to newline only inorder to deal with the spaces in the file names
#
oldIFS=$IFS
IFS=$'\n'


cat $(ls -1 $dir)


IFS=$oldIFS

Output

[sri@localhost test]$ ./test
/tmp/tmp.IuXCBzbTLj ~/test
Mon Nov 17 06:38:52 EST 2014 10.txt
Mon Nov 17 06:38:52 EST 2014 1.txt
Mon Nov 17 06:38:52 EST 2014 2.txt
Mon Nov 17 06:38:52 EST 2014 3.txt
Mon Nov 17 06:38:52 EST 2014 4.txt
Mon Nov 17 06:38:52 EST 2014 5.txt
Mon Nov 17 06:38:52 EST 2014 6.txt
Mon Nov 17 06:38:52 EST 2014 7.txt
Mon Nov 17 06:38:52 EST 2014 8.txt
Mon Nov 17 06:38:52 EST 2014 9.txt
10
1
2
3
4
5
6
7
8
9

Here is a link to some documentation on IFS. http://bash.cyberciti.biz/guide/$IFS. I think that I learned about it a long time ago by reading O'Rielly's BASH book.

  • 1
    Except with zsh, you also need to disable globbing with set -f. Leaving a variable or command substitution unquoted is the split+glob operator. You don't want the glob part here. You do want globbing for expanding *.sql though. So that should be: IFS=$'\n'; set -f; cat -- $(set +f; ls -d -- *.sql). But, as I said, you don't need ls here, and you'd still have a problem with newline characters in file names. – Stéphane Chazelas Nov 17 '14 at 12:51
  • I can't speak to zsh. I just wanted to introduce readers to the IFS variable. Certainly there are many better methods to solve this problem but I wanted to show the readers that it could be accomplished with a call to ls if that's what they feel comfortable with. – shrewmouse Nov 17 '14 at 13:10
  • There's a much simpler method for people who want to stick to familiar tools. Yes, sure, IFS exists, but it isn't a particularly useful way to solve this particular problem. And if you do use it, use it as Stéphane suggested, in a way that at least solves the problem for all non-newline characters. – Gilles Nov 18 '14 at 7:49
1

Use xargs (assuming the GNU implementation) with a custom delimiter (assuming filenames don't contain newlines):

ls -1d -- *.sql | sort -n | xargs -d "\n" cat --

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.