7

Observe:

$ ls
$ ls > list
$ cat list
list

This appears to indicate that when ls is executed that the redirection into file list has already begun and the list file is already created. A fine enough explanation at any rate, but the question is this: How can I prevent this from happening? What I expected to happen was ls would execute and its output dumped into list and that is what I want.

15

As you've noticed, the file is created before ls is run. This is due to how the shell handles its order of operations. In order to do

ls > file

the shell needs to create file and then set stdout to point to that and the finally run the ls program.

So you have some options.

  1. Create the file in another directory (eg /tmp) and then mv it to the final directory
  2. Create it as a hidden file (.file) and rename it
  3. Use grep to remove the file from the output
  4. Cheat :-)

The cheat would be something like

x=$(ls) ; printf "%s\n" "$x" > file

This causes the output of ls to be held in a variable, and then we write that out.

| improve this answer | |
  • Nice. Your last paragraph prompted me to try echo `ls` > ls which I love for being so concise, but this doesnt put an entry on each line which my particular situation has issues with (as I very much do need to deal with directories with spaces in them today)... – Steven Lu Jul 16 '16 at 0:17
  • ls -1 will force single-column output – Stephen Harris Jul 16 '16 at 0:21
  • @StephenHarris Yeah I think i might as well give you the accept so somebody gets points, this answer is 99% of the way there. Do you think there are any race conditions or failure modes of echo "$(ls -1)" > file? – Steven Lu Jul 16 '16 at 0:21
  • Yes, there are potential race conditions. If you want to use this sort of cheat you really should split the ls and the file creation into two steps. Also beware of echo potentially interpreting the output of the ls; (what if you have a file called -ne ?). That's why I used printf in my example. – Stephen Harris Jul 16 '16 at 0:24
  • Note that x=$(ls) removes all the trailing newline characters from the output of ls while your printf '%s\n' adds only one back. – Stéphane Chazelas Jul 16 '16 at 10:36
10

The output file is created by the shell before ls begins. You can get around this by using tee:

ls | tee list

To thoroughly defeat any race condition, there is always

ls | grep -vx 'list' > list

Or if you like that tee displays the results as well:

ls | grep -vx 'list' | tee list

However, as pointed out in comments, things like this often break when filenames contain strange characters. Unix filenames can generally contain any characters except for NUL and /, so parsing the output of ls is extremely difficult:

  • Assigning to a shell variable can fail if a filename ends in one or more \n.
  • Filtering with grep fails when the search term lies between \n.
  • You can separate filenames with NUL instead of \n using find, but it can be difficult to convert this into something resembling the traditional sorted, newline-separated output of ls.
  • Removing the output filename from the list may be incorrect if it already exists.

So the only truly effective way to do this is to create the output file somewhere else, and move it into place. If you will never use ls -a, then this works:

ls > .list && mv .list list

If you might be using ls -a, then .list could appear in your output but no longer exist in the directory. So then you would use a different directory, like /tmp to store the intermediate result. Of course, if you always use /tmp you run into trouble there, so you can write a script:

#!/bin/sh
OUTDIR='/tmp'
if [ "${PWD}" = '/tmp' ]; then
  OUTDIR="${HOME}"
fi
ls > "${OUTDIR}/list" && mv "${OUTDIR}/list" list

This seems overly complicated for the task, though.

But the entire cause of the issue is that the shell is creating the output file before the command begins. We can take that into consideration and just have the shell list the files for us. Then we don't even need ls at all!

printf '%s\n' * > list

This will work until you have too many files in the directory to fit into an argument list.

| improve this answer | |
  • 1
    Very neat! so here we are using the tee program to write to the file rather than leaving that task to the shell to do (which the shell ends up being overeager with for this particular situation) – Steven Lu Jul 15 '16 at 23:53
  • I actually love that it shows the output on stdout... tee is really useful. – Steven Lu Jul 15 '16 at 23:56
  • This fails to solve the problem if there are too many files in the directory. There's an inherent race condition. In an empty (or few files) directory it works, but with a lot of files the tee process can create the file before the ls has had a chance to read the whole directory. – Stephen Harris Jul 16 '16 at 0:01
  • 1
    I did it with 10,000 files and saw the race condition :-) I was in the middle of writing it as an answer until I spotted it. We can kludge with something nasty like (sleep 1 ; tee file), but it's nasty. – Stephen Harris Jul 16 '16 at 0:09
  • Yup, grep is one of the options I listed as well. Mostly I try to avoid this problem by using hidden files or other directories 'cos it's just nasty :-) – Stephen Harris Jul 16 '16 at 0:12
3

You can make the filename temporarily hidden:

ls >.list && mv .list list
| improve this answer | |
  • OK i hadn't thought of this yet. It touches the filesystem a few times but it's somewhat intelligible, which is a big plus... Yes I'm looking for something concise and not too convoluted. Thanks – Steven Lu Jul 15 '16 at 23:52
3

You can use moreutils sponge:

ls | sponge list

Or with zsh:

cp =(ls) list

With GNU ls:

ls -I list > list

(though if there had been a file called list before, that means it won't be listed).

Since ls output is sorted anyway, you can also use (assuming your filenames don't contain newline characters):

ls | sort -o list

Or to avoid the double sorting, if your ls supports -U:

ls -U | sort -o list
| improve this answer | |
2

Partial/most credit goes to @StephenHarris...

echo "`ls`" > list

equivalent to

echo "$(ls)" > list
| improve this answer | |
  • You don't need the -1 option. It's implicit whenever ls isn't writing to a terminal (such as here, where it's writing to a pipe that the shell reads from). – Gilles 'SO- stop being evil' Jul 16 '16 at 20:26
  • @Gilles Right you are. I started trying it with -1 in hopes it would fix it and it stuck around. I've edited the answer. – Steven Lu Jul 16 '16 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.