0

I am trying to process some files in directory using foreach loop for ls output:

IFS=$'\n'
for i in $(ls -1Atu); do
    echo "$i"
done

At first time i thought it works, but when i created file with name like * or filename*, shell add additional iterations, because * interpreted as wild char.
But if I escape ls output like for i in "$(ls -1Atu)";, I'll get a single iteration, and $i will contain the output from ls entirely.
How can we solve this problem? I should note that any variants with ls -1Atu preferable, as I need, what would the resulting files were sorted by atime. Thank you

2

Background reading: Why does my shell script choke on whitespace or other special characters?, Why you shouldn't parse the output of ls

Setting IFS to a newline means that only newlines, and not spaces and tabs, will be treated as separators during the expansion of the command substitution. Your method will not support file names that contain newlines; this is a fundamental limitation of ls¹.

Furthermore, and this is what you ran against, setting IFS has no effect on the other thing that happens on the expansion of a command substitution, which is globbing (wildcard matching). You can turn off globbing, and then your script will work as long as file names don't contain newlines.

IFS='
'
set -- *"$IFS"*
if [ -e "$1" ]; then
  echo >&2 "There are file names with newlines. I cannot cope with this."
  exit 2
fi
set -f
for i in $(ls -1Atu); do
    printf '%s\n' "$i"
done
set +f
unset IFS

The easy, reliable way to enumerate files by date is to use something other than a pure Bourne-style shell, such as zsh which has glob qualifiers to modify the way wildcard matches are sorted.

#!/bin/zsh
files_by_access_time=(*(Doa))

¹ You can work around it with some ls implementations, but not if you need portability.

  • @muru Fixed. You are encouraged to edit posts to edit obvious mistakes like this. – Gilles Apr 26 '15 at 10:26
2

For various reasons related to whitespace issues, etc., it is not advisable to parse the output of ls. An alternative, which uses GNU versions of find, sort, sed:

find . -mindepth 1 -maxdepth 1 -printf "%A@ %f\0" | 
  sort -rnz | 
  sed -z 's/^[0-9.]\+ //'
  • find is, of course, much more flexible than ls when it comes to listing and filtering files, but it doesn't do sorting by itself.
  • Here, I print the modification time in seconds (%A@), followed by the filename and an ASCII NUL character, the only safe character for delimiting filenames.
  • sort -rnz then takes in the NUL-delimited input (-z) and sorts numerically (-n), in descending order (since -t of ls gives newest first).
  • Then I used sed to strip off the initial time, again dealing in NUL-delimited lines (-z).

The resulting output is a stream of NUL-delimited lines. In bash (not in plain sh), you can then read them into variables by:

find . -mindepth 1 -maxdepth 1 -printf "%A@ %f\0" | 
  sort -rnz | 
  sed -z 's/^[0-9.]\+ //' |
  while IFS= read -r -d '' filename
  do 
    # do stuff with filenames
    printf "%s\n" "$filename"
  done

This is, of course, somewhat more complicated, but considerably safer.

1
  1. Do you really mean adding * in filename?
  2. Or you mean the output of ls gives filename ending in * if it has execute permission?

If only output problem of ls, you could simply solve by:

replace ls to \ls, this is to use un-aliased version of ls, which doesn't output *

1

You can use something like this:

ls -1Atu | while IFS= read -r entry; do
    echo "$entry"
done

With this example, the output is generated once, and the while read entry section causes the output from ls to be parsed line-by-line, which solves the issue with your for example where everything was getting placed in $i in a single round.

  • why so many options with ls? It's sufficient to write just ls. To be safe with other issues you should better write: ls | while IFS= read -r entry ; do printf "%s\n" "$entry" ; done. – Janis Apr 23 '15 at 0:44
  • 3
    The options for ls were specified by the OP because they want to show hidden files/folders and they want a specific sorting rule. The 1 is unnecessary but doesn't hurt. I've added the -r to my answer. Can you explain how the IFS= helps in this case? – jkt123 Apr 23 '15 at 0:50
  • ah, I missed that the OP was the source of those ls options. - Nonetheless, they are all superfluous, aren't they?! - The purpose of IFS is to prevent the trimming of whitespace from the front or rear of the names that are read by read; it's just a standard code pattern if you want your input unchanged. The same with using printf instead of echo; just a safety measure to make the code more robust. – Janis Apr 23 '15 at 2:47
  • 2
    @Janis: "superfluous"?  Huh?  As jkt123 acknowledged, -1 is unnecessary when the output from ls is not going to a terminal, but -A tells ls to include . files, -t tells it to sort by date/time instead of name, and -u tells it to sort by access time instead of mod time.  OK, -tu might be equivalent to -u in the absence of -l (lowercase L), but, if the OP cares about order, these options are not superfluous. – G-Man Apr 23 '15 at 3:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.