2

I have almost 7,000 files in a single directory which makes loading that particular directory a tedious and time-consuming one. Is there any way to list and perform operations on files in that directory faster rather than distributing the files into multiple sub-folders(which makes it even more difficult)?

I'm using ext4. But also received the same from efs filesystems.

4
  • 1
    It's better to you make filter, you have some way, find, globbing and so on , But at first you explain exactly your task. Commented Mar 7, 2015 at 20:22
  • 2
    Could you clarify what you mean by "loading"? Simply ls, or displaying the contents in a file manager (which one?), or something else? On my system listing 7,000 files with ls on an ext4 filesystem is pretty much instantaneous, as is displaying the directory in Thunar. Commented Mar 7, 2015 at 20:56
  • @SnazzySanoj please answer comments, by amending your question. Commented Mar 7, 2015 at 22:30
  • 3
    A lot of systems alias ls to something that makes it lstat every file. --color and -F do this. If you don't need ls to decorate the output, remove these aliases. Listing a 7000 file directory shouldn't be slow. Commented Mar 7, 2015 at 23:48

1 Answer 1

1

A large directory can be problematic for a number of reasons other than simply listing of files. For one thing, the time it takes to open a file in that directory will increase because the directory has to be read until the file is found. On many filesystems including ext*, directory entries not organized nor optimized for retrieval efficiency.

Answering your specific question, I think that you'll find using ls takes awhile due to the sorting involved. A solution is to run ls unsorted (if there is such an option in your distribution of ls). Specifically, I might ls unsorted to a file and the sort it. I then have the file to refer to without having to do another ls for a bit.

Another similar method is to use the find command and listout the directory contents into a file (which will be unsorted) and then go from there.

Thus my suggestion would be based on accessibility, effeciency and easily finding files, to use multiple sub-directories.

3
  • 1
    "A directory is not organized to speed up retrievals of specific entries": that completely depends on the filesystem in use, and many commonly-used filesystem types do optimize directory indices for just this ability.
    – Celada
    Commented Mar 8, 2015 at 5:09
  • OK, I qualified my response. Could you provide the commonly used filesystems that do what you state for the record?
    – mdpc
    Commented Mar 8, 2015 at 5:42
  • 2
    XFS automatically, ext4 (probably ext3 as well?) with the "dir_index" option enabled (which is part of the default options) (see mkfs.ext4 manpage), and I didn't bother to look up references but probably most of the other modern filesystem types as well: Btrfs, ZFS, JFS, all support efficient indexing for large directories.
    – Celada
    Commented Mar 8, 2015 at 7:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .