5

I'm doing a file transfer using sftp. Using the get -r folder command, I'm surprised about the order that the program is downloading the content.

It looks like it would be selecting the files it needs to download randomly. I can't believe that this is actually the case and I'm asking myself what's going on behind the scenes?

What's the order that sftp follows when downloading a folder with its content?

From what I can see so far, it is not by name nor by size.

  • 3
    like would do a find command on a directory or ls --sort=none: directory contents are not sorted – A.B Mar 17 at 10:27
  • yes you are right, just checked it doing the ls --sort=none comand. – AlexOnLinux Mar 17 at 10:30
9

When you list the directory contents with the ls command, it will sort the listing into alphanumeric order according to current locale's sorting rules by default. It is easy to assume that this is the "natural order" of things within the filesystem - but this isn't true.

Most filesystems don't sort their directories in any way: when adding a new file to a directory, the new file basically gets the first free slot in the directory's metadata structure. Sorting is only done when displaying the directory listing to the user. If a single directory has hundreds of thousands or millions of files in it, this sorting can actually require non-trivial amounts of memory and processing power.

When the order in which the files are processed does not matter, the most efficient way is to just read the directory metadata in order and process the files in the order encountered without any explicit sorting. In most cases this would mean the files will be processed basically in the order they were added to the directory, interspersed with newer files in cases where an old file was deleted and a later-added file reclaimed its metadata slot.

Some filesystems might use tree structures or something else in their internal design that might enforce a particular order for their directory entries as a side effect. But such an ordering might be based on inode numbers of the files or some other filesystem-internal detail, and so would not be guaranteed to be useful for humans for any practical purpose.

As @A.B said in the question comments, a find command or a ls -f or ls --sort=none would list the files without any explicit sorting, in whatever order the filesystem stores its directories.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.