What could be a cause for getdents returning different results on 2 systems?

Question

I have a weird situation that on one server I am getting following results:

vagrant@shopping:/vagrant/deployer-example$ uname -a
Linux shopping 4.19.0-0.bpo.9-amd64 #1 SMP Debian 4.19.118-2+deb10u1~bpo9+1 (2020-06-09) x86_64 GNU/Linux


 vagrant@shopping:/vagrant/deployer-example$ bin/php --version
PHP 8.0.3 (cli) (built: Mar  5 2021 08:36:11) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.3, Copyright (c) Zend Technologies
    with Zend OPcache v8.0.3, Copyright (c), by Zend Technologies
vagrant@shopping:/vagrant/deployer-example$ sudo ldd --version
ldd (Debian GLIBC 2.24-11+deb9u4) 2.24
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
vagrant@shopping:/vagrant/deployer-example$ php -r 'var_dump(glob("/vagrant/deployer-example/config/{routes}/*.yaml", GLOB_BRACE));'
array(4) {
  [0]=>
  string(56) "/vagrant/deployer-example/config/routes/annotations.yaml"
  [1]=>
  string(55) "/vagrant/deployer-example/config/routes/easy_admin.yaml"
  [2]=>
  string(52) "/vagrant/deployer-example/config/routes/monitor.yaml"
  [3]=>
  string(59) "/vagrant/deployer-example/config/routes/nelmio_api_doc.yaml"
}
vagrant@shopping:/vagrant/deployer-example$ php -r 'var_dump(glob("/vagrant/deployer-example/config/{routes}/*.yaml/", GLOB_BRACE));'
array(4) {
  [0]=>
  string(56) "/vagrant/deployer-example/config/routes/annotations.yaml"
  [1]=>
  string(55) "/vagrant/deployer-example/config/routes/easy_admin.yaml"
  [2]=>
  string(52) "/vagrant/deployer-example/config/routes/monitor.yaml"
  [3]=>
  string(59) "/vagrant/deployer-example/config/routes/nelmio_api_doc.yaml"
}

But on second system

deployer-example@s2-stg-s01:~/deployer/current$ uname -a
Linux s2-stg-s01 4.19.0-0.bpo.9-amd64 #1 SMP Debian 4.19.118-2+deb10u1~bpo9+1 (2020-06-09) x86_64 GNU/Linux
deployer-example@s2-stg-s01:~/deployer/current$ bin/php --version
PHP 8.0.3 (cli) (built: Mar  5 2021 08:36:11) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.3, Copyright (c) Zend Technologies
    with Zend OPcache v8.0.3, Copyright (c), by Zend Technologies
deployer-example@s2-stg-s01:~/deployer/current$ ldd --version
ldd (Debian GLIBC 2.24-11+deb9u4) 2.24
Copyright © 2016 Free Software Foundation, Inc.
Dies ist freie Software; in den Quellen befinden sich die Lizenzbedingungen.
Es gibt KEINERLEI Garantie; nicht einmal für die TAUGLICHKEIT oder
VERWENDBARKEIT FÜR EINEN ANGEGEBENEN ZWECK.
Implementiert von Roland McGrath und Ulrich Drepper.
deployer-example@s2-stg-s01:~/deployer/current$ php -r 'var_dump(glob("/home/deployer-example/deployer/releases/437/config/{routes}/*.yaml", GLOB_BRACE));'
array(4) {
  [0]=>
  string(75) "/home/deployer-example/deployer/releases/437/config/routes/annotations.yaml"
  [1]=>
  string(74) "/home/deployer-example/deployer/releases/437/config/routes/easy_admin.yaml"
  [2]=>
  string(71) "/home/deployer-example/deployer/releases/437/config/routes/monitor.yaml"
  [3]=>
  string(78) "/home/deployer-example/deployer/releases/437/config/routes/nelmio_api_doc.yaml"
}
deployer-example@s2-stg-s01:~/deployer/current$ php -r 'var_dump(glob("/home/deployer-example/deployer/releases/437/config/{routes}/*.yaml/", GLOB_BRACE));'
array(0) {
}

I was debugging it with strace and found that for first system it reports:

open("/vagrant/deployer-example/config/routes", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, {st_dev=makedev(0, 44), st_ino=86530891, st_mode=S_IFDIR|0755, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=224, st_atime=2021-03-12T15:24:14+0000, st_mtime=2021-03-12T15:19:23+0000, st_ctime=2021-03-12T15:19:23+0000}) = 0
brk(0x562b5bb63000)                     = 0x562b5bb63000
getdents(3, [{d_ino=1031795, d_off=1, d_reclen=24, d_name=".", d_type=DT_UNKNOWN}, {d_ino=1031796, d_off=2, d_reclen=24, d_name="..", d_type=DT_UNKNOWN}, {d_ino=1031797, d_off=3, d_reclen=40, d_name="annotations.yaml", d_type=DT_UNKNOWN}, {d_ino=1031798, d_off=4, d_reclen=24, d_name="dev", d_type=DT_UNKNOWN}, {d_ino=1031799, d_off=5, d_reclen=40, d_name="easy_admin.yaml", d_type=DT_UNKNOWN}, {d_ino=1031800, d_off=6, d_reclen=32, d_name="monitor.yaml", d_type=DT_UNKNOWN}, {d_ino=1031801, d_off=7, d_reclen=40, d_name="nelmio_api_doc.yaml", d_type=DT_UNKNOWN}], 32768) = 224
newfstatat(3, "annotations.yaml", {st_dev=makedev(0, 44), st_ino=87279600, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=135, st_atime=2021-03-12T15:19:24+0000, st_mtime=2021-03-12T15:19:23+0000, st_ctime=2021-03-12T15:19:23+0000}, 0) = 0
newfstatat(3, "easy_admin.yaml", {st_dev=makedev(0, 44), st_ino=87279603, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=127, st_atime=2021-03-05T16:00:41+0000, st_mtime=2021-03-05T16:00:41+0000, st_ctime=2021-03-05T16:00:41+0000}, 0) = 0
newfstatat(3, "monitor.yaml", {st_dev=makedev(0, 44), st_ino=87279604, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=137, st_atime=2021-03-05T16:00:41+0000, st_mtime=2021-03-05T16:00:41+0000, st_ctime=2021-03-05T16:00:41+0000}, 0) = 0
newfstatat(3, "nelmio_api_doc.yaml", {st_dev=makedev(0, 44), st_ino=87279605, st_mode=S_IFREG|0644, st_nlink=1, st_uid=1000, st_gid=1000, st_blksize=4096, st_blocks=8, st_size=367, st_atime=2021-03-05T16:00:41+0000, st_mtime=2021-03-05T16:00:41+0000, st_ctime=2021-03-05T16:00:41+0000}, 0) = 0
getdents(3, [], 32768)                  = 0
close(3)                                = 0

but for second one

open("/home/deployer-example/deployer/releases/437/config/routes", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
fstat(5, {st_dev=makedev(8, 17), st_ino=123601048, st_mode=S_IFDIR|0755, st_nlink=3, st_uid=1084, st_gid=1084, st_blksize=4096, st_blocks=8, st_size=4096, st_atime=2021-03-12T16:18:27+0100.512902323, st_mtime=2021-03-12T16:17:55+0100.984164843, st_ctime=2021-03-12T16:17:55+0100.984164843}) = 0
brk(0x557145c2a000)                     = 0x557145c2a000
getdents(5, [{d_ino=123601055, d_off=2396884471359371616, d_reclen=40, d_name="nelmio_api_doc.yaml", d_type=DT_REG}, {d_ino=123601053, d_off=2713394375622026215, d_reclen=32, d_name="monitor.yaml", d_type=DT_REG}, {d_ino=123601049, d_off=3087430337415459848, d_reclen=40, d_name="easy_admin.yaml", d_type=DT_REG}, {d_ino=123601050, d_off=7440370618885146051, d_reclen=24, d_name="dev", d_type=DT_DIR}, {d_ino=123600993, d_off=8681975314043990400, d_reclen=24, d_name="..", d_type=DT_DIR}, {d_ino=123601479, d_off=9220757504948275036, d_reclen=40, d_name="annotations.yaml", d_type=DT_REG}, {d_ino=123601048, d_off=9223372036854775807, d_reclen=24, d_name=".", d_type=DT_DIR}], 32768) = 224
getdents(5, [], 32768)                  = 0
close(5)                                = 0

Both systems are running under same Debian version, under EXT4 filesystem and are provisioned with same Ansible scripts, so should also have same core packages.

How can I narrow this down to the culprit? What possible reasons there could be why these two servers behave differently?

Both straces show getdents() behaving exactly the same, returning 7 entries with a total size of 224. I guess you're mainly concerned about the PHP glob() function behaving differently. There's a slight difference in the PHP versions used, perhaps some PHP configuration variables might be different as well. — TooTea, Mar 12, 2021 at 19:56
glob("*/") returning things that don't end in / looks like a bug in that glob() to me. — Stéphane Chazelas, Mar 12, 2021 at 20:06
Those DT_UNKNOWN and st_nlink=1 on the directory suggest the first one is not ext4. Can you confirm with df -T /vagrant/deployer-example/config/routes for instance? — Stéphane Chazelas, Mar 12, 2021 at 20:27
I can reproduce a similar bug in that php -r 'var_dump(glob("./*/"));' (but not php -r 'var_dump(glob("*/"));' !?) returns broken symlinks. I can reproduce your problem on a fs formatted as a minix fs as well (where directories don't store file types either and d_type is DT_UNKNOWN) — Stéphane Chazelas, Mar 12, 2021 at 20:44
Yeah, even though both systems use the same libc, the first system triggers a corner case (weird filesystem that does not return entry type in getdents data) in the glibc code. The second system just does what 99% filesystems do today and returns the data immediately, triggering a more efficient code path (and one that is the most tested nowadays). — TooTea, Mar 12, 2021 at 20:48

gadelat · Accepted Answer · 2021-04-19 07:40:18Z

5

As could be seen thanks to verbose mode in strace enabled (-v), these getdents calls were returning d_type=DT_REG on first system, but on second one d_type=DT_UNKNOWN. Reason for this is that while in first case filesystem used is ext4, in second case it is prl_fs. prl_fs filesystem itself apparently doesn't return known d_type.

This triggers an edge case in glob() function in glibc. There was a glibc bug issue reported for this at https://sourceware.org/bugzilla/show_bug.cgi?id=25659

I've also contacted Parallels support where I ask them to stop returning DT_UNKNOWN as a d_type. Since Parallels 16.5.0-49183, they fixed this :)

edited Apr 19, 2021 at 7:40

answered Mar 13, 2021 at 8:55

gadelat

4951 gold badge5 silver badges15 bronze badges

1

Note that the type of a file is something that belongs with the file. Some filesystems (most these days) also duplicate that information in the entries of all directories the file is linked to (for performance improvement purposes) but don't have to. I'd expect that prl_fs is some sort of "network" file system between guest and host. If the host OS doesn't make that information readily available without extra effort, it would be reasonable for the prl_fs implementation not to include it in the directory entries. That would be a perfectly valid thing to do.
– Stéphane Chazelas
Mar 13, 2021 at 9:17

Add a comment |

Stack Exchange Network

current community

your communities

more stack exchange communities

What could be a cause for getdents returning different results on 2 systems?

1 Answer 1

Your Answer

Not the answer you're looking for? Browse other questions tagged system-calls or ask your own question.

Hot Network Questions

What could be a cause for getdents returning different results on 2 systems?

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged system-calls or ask your own question.

Related

Hot Network Questions