I have a CentOS-6 machine that I'm working on that is running a live database. It's having a problem with the /mnt directory. I cannot ls the directory for some reason. stat is working and shows this output:

  File: `/mnt/'
  Size: 4096        Blocks: 8          IO Block: 4096   directory
Device: ca02h/51714d    Inode: 16321       Links: 9
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2011-09-23 06:50:20.000000000 -0500
Modify: 2012-10-29 11:46:33.000000000 -0500
Change: 2012-10-29 11:46:33.000000000 -0500

I can also cd into the directory and issue both the touch and mkdir commands successfully (or at least without errors. However, if I try to run ls in the directory it just hangs. At that point neither Ctrl+C nor Ctrl+Z have any effect and my system locks up. I have to close my ssh window and reconnect to do anything else.

I'm at a bit of a loss here, does anyone have any ideas what might be causing this? Or, does anyone know how I can try to fix it, without rebooting the machine (I've encountered this once before on the same directory on a different machine and it was corrected by rebooting) since this is a live machine, I can't actually reboot.

These are the filesystems which are mounted in that directory:

blob.XXXXX.com:/blend on /mnt/blend type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)
blob.XXXXX.com:/new_log on /mnt/new_log type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)
blob.XXXXX.com:/new_backup on /mnt/new_backup type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)
blob.XXXXX.com:/vz on /mnt/vz type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)
blob.XXXXX.com:/git on /mnt/git type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)

Interestingly enough, the output of strace ls /mnt shows the mount directories in there.

  • What filesystems do you have mounted? – Random832 Oct 29 '12 at 16:55
  • pastebin.com/xgqYsrLC – CRThaze Oct 29 '12 at 17:19
  • Have you managed to get it to freeze under strace? (with --color=auto or some other option) – Random832 Oct 30 '12 at 17:00

One thing is, ls attempts to stat() every file within the directory (which includes, in this case, the root of every mounted file system), which neither of the commands you attempted successfuly do so. So the problem is likely with one of the mounted filesystems, and not with /mnt itself.

To know more, you need to figure out what filesystems you have mounted in /mnt, and try each of them individually (ls -d /mnt/foo) to see which one causes the problem.

  • Neither of them seem to be having an issue. – CRThaze Oct 29 '12 at 17:23
  • 1
    so if you ls -d everything in /mnt, one at a time, none of them freeze? And if you ls /mnt again it still freezes again? – Random832 Oct 29 '12 at 19:32
  • 1
    Can you pastebin the strace output? Maybe this will show where it freezes. – Random832 Oct 29 '12 at 19:33
  • 3
    @0x783czar When you run strace ls, it bypasses your shell aliases. You probably have an alias that causes ls to actually execute ls --color=auto or some such. The alias causes ls to stat each directory entry; a bare ls /mnt doesn't. – Gilles Oct 29 '12 at 22:41
  • 1
    Yet, I was able to stat every file in the directory manually. (there were only about 7 as can be seen in the strace output, unless I'm missing something) – CRThaze Oct 30 '12 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.