1

I notice that the number of used inodes on the OCFS2 partition is around 28 million. I moved a folder that contained 518K files and the number became 22 million.

The following script finds the number of files, it was around 65,000 and number of folders was 40.

My question, where are the number 22 million is coming from ? is that normal ?

[root@bopapp1 ~]# cat /etc/issue
Oracle Linux Server release 6.5
Kernel \r on an \m

[root@bopapp1 ~]# uname -a
Linux bopapp1 3.8.13-16.2.1.el6uek.x86_64 #1 SMP Thu Nov 7 17:01:44 PST 2013 x86_64 x86_64 x86_64 GNU/Linux

[root@bopapp1 ~]# df -i
Filesystem                        Inodes    IUsed    IFree IUse% Mounted on
/dev/mapper/vg_testapp2-lv_root  3276800   765283  2511517   24% /
tmpfs                            3089556        3  3089553    1% /dev/shm
/dev/sda1                         128016       55   127961    1% /boot
/dev/mapper/vg_testapp2-lv_home  4882432     9662  4872770    1% /home
/dev/dm-4                       39321087 22543323 16777764   58% /u
/dev/dm-5                       26214055  8319457 17894598   32% /usr/oracle

[root@bopapp1 ~]# cd /u

[root@bopapp1 u]# for i in `find . -type d `; do echo `ls -a $i 2> /dev/null | wc -l` $i; done | sort -n | awk '{sum+=$1} END{print "sum=",sum}'
sum= 62950
  • 1
    You're counting directory entries, not files. For instance, you're counting hardlinks (including the . and .. entries) several times. You can't use use command substitution like that, that's the split+glob operator (so you'll fail to list the directories that have blanks or possibly globs in their path). – Stéphane Chazelas Oct 7 '15 at 11:15
  • ok, I understand that, but is possible that the hard links will use the rest of 22 million inodes ? – Osama Jaber Oct 7 '15 at 11:18
  • counting the hard links several times would increase the count. Here the problem is more likely the one you're hiding with your 2> /dev/null of improperly using the split+glob operator. – Stéphane Chazelas Oct 7 '15 at 11:31
  • There are also filesystems that dynamically adjust the number of inodes. – wurtel Oct 7 '15 at 12:52
2

I just realised we were talking of an OCFS2 cluster file system here and not a regular on disk local file system. This section applies to OCFS2 specifically. See below for normal file systems.

If I create a new blank 1GiB OCFS2 file system and mount it, I see a used inode count of 36378.

After creating 100 empty files, I don't see this number increasing.

After creating 10000 files, I see it going up to 72730.

If I delete all those files, the number doesn't go down, but if I unmount the FS and mount it back, I see it changing to 45634.

So, though I can't tell you why as I'm not familiar with this file system, it seems the IUsed count returned by df does not match the number of i-nodes, or that on that file system, i-nodes are not solely used for files or that the system doesn't track its number of used or free i-nodes other than with very gross granularity.

If you run o2info --freeinode from ocfs2-tools 1.8.2 or above, you can get a number that is closer to expected:

$ du -ax ///mnt/1 | grep -c ///
102
$ df -i /mnt/1
Filesystem     Inodes IUsed  IFree IUse% Mounted on
/dev/loop0     262144 45634 216510   18% /mnt/1
$ o2info --freeinode /mnt/1
Slot            Space           Free
  0             10240          10129
  1                 0              0
Total           10240          10129
$ echo $((10240-10129))
111

$ touch /mnt/1/a{1..100}
$ o2info --freeinode /mnt/1
Slot            Space           Free
  0             10240          10029
  1                 0              0
Total           10240          10029
$ o2info --freeinode /mnt/1 | awk 'END{print $2-$3}'
211

$ touch /mnt/1/b{1..20000}
$ o2info --freeinode /mnt/1
Slot            Space           Free
  0             20480            259
  1                 0              0
Total           20480            259
$ o2info --freeinode /mnt/1 | awk 'END{print $2-$3}'
20221

Now, for conventional file systems...

To get a match for the inode count by crawling directory entries, you'd need to keep a track of which files you've seen based on inode numbers.

find /u -xdev -printf '%i\n' | sort -u | wc -l

du (at least the GNU implementation) will also skip multiple instances of a same file (hard links), so you can use:

 du -xa ///u | grep -c ///

which should give you the same result as the find one.

I'd expect the result to be slightly less than the df -i count as a few file systems (including ext4) have a few special inodes that are not linked to any directory for internal use.

It will also fail to count files that have been deleted but are still open by some process (see lsof +aL1 /u for the list) and those that are masked by another file system mounted within (not the case in your example).

Using command substitution (`...`) like you do is wrong. That's invoking the split+glob operator which by default splits on blanks and performs globbing on the resulting words. So that can only give you a meaningful result if none of the file names contain space, tab, newline, *,?, or [ characters.

ls -a includes the . and .. entries which you don't want to count as they are hard links for some inodes you've already counted (except for the . entry in /u itself). And similarly, you're going to count other hard links several times.

0

You will never be able to match these numbers as there are open but deleted files that are usinf inodes but that are unvisible for users that traverse the filesystem.

There are also hidden inodes (e.g. used by ACL definitions and extended attribute files) that cannot be seen with the find command.

On the other side, you would need to prevent to count hard linked times multiple times. This could be achieved via:

find /u -xdev -ls | sort -n -u +0 | wc -l

but note that this will usually count too few files (see above).

  • 1
    What do you mean by ACL definitions and extended attribute files? If you mean the POSIX-draft-type ACLs or extended attributes of files, then on Linux (ext4 and xfs at least, btrfs being not inode-based), they are not stored on a separate inode (and I can't see why any FS would use inodes to store them). – Stéphane Chazelas Oct 8 '15 at 8:27
  • 1
    Note that the sort -n -u +0 syntax is long obsolete and deprecated and all the sort implementations commonly found on modern Linux-based systems like the OP's will not recognise it (SUSv3 even required that command to sort the file called +0 there, SUSv4 lifted it). The +0 is superfluous anyway since you're using -n (and without it, that would have had to be +0 -1 or its modern equivalent -k 1,1). Note that your find|sort|wc command assumes file names don't contain newline characters. – Stéphane Chazelas Oct 8 '15 at 8:51
  • 1
    it's beside the point. The question is about Oracle Linux. Those NFSv4 ACLs on Linux are only supported via an unofficial patch which as far as I know only the OpenSuse distribution applies. On Linux "native" file systems, extended attributes are not stored in separate inodes, even in XFS which is the only one I know that doesn't have a limit on the number and size of them. – Stéphane Chazelas Oct 8 '15 at 9:04
  • 1
    Your command doesn't work on the OP's system (and most of the systems users of this site use) or on some systems (including UNIX genetic ones like Solaris 10) where there are filenames or symlink targets with newline characters. Mine works on the OP's system regardless of the names of the files. – Stéphane Chazelas Oct 8 '15 at 10:46
  • 1
    Note that GNU sort supports sort +0 on systems where _POSIX2_VERSION is strictly less than 200112. If you have a script that was written for such an old standard and uses +0, you can make it work with GNU sort by setting the environment variable _POSIX2_VERSION to something like 199209. See here for details. – Stéphane Chazelas Oct 8 '15 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.