Why do some umask values not take effect?

I'm trying to understand permissions better, so I'm doing some "exercises". Here's a sequence of commands that I'm using with their respective output:

0022
\$ touch file1
\$ ls -l file1
-rw-r--r-- 1 user group 0 Mar 16 12:55 file1
\$ mkdir dir1
\$ ls -ld dir1
drwxr-xr-x 2 user group 4096 Mar 16 12:55 dir1

That makes sense because we know that the default file permissions are 666 (rw-rw-rw-) and directories default permissions are 777 (rwxrwxrwx). If I subtract the umask value from these default permissions I have 666-022=644, rw-r--r--, for the file1, so it's coherent with the previous output; 777-022=755, rwx-r-x-r-x, for the dir1, also coherent.

But if I change the umask from 022 to 021 it isn't any more.

Here is the example for the file:

\$ touch file2
\$ ls -l file2
-rw-r--rw- user group 0 Mar 16 13:33 file2

-rw-r--rw- is 646 but it should be 666-021=645. So it doesn't work according to the previous computation.

Here is the example for the directory:

\$ touch dir2
\$ ls -ld dir2
drwxr-xrw- 2 user group 4096 Mar 16 13:35 dir2

drwxr-xrw- is 756, 777-021=756. So in this case the result is coherent with the previous computation.

Can somebody explain why?

EXPLANATION

As pointed out in the answers: umask's value is not mathematically subtracted from default directory and file's permissions.

The operation effectively involved is a combination of AND (&) and NOT (!) boolean operators. Given:

R = resulting permissions
D = default permissions
U = current umask

R = D & !U

For example:

666& !0053 = 110 110 110 &
!000 101 011
110 110 110 &
111 010 100
= 110 010 100 = 624 = rw--w-r--

777& !0022 = 111 111 111 &
!000 010 010
111 111 111 &
111 101 101
= 111 101 101 = 755 = rwxr--xr-x

TIP

An easy way to quickly know the resulting permissions (at least it helped me) is to think that we can use just 3 decimal values:

r = 100 = 4
w = 010 = 2
x = 001 = 1

Permissions will be a combination of these 3 values.
" " is used to indicate that the relative permission is not given.

666 = 4+2+" "   4+2+" "   4+2+" "  = rw rw rw

So if my current umask is 0053 I know I'm removing read and execution (4+1) permission from group and write and execution (2+1) from other resulting in

4+2     " "+2+" "     4+" "+" "   = 624 = rw--w-r--

(group and other already hadn't execution permission)

umask is a mask, it’s not a subtracted value. Thus:

• mode 666, mask 022: the result is 666 & ~022, i.e. 666 & 755, which is 644;
• mode 666, mask 021: the result is 666 & ~021, i.e. 666 & 756, which is 646.

Think of the bits involved. 6 in a mode means bits 1 and 2 are set, read and write. 2 in a mask masks bit 1, the write bit. 1 in a mask masks bit 0, the execute bit.

Another way to represent this is to look at the permissions in text form. 666 is rw-rw-rw-; 022 is ----w--w-; 021 is ----w---x. The mask drops its set bits from the mode, so rw-rw-rw- masked by ----w--w- becomes rw-r--r--, masked by ----w---x becomes rw-r--rw-.

You need to think in binary, not decimal. Specifically, there are three 3-bit binary numbers: one each for Owner, Group, and Other. Each with values ranging from 000 to 111 (0-7 in decimal).

e.g. rw-rw-rw (666) is 110 110 110.

The umask value is a mask specifying which bits will be on or off (1 or 0) when creating a new file or directory. e.g. 022 decimal is 000 010 010 binary, while 021 decimal is 000 010 001

The permission bits are AND-ed together with the negated umask to arrive at the final value. "negated" means that all bits are inverted, i.e. all 1s flipped to 0, and vice-versa. e.g. NOT 022 (000 010 010) = 755 (111 101 101)

Example: 666 & !022 = 644. In binary, that's:

Owner  Group  Other     mode
110    110    110      666
&  111    101    101      755   (this is the negated 022)
---    ---    ---      ---
110    100    100      644

Also, 777 & !022 = 755:

Owner  Group  Other     mode
111    111    111      777
&  111    101    101      755
---    ---    ---      ---
111    101    101      755

Note how the final value of each bit can only be 1 if it is 1 in both the original permission value (666 or 777) AND in the negated umask. If either of them is 0, the result is 0. That is, 1 & 1 = 1, while 1 & 0 = 0.

Strictly speaking there's a fourth 3-bit binary number for the setuid, setgid, and sticky bits. That's why you often see permissions and masks specified with a leading 0 (or some other leading number from 0-7). e.g. 0777 or 2755.

• Well technically it's octal, not decimal, but that doesn't really change the core of the answer. – David Z Mar 16 '18 at 22:30
• @DavidZ: OP thought it was decimal (see the subtraction example in the Q), which is presumably what cas was referring to. – Lightness Races in Orbit Mar 16 '18 at 22:53
• @Lightness the subtraction examples given by the OP work whether they’re octal or decimal, given that no digit reaches 8 and there’s never a need for a carry. I agree it’s possible the OP was thinking in decimal, but nothing in the question proves that. – Stephen Kitt Mar 17 '18 at 13:31
• @StephenKitt: Looks like my finger must have slipped on the keypad when I was writing examples into calc.exe to prove it 😂 – Lightness Races in Orbit Mar 17 '18 at 14:00
• @cas yes, I wasn’t disagreeing with that part (it’s the basis for my answer too), just with Lightness’ assertion that the OP was calculating in decimal rather than octal. – Stephen Kitt Mar 17 '18 at 14:19