ptys not working in chroot under lxc

Question

I am trying to use chroots under lxc for development. I have enabled the "nesting" option in the lxc container configuration and bound mounted proc and devpts into my chroot as I would if the chroots were on a normal Linux box.

Unfortunately when I try and use stuff in the chroot that needs ptys (for example the "script" command) I get errors like

root@manualdev:~# chroot /chroots/jessie-staging/
root@manualdev:/# script
script: openpty failed: No such file or directory
Terminated
root@manualdev:/#

System information:

• Host kernel is 4.4.0-79-generic
• Host distro is Ubuntu xenial
• Host architecture is arm64
• Container distro is Debian stretch
• Container and chroot architecture is armhf
• Chroot distro is Raspbian (tested with jessie, stretch and buster)

Answer 1

The fix for this (found by educated guesswork) was to execute the following commands in the chroot.

rm /dev/ptmx
ln -s /dev/pts/ptmx /dev/ptmx

I'm not 100% sure but I believe the reason this is needed is that lxc is using "multiple instance mode" for /dev/pts . As per the documentation at https://github.com/torvalds/linux/blob/v4.4/Documentation/filesystems/devpts.txt

If CONFIG_DEVPTS_MULTIPLE_INSTANCES=y and 'newinstance' option is specified, the mount is considered to be in the multi-instance mode and a new instance of the devpts fs is created. Any ptys created in this instance are independent of ptys in other instances of devpts. Like in the single-instance mode, the /dev/pts/ptmx node is present. To effectively use the multi-instance mode, open of /dev/ptmx must be a redirected to '/dev/pts/ptmx' using a symlink or bind-mount.

Looking at more recent versions of that file it seems that this may not be needed with more recent kernels.

Answer 2

The devpts filesystem which manages pseudo-terminals needs to be mounted:

magic command is:

(as root) # mount -t devpts devpts /dev/pts

Here's a session which checks that the Pseudo Terminal System works:

root@e7440:/me/media/20230104a# mount |grep pts # Check, yup no devpts mounted
root@e7440:/me/media/20230104a# ls -lai /dev/pt*   # /dev/ptmx exists, but /dev/pts is empty
233300 crw-rw-rw- 1 root root 5, 2 Jan  4 23:07 /dev/ptmx

/dev/pts:
total 8
237675 drwxr-xr-x 2 root root 4096 Jan  4 23:04 .
229757 drwxr-xr-x 4 root root 4096 Jan  3 14:31 ..
root@e7440:/me/media/20230104a# mount -t devpts devpts /dev/pts
root@e7440:/me/media/20230104a# mount |grep pts
devpts on /dev/pts type devpts (rw)
root@e7440:/me/media/20230104a# # Let's test the pty system using the script command.
root@e7440:/me/media/20230104a# # the script command will log the shell to its_alive.txt
root@e7440:/me/media/20230104a# script its_alive.txt
Script started, file is its_alive.txt
root@e7440:/me/media/20230104a# ls -lai /dev/pt*  # Now after the mount, things have changed!
233300 crw-rw-rw- 1 root root 5, 2 Jan  4 23:08 /dev/ptmx

/dev/pts:
total 4
     1 drwxr-xr-x 2 root root      0 Jan  4 23:08 .
229757 drwxr-xr-x 4 root root   4096 Jan  3 14:31 ..
     3 crw------- 1 root root 136, 0 Jan  4 23:08 0
     2 c--------- 1 root root   5, 2 Jan  4 23:08 ptmx
root@e7440:/me/media/20230104a# # Note the 0 entry! That's the pty we are using to log this with the script command!
root@e7440:/me/media/20230104a# # Also note it is using inode 3 on the devpts  file system instance
root@e7440:/me/media/20230104a# # And that it is a 'c' a character device node.
root@e7440:/me/media/20230104a# mount |grep pts
devpts on /dev/pts type devpts (rw)
root@e7440:/me/media/20230104a# umount /dev/pts  # Let's go back to no ptys, but wait ...
umount: /dev/pts: target is busy
        (In some cases useful info about processes that
         use the device is found by lsof(8) or fuser(1).)
root@e7440:/me/media/20230104a# # Naughty!  We're using a pty, so we can't get rid of ptys
root@e7440:/me/media/20230104a# exit   # We are leaving the script session so its_alive.txt will get closed.
exit
Script done, file is its_alive.txt
root@e7440:/me/media/20230104a# #### Now we can get rid of ptys entirely as no ptys are in use
root@e7440:/me/media/20230104a# ls -lai /dev/pt*
233300 crw-rw-rw- 1 root root 5, 2 Jan  4 23:09 /dev/ptmx

/dev/pts:
total 4
     1 drwxr-xr-x 2 root root    0 Jan  4 23:08 .
229757 drwxr-xr-x 4 root root 4096 Jan  3 14:31 ..
     2 c--------- 1 root root 5, 2 Jan  4 23:08 ptmx
root@e7440:/me/media/20230104a# ### See no 0 entry (but the /dev/pts/ptmx is there because devpts is mounted)
root@e7440:/me/media/20230104a# mount |grep pts
devpts on /dev/pts type devpts (rw)
root@e7440:/me/media/20230104a# umount /dev/pts
root@e7440:/me/media/20230104a# mount |grep pts
root@e7440:/me/media/20230104a# ls -lai /dev/pt*
233300 crw-rw-rw- 1 root root 5, 2 Jan  4 23:09 /dev/ptmx

/dev/pts:
total 8
237675 drwxr-xr-x 2 root root 4096 Jan  4 23:04 .
229757 drwxr-xr-x 4 root root 4096 Jan  3 14:31 ..
root@e7440:/me/media/20230104a# # Let's see that our session log exists
root@e7440:/me/media/20230104a# ls -l its*
-rwxr-xr-x 1 root root 1025 Jan  4 23:09 its_alive.txt
root@e7440:/me/media/20230104a# # Let's see the session log contents
root@e7440:/me/media/20230104a# grep -n "^"  its_alive.txt  #Line numbers, please!
1:Script started on Wed Jan  4 23:08:47 2023
2:root@e7440:/me/media/20230104a# ls -lai /dev/pt*
3:233300 crw-rw-rw- 1 root root 5, 2 Jan  4 23:08 /dev/ptmx
4:
5:/dev/pts:
6:total 4
7:     1 drwxr-xr-x 2 root root      0 Jan  4 23:08 .
8:229757 drwxr-xr-x 4 root root   4096 Jan  3 14:31 ..
9:     3 crw------- 1 root root 136, 0 Jan  4 23:08 0
10:     2 c--------- 1 root root   5, 2 Jan  4 23:08 ptmx
11:root@e7440:/me/media/20230104a# # Note the 0 entry! That's the pty we are using to log this with the script command!
12:root@e7440:/me/media/20230104a# mount |grep pts
13:devpts on /dev/pts type devpts (rw)
14:root@e7440:/me/media/20230104a# umount /dev/pts
15:umount: /dev/pts: target is busy
16:        (In some cases useful info about processes that
17:         use the device is found by lsof(8) or fuser(1).)
18:root@e7440:/me/media/20230104a# # Naughty!  We're using a pty
19:root@e7440:/me/media/20230104a# exit
20:exit
21:
22:Script done on Wed Jan  4 23:09:46 2023
root@e7440:/me/media/20230104a#  # Yup, life is good!

Sources:

• https://www.cyberciti.biz/faq/linux-mount-devpts/#:~:text=mount%20-t%20devpts%20devpts%20/dev/pts
• https://en.wikipedia.org/wiki/Devpts
• https://web.archive.org/web/20230105042607/http://faculty.cooper.edu/lent/random/GNU_screen_command.html
• Thesis: Virtual Terminal Service, 1987, Christopher Lent, The Cooper Union for the Advancement of Science and Art: Albert Nerken School of Engineering