What I would like to achieve is an interactive program that runs either before or after asking the user for the password, but won't handle over the access to the computer unless it exited with success. To make it somewhat more understandable, here's an example:
I would like to gain access to my computer, by first writing my username, then my password, and after that answering a simple randomly generated mathematical question correctly.
For this to work, I use the following system-auth file:
auth required pam_unix.so try_first_pass nullok nodelay
auth optional pam_faildelay.so delay=600000
auth optional pam_exec.so stdout /home/math
auth optional pam_permit.so
auth required pam_env.so
The problem is, that the program named math can't handle inputs from the user, as it automatically reads an EOF from PAM, which essentially renders it useless. I have also tried the following variant of the questionable line in which case it reads in the password, which is also not what I want:
auth optional pam_exec.so stdout expose_authtok /home/math