What are the POSIX "mandatory utilities"?

Question

The POSIX docs here and here refer to "mandatory utilities", but I can't find any listing of such utilities. Is there one somewhere in the POSIX docs?

Granted, the links given above point to older version of the docs. Maybe the nomenclature has changed since then (E.g., maybe what used to be called "mandatory utilities" are now called "required utilities", or "obligatory utilities", or "core utilities", etc.) or the mandatory/optional distinction has been dropped altogether? Clarifications welcome.

Answer 1

From one of the sections that you cite:

Optional utilities that are present only on systems supporting the associated option; see Codes for information on the options in this volume of IEEE Std 1003.1-2001

The mandatory utilities are the ones that are not marked as optional.

For example basename has no annotation to indicate that it's optional, so it's mandatory. alias is annotated as UP, so it's only mandatory if an implementation claims to include the User Portability Utilities option. command is mandatory, but the -v and -V options are not unless the implementation claims to include the User Portability Utilities option.

Answer 2

It always depends on how "compliant" a system is, but for the POSIX utilities, you can check the lists below. If it is marked as "DEVELOPMENT" then it is considered optional:

"Mandatory" utilities:

alias - define or display aliases
ar - create and maintain library archives
asa - interpret carriage-control characters
at - execute commands at a later time
awk - pattern scanning and processing language
basename - return non-directory portion of a pathname
batch - schedule commands to be executed in a batch queue
bc - arbitrary-precision arithmetic language
bg - run jobs in the background
c99 - compile standard C programs
cal - print a calendar
cat - concatenate and print files
cd - change the working directory
chgrp - change the file group ownership
chmod - change the file modes
chown - change the file ownership
cksum - write file checksums and sizes
cmp - compare two files
comm - select or reject lines common to two files
command - execute a simple command
compress - compress data
cp - copy files
crontab - schedule periodic background work
csplit - split files based on context
cut - cut out selected fields of each line of a file
date - write the date and time
dd - convert and copy a file
df - report free disk space
diff - compare two files
dirname - return the directory portion of a pathname
du - estimate file space usage
echo - write arguments to standard output
ed - edit text
env - set the environment for command invocation
ex - text editor
expand - convert tabs to spaces
expr - evaluate arguments as an expression
false - return false value
fc - process the command history list
fg - run jobs in the foreground
file - determine file type
find - find files
fold - filter for folding lines
fort77 - FORTRAN compiler (FORTRAN)
fuser - list process IDs of all processes that have one or more files open
gencat - generate a formatted message catalog
getconf - get configuration values
getopts - parse utility options
grep - search a file for a pattern
hash - remember or report utility locations
head - copy the first part of files
iconv - codeset conversion
id - return user identity
ipcrm - remove an XSI message queue, semaphore set, or shared memory segment identifier
ipcs - report XSI interprocess communication facilities status
jobs - display status of jobs in the current session
join - relational database operator
kill - terminate or signal processes
link - call link function
ln - link files
locale - get locale-specific information
localedef - define locale environment
logger - log messages
logname - return the user's login name
lp - send files to a printer
ls - list directory contents
m4 - macro processor
mailx - process messages
man - display system documentation
mesg - permit or deny messages
mkdir - make directories
mkfifo - make FIFO special files
more - display files on a page-by-page basis
mv - move files
newgrp - change to a new group
nice - invoke a utility with an altered nice value
nl - line numbering filter
nohup - invoke a utility immune to hangups
od - dump files in various formats
paste - merge corresponding or subsequent lines of files
patch - apply changes to files
pathchk - check pathnames
pax - portable archive interchange
pr - print files
printf - write formatted output
ps - report process status
pwd - return working directory name
qalter - alter batch job
qdel - delete batch jobs
qhold - hold batch jobs
qmove - move batch jobs
qmsg - send message to batch jobs
qrerun - rerun batch jobs
qrls - release batch jobs
qselect - select batch jobs
qsig - signal batch jobs
qstat - show status of batch jobs
qsub - submit a script
read - read from standard input into shell variables
renice - set nice values of running processes
rm - remove directory entries
rmdir - remove directories
sed - stream editor
sh - shell, the standard command language interpreter
sleep - suspend execution for an interval
sort - sort, merge, or sequence check text files
split - split a file into pieces
strings - find printable strings in files
stty - set the options for a terminal
tabs - set terminal tabs
tail - copy the last part of a file
talk - talk to another user
tee - duplicate standard input
test - evaluate expression
time - time a simple command
touch - change file access and modification times
tput - change terminal characteristics
tr - translate characters
true - return true value
tsort - topological sort
tty - return user's terminal name
type - write a description of command type
ulimit - set or report file size limit
umask - get or set the file mode creation mask
unalias - remove alias definitions
uname - return system name
uncompress - expand compressed data
unexpand - convert spaces to tabs
uniq - report or filter out repeated lines in a file
unlink - call the unlink function
uucp - system-to-system copy
uudecode - decode a binary file
uuencode - encode a binary file
uustat - uucp status enquiry and job control
uux - remote command execution
vi - screen-oriented (visual) display editor
wait - await process completion
wc - word, line, and byte or character count
who - display who is on the system
write - write to another user
xargs - construct argument lists and invoke utility
zcat - expand and concatenate data

DEVELOPMENT ("optional") utilities:

admin - create and administer SCCS files (DEVELOPMENT)
cflow - generate a C-language flowgraph (DEVELOPMENT)
ctags - create a tags file (DEVELOPMENT, FORTRAN)
cxref - generate a C-language program cross-reference table (DEVELOPMENT)
delta - make a delta (change) to an SCCS file (DEVELOPMENT)
get - get a version of an SCCS file (DEVELOPMENT)
lex - generate programs for lexical tasks (DEVELOPMENT)
make - maintain, update, and regenerate groups of programs (DEVELOPMENT)
nm - write the name list of an object file (DEVELOPMENT)
prs - print an SCCS file (DEVELOPMENT)
rmdel - remove a delta from an SCCS file (DEVELOPMENT)
sact - print current SCCS file-editing activity (DEVELOPMENT)
sccs - front end for the SCCS subsystem (DEVELOPMENT)
strip - remove unnecessary information from strippable files (DEVELOPMENT)
unget - undo a previous get of an SCCS file (DEVELOPMENT)
val - validate SCCS files (DEVELOPMENT)
what - identify SCCS files (DEVELOPMENT)
yacc - yet another compiler compiler (DEVELOPMENT)

METHOD

# ksh93: Version AJM 93u+ 2012-08-01
## lots of ugly, lazy hacks but 
## it should be readable enough to give you the idea

cd /tmp

[ ! -s /tmp/posix.list ] && curl --location --silent http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ | sed -e 's/<[^>]*>//g' | awk '/^[a-z].*.html/ { print $1 }' | tee /tmp/posix.list

rm -f /tmp/posix.txt /tmp/all_tools.txt

echo '#!/bin/sh' > /tmp/clean_all.sh

cat /tmp/posix.list | while read P
do
    echo
    p=$(basename "${P}" .html)
    echo "Getting ${p} details"
    [ ! -s "${p}.txt" ] && curl --location --silent "http://pubs.opengroup.org/onlinepubs/9699919799/utilities/${P}" | sed -e 's/<[^>]*>//g' > "${p}.txt"
    grep -A3 ^NAME "${p}.txt" | grep ^${p} | tee -a /tmp/posix.txt
    echo "${p}.txt" >> /tmp/all_tools.txt
done

sed 's|^|rm -f /tmp/|' < /tmp/all_tools.txt >> /tmp/clean_all.sh
chmod +x /tmp/clean_all.sh
echo 'rm -f /tmp/posix.list' >> /tmp/clean_all.sh
echo 'rm -f /tmp/posix.txt /tmp/all_tools.txt' >> /tmp/clean_all.sh

grep -v DEVELOPMENT /tmp/posix.txt
grep DEVELOPMENT /tmp/posix.txt

Answer 3

Answers of Kajukenbo and schily are incorrect

I have not enough "reputation" to "comment" on the answers, but I have to say that they are just wrong or, to be fair, misled by a POSIX specifications layout that was made for printers, not for browsers.

Misled by missing introduction / bad specs layout

The OP links correctly to the introduction of chapter 4: "Utilities" of POSIX' "Shell & Utilities" (= "XCU" part of POSIX) where a distinction between "mandatory" and "optional" utilities is given - see Gilles' explanation (as excellent as usual, but outdated for alias as example, see below). Unfortunately in the POSIX 2004 specifications version there are no links to the rest of this chapter, and esp. not to the utilities itself. So eg. Kajukenbo obviously searched elsewhere and found the "utilities" index of the POSIX 2017 specifications - but only the index (see the "idx" in the url) and nothing else from XCU's chapter 4 to explain how to read this list.

The attempt to interpret the pure list of utilities leads to a "mandatory" list that mixes mandatory and optional utilities and creating an own (non-POSIX conform) category for "optional".

Proper specs navigation

Now we have the 2018 edition of POSIX 2017 with some improvements for browsing, so if you consult the XCU 2018 edition, you have "Previous" and "Next" in header and footer to navigate and a (badly named) "Home" that leads to the chapter's TOC. Unfortunately chapter 4 gives no simple list of all utilities, you have to consult the index link found by Kajukenbo. From there or following the order of chapter 4 you get to descriptions for each utility considered by POSIX. In these descriptions the decisive element is not the presence or absence of the word "DEVELOPMENT" that makes admin non-mandatory (optional) and alias mandatory, but the presence or absence of a "margin code", given as a superscript link in square brackets in the upper left corner of the SYNOPSIS paragraph like "[XSI]" for admin. Note that esp. "XSI" is referred to in the POSIX specs under Conformance as "may support ... XSI", not "must".

The upper left corner Codes are introduced in the introduction of XCU's chapter 4 (see above) with a link to a code list that also has a section explaining this "Margin Code Notation". The "shading" of the SYNPOSIS paragraph mentioned there is usually rendered by some steel blue background, but this is by nature just a differentiating feature, not an identifying one, i.e. you may notice it only on switching between descriptions of mandatory and optional utilities, but not when directly accessing an optional one (and a blue link label on a steel blue background is not the best design idea).

POSIX versions

Note that since Gilles' answer alias has changed from optional in POSIX 2004 to mandatory in POSIX 2008. So you have to take also in account the POSIX version against which compliance is claimed or searched. This might be confusing due to "standard" vs. "revision" vs. "edition" vs. "version". Here you best use the IEEE standard number as given in the header of specifications on opengroup.com, eg. the current "IEEE Std 1003.1-2017". The IEEE versioning is well documented in Wikipedia's POSIX entry, while the division in parts like XCU is well covered in Wikipedia's SUS entry which, on the other hand, sticks to POSIX "editions" instead of IEEE numbers.

Painful as it is, you might benefit by delving deeper into some UNIX and / or engineer's way of thinking.

Answer 4

All utilities are mandatory except when they are marked special.

Look e.g. at the documentation for the get command that is maked with the "Development" tag.

Check e.g. Chapter 1.7.1 and look for the tag "optional".

Answer 5

I can't find any listing of such utilities. Is there one somewhere in the POSIX docs?

No. You have to check each utility individually, and, based on this, could create a list on your own. In reality, however, you would always rather need detailed information on one utility than superficial information on many.

Granted, the links given above point to older version of the docs

They do. But you may have also to work with "older" systems that don't know newer ones.

Maybe the nomenclature has changed since then

No. That's an industry standard. Industry standards would not survive if they change too much and / or too often. Don't confuse it with an end-user product of a single manufacturer who might be eager to change much and often so that customers are forced to buy anew.