Barring some temporary bug report/fix (not found in the port's changelog), FreeBSD does not require xterm to be setuid, and has not used that with xterm since 2011. The port maintainer's comment said
- Don't set suid bit. In our implementation, grantpt() and unlockpt() don't
actually have any use, because PTY's are created on the fly and already have
proper permissions upon creation (see src/lib/libc/stdlib/ptsname.c) 
Which refers to changes made in 2008:
The pts(4) driver also implements this feature, which means
posix_openpt() will now return PTY's that are created on the fly.
Given this question in 2014 sounds as if OP was looking at an old system. Or (this does happen), OP's system may have been modified by someone, adding the setuid bit.
Depending on the platform (and its age), xterm would use setuid or setgid for these operations:
- opening the pseudo terminal
- updating utmp (or utmpx)
Originally, BSD (and other) pseudo-terminal implementations required the application to find the special device and then open it using root setuid rights. Unix98 did away with the first part (finding the special device), starting in 1999. Most pseudo-terminal implementations by around 2005 had eliminated the need for the application to be installed using setuid. It was overlooked in FreeBSD for whatever reason.
Likewise, applications may need group permissions (via setgid) to update the utmp (or utmpx) file which is used by
xterm has provided for using the utempter library for this purpose since 2000. The FreeBSD port for xterm uses this feature, so it does not require setgid either.
Other programs may be installed as setuid or setgid for the same operations, but by 2014 it was unlikely that any terminal program needed the setuid permissions for the root user. When used in other programs (such as urxvt) it likely has been overlooked by their maintainers.