If you're on GNU/Linux and feel up to something "new"(sic)
pam_cap.so can give particular users
CAP_SYS_TIME which should let regular users change the system time without
su. You basically just configure it as a session module for however they're authenticating (
login or whatever), add the user/capabilities you want to
/etc/security/capabilities.conf then do a
setcap CAP_SYS_TIME+ei /bin/date and you're good to go. pam_cap can't do groups yet, though, so you would have to copy/paste for each user you want to do it.
If you want all system users to be able to change system time, you can forget all the pam_cap stuff, and just do a
setcap CAP_SYS_TIME+ep /bin/date which is essentially Capabilities' analog for adding the setuid bit to a binary (similarly, you can't do it on scripts and if you write to the executable file capability flags and setuid/setgid bits are cleared by the kernel).
date is pretty safe to run through sudo (not many ways you can try to exploit such a simple utility except through out-right substitution, but the attacker would have to find a way to write to directories only root has write permissions to) so that's do-able, but capabilities make privilege granting to end-users seem more seamless (they don't have to think "oh I have to sudo on this" they just go do the thing they're trying to do and it works) and restricted (you're ony given them a specific set of elevated privileges). sudo is a setuid-only mechanism (for the time being at least) and can only think in terms of this or that executable, and whatever that file does (whether binary or script you wrote), it can do it with full root privileges, even privileges that have nothing to do with why you gave the user access to the executable.
setuid is an old old old mechanism that it basically on its way out on most GNU/Linux and Unix implementations. Giving full root access is for the birds, why give CAP_SYS_ADMIN or CAP_NET_ADMIN if you're just trying to let the users change system time? Sure I can't think of a way to exploit
/bin/date to do something evil, but most exploits are non-intuitive until they're discovered and demonstrated (hopefully the software doesn't contain obvious exploits, I mean).
If you're on *BSD or Solaris, they have the same concept they're just called "Privileges" instead of "capabilities" since "capabilities" was already the name of a security-related mechanism on those platforms (roughly analogous to LxC and/or SELinux, fwiw). The implementation details differ between privileges and Linux capabilities but the base concept is the same (it's also how Solaris 11 implements the "root as role" security model).