It would seem to be something that you have to configure through the Web GUI in FreeNAS. From the sound of this blog post titled: "Freenas 9.2.0 with Transmission and Couchpotato/Sickbeard as a DLNA-Server" you need to make changes through several GUIs in order to configure/enable Transmission so that it will work correctly.
That blog post goes on to caution you if you're using Transmission in a jail.
The "watch"-directory where Transmission looks for torrent-files is normally:
That’s 50 characters and if you add the jail-directory to it, it exceeds Freenas’ directory-length limit of 88 characters, so you have to change this. Open up the Transmission-settings in the Plugins-tab of Freenas and edit the “Watch Directory” to “/media/torrentfiles” and the Download-Directory to “/media/downloads”.
I found this thread that sounds like your issue, titled: "plugins wont start, in openpam_check_desc_owner_perms(): /etc/pam.d/su: insecure ownership or permis...".
This thread seemed to conclude with there was a permissions problem with
/etc/pam.d/*. Changing the permissions to 755, applied to
/etc/pam.d/* was reported to fix the issue.