The basic reason (of why this is a bad idea) is that no user (root, admin or other) should ever have access to another's user password.
Simply because the password is a means of authentication. If I know some other user's password, I know their credentials (username + password), so I can login as that user, impersonating him (or her or it.)
Any action I do when logged in as that user, the other user will be held responsible for. And that is not how authentication should work.
The actions can be disastrous, like deleting a whole bunch of important files, erasing hard disks, erasing backups, shutting down nuclear power plans, etc.
Or just illegal. Imagine a bank institution where I (the admin) have access to all passwords. Using the cashier's password I can order a move of a million dollars from the president's bank account to the window cleaner's bank account. Then use the cashier's superior password to approve the transaction. Then approve a check from the window cleaner's account to my own off-shore bank account.
Then I go for long vacation in the Bahamas ...
In that view, the hashing of the passwords and the use of separate shadow files can be seen as a means to enforce this rule (no user should be able to impersonate another).
And as @Miral's commented*, there is the exception of
su which while allowing impersonation (and kinds of throws off the argument above), is also keeping a log of its use (so it changes the rules to "only admins can impersonate others but a log is kept").
* The bank example was probably not the best. In any environment where security is crucial, more means of authentication and authorization are usually required than just one password.