How do I change permissions on a directory without losing setgid?

Question

When I change permissions for other or user on a setgid directory, the directory loses its setgid. How do I make the change without loosing it? sudo is not an option. Is it possible?

Here is some context.

$ whoami
webmin
$ groups
webmin
$ echo $SHELL
/bin/bash
$ uname -srvm
Linux 2.6.38-12-server #51-Ubuntu SMP Wed Sep 28 16:07:08 UTC 2011 x86_64

Here is an example.

$ ls -la
drwxr-s--- 4 webmin www-data    4096 2011-11-03 10:59 .
drwxr-s--- 4 webmin www-data    4096 2011-10-26 15:53 ..
$ mkdir libraries
$ ls -ld libraries
drwxr-sr-x 2 webmin www-data    4096 2011-11-03 11:01 libraries
$ chmod o= libraries
$ ls -ld libraries
drwxr-x--- 2 webmin www-data    4096 2011-11-03 11:01 libraries
      ^
       `- The problem

The same happens if I modify the user's permissions on the directory.

The following fails too.

$ chmod g=rxs,o= libraries

=== UPDATE ====

Kevin's answer led me to what I believe is the cause of the problem. Apparently, you must be a member of the group assigned to the file. For example

$ whoami
webmin
$ groups
webmin www-data      <---- Now we are in the www-data group.
$ mkdir t
$ ls -ld t
drwxr-sr-x 2 webmin www-data 4096 2011-11-03 12:03 t
$ chmod o= t
$ ls -ld t
drwxr-s--- 2 webmin www-data 4096 2011-11-03 12:03 t
      ^
       `- Yeah!

So is the answer is "yes" as long as you are a member of the group, otherwise "no"?

Answer 1

$ ls -ld testdir
drwxrwxrwx ... testdir
$ chmod 2750 testdir
$ ls -ld testdir
drwxr-s--- ... testdir

Is that what you want?

Edit: I checked on my system, and setting the o perms and even changing the group didn't change the setgid bit. Not sure why it did on yours.

Answer 2

If you are not a member of the group assigned to the directory, then if you modify any permissions, you will lose setgid on that directory.

Options

1. You could change your umask before you create the directory, avoiding the need to modify the permissions on the directory after you create it. That way you won't lose setuid on that directory. (Demonstrated at the end of this answer.)

2. Become a member of the group (requires sudo)

   $ sudo usermod -a -G www-data webmin

3. Change the permissions using sudo

   $ chmod g=rxs,o= libraries

The last two violate the constraints of the question, so the answer is option 1.

$ umask 0027
$ mkdir libraries
$ ls -ld libraries
drwxr-s--- 2 webmin www-data 4096 2011-11-03 12:03 libraries