You are partially right. On a directory the
x permission signifies the permission to use that directory to get at objects it references. The
r right is needed to read the directory, that is, list the names of the objects it references (note that if your permission is
r-- you can list the names, but not access the objects; if it is
--x you can access the objects as long as you are able to give the right name, but not list them). The
w permission means to modify the directory, that is, renaming objects referenced, delete references, or add new ones. Again, this is completely independent of the other permissions.
Another aspect is that there are 3 sets of permissions, for the user, the group, and others (sometimes named ugo). The user set of bits is for the owner of the file, the group for users belonging to the file's group. When checking permissions, if the user trying an access is the file's owner, the user permissions apply. If the user isn't the file's owner, but belongs to the file's group, the group permissions apply. Only if none of the previous cases apply are the other permissions checked.
Note that as the permission bits are completely independent, the above means there can be files to which neither the owner nor the group have access, but others have full permissions. A bit weird, but consistent and simple.