You can turn off access time recording for some file systems (look for
/etc/fstab). To test this, you could try to
touch -a file; ls -l --time=atime file, then repeat
touch -a file; ls -l --time=atime file to see if it changes.
ls behavior is explained by
ls -l --time=atime shows access time in the date/time column. To sort by this key you need to use
--sort=time as well.
If you by "the time is not changed" mean that it shows the same result, that is either because atime is disabled for the file system (see
/etc/fstab) or because the access time is really the same as the modification time (quite common).
Note that listing files does not change their access time - The files are not accessed by
ls, only the inode cache.