LC_COLLATE=C is not always sufficient to sort uppercase before lowercase. You may need to set
That will also take into account non-alphanumeric and even non-printable characters, but if you don't want that there are options
-i (described in
man sort) to will turn that off.
It will probably fail badly with multibyte input though, such as UTF-8 with non-ASCII characters.
To get lowercase (in order) before uppercase (in order), the best way I can think of that doesn't involve breaking out a full-fledged programming language is inverting the case of all the letters before the sort, and inverting them back afterwards.
tr 'a-zA-Z' 'A-Za-z' < file | LC_ALL=C sort | tr 'a-zA-Z' 'A-Za-z'