I have a bunch of directories and subdirectories that contain files with special characters, like this file:
robbie@phil:~$ ls test�sktest.txt test?sktest.txt
Find reveals an escape sequence:
robbie@phil:~$ find test�sktest.txt -ls 424512 4000 -rwxr--r-x 1 robbie robbie 4091743 Jan 26 00:34 test\323sktest.txt
The only reason I can even type their names on the console is because of tab completion. This also means I can rename them manually (and strip the special character).
I've set LC_ALL to UTF-8, which does not seem to help (also not on a new shell):
robbie@phil:~$ echo $LC_ALL en_US.UTF-8
I'm connecting to the machine using ssh from my mac. It's an Ubuntu install:
robbie@phil:~$ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=7.10 DISTRIB_CODENAME=gutsy DISTRIB_DESCRIPTION="Ubuntu 7.10"
Shell is Bash, TERM is set to xterm-color.
These files have been there for quite a while, and they have not been created using that install of Ubuntu. So I don't know what the system encoding settings used to be.
I've tried things along the lines of:
find . -type f -ls | sed 's/[^a-zA-Z0-9]//g'
But I can't find a solution that does everything I want:
- Identify all files that have undisplayable characters (the above ignores way too much)
- For all those files in a directory tree (recursively), execute mv oldname newname
- Optionally, the ability to transliterate special characters such as ä to a (not required, but would be awesome)
- Correctly display all these files (and no errors in applications when trying to open them)
I have bits and pieces, like iterating over all files and moving them, but identifying the files and formatting them correctly for the mv command seems to be the hard part.
Any extra information as to why they do not display correctly, or how to "guess" the correct encoding are also welcome. (I've tried convmv but it doesn't seem to do exactly what I want: http://j3e.de/linux/convmv/)