ls has some switches (like --quote-name, --escape, --literal) for dealing with unprintable characters, but in this case it seems the character is "printable" but not "typeable" (at least on my keyboard!), so none of these switches seem to help.
Therefore, as a general "brute force" approach to get rid of files with any characters in their names, you can do this:
$ /bin/ls -1A|cat -n # list all files (except . and ..), 1 per line, add line numbers
Find the line containing the offending file. Quite likely it will be the 1st line, but let's say it's the 5th. Print line 5 and hex encode it:
$ /bin/ls -1A|sed -n 5p|xxd -g 1
0000000: e2 99 ab 0a ....
Ignoring the 0a (newline) character, construct an escape string, and use the -e option of echo to translate the escapes:
$ echo -e '\xe2\x99\xab'
Now you can copy/move/delete it like this:
$ cp -vi $(echo -e '\xe2\x99\xab') better_name
‘♫’ -> ‘better_name’
Also, if you're not confined to using shell script, you could do it in Python like this:
>>> import os
[ ..., '\xe2\x99\xab', ... ]
>>> print '\xe2\x99\xab'
>>> import shutil
>>> shutil.copy('\xe2\x99\xab', 'better_name')
Using this approach, you can process many files, you just have to write the logic for selecting the correct files, and renaming them without clobbering, etc:
for f in os.listdir('.'):
if not f.isalnum():
newname = generate_newname(f)
if not os.path.exists(newname):
print newname, 'already exists!'