30

I have recently come across a file whose name begins with the character '♫'. I wanted to copy this file, feed it into ffmpeg, and reference it in various other ways in the terminal. I usually auto-complete weird filenames but this fails as I cannot even type the first letter.

I don't want to switch to the mouse to perform a copy-paste maneuver. I don't want to memorize a bunch of codes for possible scenarios. My ad hoc solution was to switch into vim, paste !ls and copy the character in question, then quit and paste it into the terminal. This worked but is quite horrific.

Is there an easier way to deal with such scenarios?

NOTE: I am using the fish shell if it changes things.

  • 7
    Can you use other parts of the file to form a regex to work with it? *restoffile.avi or something like this? – slm Oct 6 '14 at 12:52
  • 1
    In this case the remaining name was a mixture of Kanji and Katakana (japanese script), so not with ease. – ZirconCode Oct 6 '14 at 13:08
  • 3
    Understood, just thought I'd ask. Does jimmij's answer solve it then? Also would you mind pasting a screenshot of the offending files? It would likely be helpful to others that may read this later on. – slm Oct 6 '14 at 13:10
  • 1
    I'm trying to get it to work right now. I don't know how to post a screeny but running the following commands will give you my mock problem: touch '♫ 漢字カ' touch '♫ 漢字タ' – ZirconCode Oct 6 '14 at 13:16
  • 1
    With zsh you can use options to have tab give you a menu from which you can select the appropriate file. – Kevin Oct 7 '14 at 6:08

10 Answers 10

35

If the first character of file name is printable but neither alphanumeric nor whitespace you can use [[:punct:]] glob operator:

$ ls *.txt
f1.txt  f2.txt  ♫abc.txt
$ ls [[:punct:]]*.txt
♫abc.txt
  • Hmm I did not know about these glob operators, I read up on them and learned a bit (thank you), it solves the problem I had which is a single weird file in my dir.Now I have this problem with a large multitude of files, should I ask a new question or update this one? – ZirconCode Oct 6 '14 at 13:30
  • I have accepted your answer, I will post the second scenario tomorrow when I have time. Thank you for the help. – ZirconCode Oct 6 '14 at 13:34
7

ls has some switches (like --quote-name, --escape, --literal) for dealing with unprintable characters, but in this case it seems the character is "printable" but not "typeable" (at least on my keyboard!), so none of these switches seem to help.

Therefore, as a general "brute force" approach to get rid of files with any characters in their names, you can do this:

$ /bin/ls -1A|cat -n  # list all files (except . and ..), 1 per line, add line numbers
     1  ♫
     2  f1.txt
     3  f2.txt

Find the line containing the offending file. Quite likely it will be the 1st line, but let's say it's the 5th. Print line 5 and hex encode it:

$ /bin/ls -1A|sed -n 5p|xxd -g 1
0000000: e2 99 ab 0a                                      ....

Ignoring the 0a (newline) character, construct an escape string, and use the -e option of echo to translate the escapes:

$ echo -e '\xe2\x99\xab'
♫

Now you can copy/move/delete it like this:

$ cp -vi $(echo -e '\xe2\x99\xab') better_name
‘♫’ -> ‘better_name’

Also, if you're not confined to using shell script, you could do it in Python like this:

$ python
>>> import os
>>> os.listdir('.')
[ ..., '\xe2\x99\xab', ... ]
>>> print '\xe2\x99\xab'
♫
>>> import shutil
>>> shutil.copy('\xe2\x99\xab', 'better_name')

Using this approach, you can process many files, you just have to write the logic for selecting the correct files, and renaming them without clobbering, etc:

for f in os.listdir('.'):
  if not f.isalnum():
    newname = generate_newname(f)
    if not os.path.exists(newname):
      shutil.copy(f, newname)
    else:
      print newname, 'already exists!'
6

The simplest that occurs to me is ls [^a-zA-Z0-9]* and it does the trick for me, but terdon's answer is better in bringing attention to the extglob shell option or even a shell-independent approach.

  • This is a decent-enough stab at it. You could ls [^[:alnum:]]* for the same thing. But it's better to use the character class it is, rather than the class(es) it isn't; hence ls [[:punct:]]* will list this file. – Rich Aug 14 at 15:54
5

A similar approach would be to list all files that don't begin with "normal" characters. In bash you can do this with

$ shopt -s extglob
$ ls !([[:alpha:]]*)

However, that does not seem to be available to fish, so you could use find instead:

$ find . -type f -not -name '[[:alpha:]]*'
4

Rename symlinks

One approach to handle file names with special characters - as first characters or elsewhere in the filename is to rename to simpler names.

This can be used even if you need to keep the original filenames: Rename a copy of the filenames.
That can be done by copying the files, but also by creating symlinks or hardlinks to the files, and rename these. cp creates symlinks instead of copies with the option -s (-l for hardlinks).

Use "detox" to clean names

For renaming to clean file names, detox can be used; It renames files to clean up file names according to various rules as defined in a detoxrc file. By default, the UTF8 characters are just removed; With the option -s utf_8-only they are replaced by _:

$ touch '♫ 漢字カ' ♫foo
$ ls -1
♫foo
♫ 漢字カ
$ detox -s utf_8-only * 
$ ls -1                
_ ___
_foo


"detox" on symlinks

Combined with working on symlinks like described above:

$ mkdir orig
$ cd orig 
$ touch '♫ 漢字カ' ♫foo
$ cd ..
$ mkdir clean
$ cd clean 
$ cp -s ../orig/* .
$ ll               
lrwxrwxrwx 1 14 Oct  8 05:52 ♫foo -> ../orig/♫foo
lrwxrwxrwx 1 21 Oct  8 05:52 ♫\ 漢字カ -> ../orig/♫\ 漢字カ
$ ls -1
♫foo
♫ 漢字カ
$ detox --special -s utf_8-only *
$ ll                                
lrwxrwxrwx 1 21 Oct  8 05:52 _\ ___ -> ../orig/♫\ 漢字カ
lrwxrwxrwx 1 14 Oct  8 05:52 _foo -> ../orig/♫foo
2

I don't use fish, but the documentation says that you can enter a Unicode character by prefixing its hex character code with \u (for 16-bit characters) or \U (for 32-bit characters). I think the code for is 491eb, so you could do:

mv \U000491ebabc.mp3 abc.mp3

to rename ♫abc.mp3.

Note that you need the leading zeroes, otherwise abc at the end will be treated as hex digits, and part of the character code; for a 32-bit character you need to enter 8 digits.

2

I don't know if it was already the case in 2014 when you asked the question, but in current versions of fish (as of 2019), you can press Tab twice, to get a zsh-style selection where you can use the arrow keys to visually select the file you want without having to type any part of the filename.

2

Fish does not support bracket wildcards¹ by design.

function find_special_filename
    find ! -path './.*' -name '[^-.a-zA-Z0-9_]*' $argv
end

The command does not search in hidden directories and displays filenames that do not begin with the characters letters, digits, . _ - (c.f. the documentation of find).

Note: $argv is a special array variable (Fish shell) that contains the function arguments therefore the underlying command may receive any expression (e.g. alias).

find_special_filename -exec mv '{}' misc/ \;

¹ In fact, Fish supports bracket expansion (array variable expansion) but Bash uses another terminology (parameter and filename expansions).

1

Use zsh and type what comes next. ZSH supports fuzzy auto complete and can deal with it. (Its especially nice with the OH-MY-ZSH plugin.)

0

You didn’t say whether you want to keep these problematic filenames.  One solution might be to “fix” the problem once and for all by renaming (some or all of) your files to names that you can type by running this script:

#!/bin/sh
for old in *
do
      printf "%s ...? " "$old"
      if read new  &&  [ "$new" != "" ]
      then
             mv -i "$old" "$new"
      fi
done

This will list your existing filenames, each followed by ...?.  Just type Enter to leave a file as is; or type a new name to rename it.  The -i option will cause it to ask you to confirm overwriting if you specify the name of another existing file.

This script can be modified in several ways:

  • You could modify the wildcard (*) to something more restrictive, e.g., *.avi *.mov, so you don’t have to look at every file.
  • You could change the mv to cp, so you keep a copy of the file with its current name and create a (temporary?) copy with a typeable name.
  • You could create a new filename that is based on the existing filename.  For example,

    if read pfx  &&  [ "$pfx" != "" ]
    then
            mv -i "$old" "$pfx$old"
    fi
    

    which lets you slap a prefix in front of the old name. If you chose a unique prefix, this will let you use auto-complete.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.