I am trying to change all characters that are not letters, numbers, dashes, dots, or underscores to an underscore in file names in a directory. So far I have the following:

ls *.mp3 | sed 's/[^0-9a-zA-Z._-]/_/g'

This works except that it just shows me the results, it does not actually change the names. What else do I need to do to get it to change the names?

  • You're saying letter, but your a-z suggests you only want letters in the latin script. a-z would typically include é but not ẑ for instance which comes after z. I take it you don't want Greek/Cyrillic/Arabic/Korean... letters? What about latin letters with diacritics or followed by combining characters? What about other variants of the latin letters that don't fall in the ASCII range like 🅰, ᴀ, Ɐ, A, tag letters... – Stéphane Chazelas Nov 20 '20 at 8:09
  • Kojow7, can you please be more specific about what special characters means here? As Stéphane Chazelas has pointed out, working with multi-byte characters is tricky, but if you meant only &*$... by special characters, then the solution is not arduous. – Quasímodo Nov 20 '20 at 17:32
  • @Quasímodo basically anything that is not a digit, a letter of the 26-letter English alphabet, dot, underscore, or dash. Most of the situations I am coming across include: spaces, quotes, apostrophes, and question marks. – kojow7 Nov 20 '20 at 17:55

Don't parse Ls. That seems to work well, but for complex cases it may fail.

With Bash's parameter expansion:

for f in *.mp3; do mv -- "$f" "${f//[!0-9a-zA-Z.-]/_}"; done

With Rename:

rename -- 's/[^0-9a-zA-Z.-]/_/g' *.mp3

If you only have standard POSIX tools and assuming no newline characters in the file names,

for f in *.mp3; do 
    mv -- "$f" "$(printf '%s\n' "$f" | sed 's/[^0-9a-zA-Z.-]/_/g')"
  • Except with the rename-based one, you'll need export LC_ALL=C or otherwise those ranges could match thousands of characters beside the English letter/digits and could also fail on filenames that don't form valid text in the locale. That would however mean that if there are UTF-8 encoded non-ASCII characters in those file names, each byte of those characters would be converted to _. – Stéphane Chazelas Nov 20 '20 at 7:56
  • Note that the behaviour of POSIX sed is unspecified on non-text input, like that output of printf not ending in a newline character. Also beware that that sed command won't replace newline characters in those filenames. – Stéphane Chazelas Nov 20 '20 at 8:12
  • @StéphaneChazelas I was thinking only of ASCII special characters (&*$...). I guess replacing 0-9a-zA-Z by [:alnum:] wouldn't solve the problem for ranges outside English? I hope the asker will answer to our clarification requests, meanwhile I will fix that thing with Sed. Thank you, much appreciated! – Quasímodo Nov 20 '20 at 17:29

With zsh:

autoload -Uz zmv # best in ~/.zshrc
zmv '(*).mp3' '${f//[^0-9a-zA-Z_.-]/_}'

In zsh, ranges are based on codepoint value, so 0-9 includes 0123456789 only (and not 0123456789٠١٢٣٤٥٦٧٨۰۱۲۳۴۵۶۷۸߀߁߂߃߄߅߆߇߈०१२३४५६७८০১২৩৪৫৬৭৮੦੧੨੩੪੫੬੭੮૦૧૨૩૪૫૬૭૮୦୧୨୩୪୫୬୭୮௦௧௨௩௪௫௬௭௮౦౧౨౩౪౫౬౭౮౸౹౺౻౼౽౾೦೧೨೩೪೫೬೭೮൦൧൨൩൪൫൬൭൮෦෧෨෩෪෫෬෭෮๐๑๒๓๔๕๖๗๘໐໑໒໓໔໕໖໗໘༠༡༢༣༤༥༦༧༨༪༫༬༭༮༯༰༱༳၀၁၂၃၄၅၆၇၈႐႑႒႓႔႕႖႗႘፩፪፫፬፭፮፯፰០១២៣៤៥៦៧៨៰៱៲៳៴៵៶៷៸᠐᠑᠒᠓᠔᠕᠖᠗᠘᥆᥇᥈᥉᥊᥋᥌᥍᥎᧐᧑᧒᧓᧔᧕᧖᧗᧘᧚᪀᪁᪂᪃᪄᪅᪆᪇᪈᪐᪑᪒᪓᪔᪕᪖᪗᪘᭐᭑᭒᭓᭔᭕᭖᭗᭘᮰᮱᮲᮳᮴᮵᮶᮷᮸᱀᱁᱂᱃᱄᱅᱆᱇᱈᱐᱑᱒᱓᱔᱕᱖᱗᱘⁰⁴⁵⁶⁷⁸₀₁₂₃₄₅₆₇₈⅐⅑⅒⅓⅔⅕⅖⅗⅘⅙⅚⅛⅜⅝⅞⅟ↅ↉①②③④⑤⑥⑦⑧⑩⑪⑫⑬⑭⑮⑯⑰⑱⑲⑳⑴⑵⑶⑷⑸⑹⑺⑻⑽⑾⑿⒀⒁⒂⒃⒄⒅⒆⒇⒈⒉⒊⒋⒌⒍⒎⒏⒑⒒⒓⒔⒕⒖⒗⒘⒙⒚⒛⓪⓫⓬⓭⓮⓯⓰⓱⓲⓳⓴⓵⓶⓷⓸⓹⓺⓻⓼⓾⓿❶❷❸❹❺❻❼❽❿➀➁➂➃➄➅➆➇➉➊➋➌➍➎➏➐➑➓〇〡〢〣〤〥〦〧〨㉈㉉㉊㉋㉌㉍㉎㉏㉑㉒㉓㉔㉕㉖㉗㉘㉙㉚㉛㉜㉝㉞㉟㊱㊲㊳㊴㊵㊶㊷㊸㊹㊺㊻㊼㊽㊾㊿㋀㋁㋂㋃㋄㋅㋆㋇㋉㋊㋋㍘㍙㍚㍛㍜㍝㍞㍟㍠㍢㍣㍤㍥㍦㍧㍨㍩㍪㍫㍬㍭㍮㍯㍰㏠㏡㏢㏣㏤㏥㏦㏧㏩㏪㏫㏬㏭㏮㏯㏰㏱㏲㏳㏴㏵㏶㏷㏸㏹㏺㏻㏼㏽㏾꘠꘡꘢꘣꘤꘥꘦꘧꘨꣐꣑꣒꣓꣔꣕꣖꣗꣘꤀꤁꤂꤃꤄꤅꤆꤇꤈꧐꧑꧒꧓꧔꧕꧖꧗꧘꧰꧱꧲꧳꧴꧵꧶꧷꧸꩐꩑꩒꩓꩔꩕꩖꩗꩘꯰꯱꯲꯳꯴꯵꯶꯷꯸012345678𐄇𐄈𐄉𐄊𐄋𐄌𐄍𐄎𐅂𐅃𐅈𐅏𐅘𐅙𐅚𐅛𐅜𐅝𐅞𐅟𐅳𐆊𐋡𐋢𐋣𐋤𐋥𐋦𐋧𐋨𐌠𐌡𐏑𐏒𐒠𐒡𐒢𐒣𐒤𐒥𐒦𐒧𐒨𐡘𐡙𐡚𐡹𐡺𐡻𐡼𐡽𐢧𐢨𐢩𐢪𐢫𐢬𐣻𐣼𐤖𐤚𐤛𐧀𐧁𐧂𐧃𐧄𐧅𐧆𐧇𐩀𐩁𐩂𐩃𐩽𐪝𐫫𐫬𐭘𐭙𐭚𐭛𐭸𐭹𐭺𐭻𐮩𐮪𐮫𐮬𐳺𐳻𐹠𐹡𐹢𐹣𐹤𐹥𐹦𐹧𑁒𑁓𑁔𑁕𑁖𑁗𑁘𑁙𑁦𑁧𑁨𑁩𑁪𑁫𑁬𑁭𑁮𑃰𑃱𑃲𑃳𑃴𑃵𑃶𑃷𑃸𑄶𑄷𑄸𑄹𑄺𑄻𑄼𑄽𑄾𑇐𑇑𑇒𑇓𑇔𑇕𑇖𑇗𑇘𑇡𑇢𑇣𑇤𑇥𑇦𑇧𑇨𑋰𑋱𑋲𑋳𑋴𑋵𑋶𑋷𑋸𑑐𑑑𑑒𑑓𑑔𑑕𑑖𑑗𑑘𑓐𑓑𑓒𑓓𑓔𑓕𑓖𑓗𑓘𑙐𑙑𑙒𑙓𑙔𑙕𑙖𑙗𑙘𑛀𑛁𑛂𑛃𑛄𑛅𑛆𑛇𑛈𑜰𑜱𑜲𑜳𑜴𑜵𑜶𑜷𑜸𑣠𑣡𑣢𑣣𑣤𑣥𑣦𑣧𑣨𑱐𑱑𑱒𑱓𑱔𑱕𑱖𑱗𑱘𑱚𑱛𑱜𑱝𑱞𑱟𑱠𑱡𒐀𒐁𒐂𒐃𒐄𒐅𒐆𒐈𒐉𒐊𒐋𒐌𒐍𒐏𒐐𒐑𒐒𒐓𒐕𒐖𒐗𒐘𒐙𒐚𒐛𒐜𒐞𒐟𒐠𒐡𒐢𒐣𒐤𒐥𒐦𒐧𒐨𒐩𒐪𒐬𒐭𒐮𒐯𒐰𒐱𒐴𒐵𒐶𒐷𒐸𒐹𒐺𒐻𒐼𒐽𒐾𒐿𒑀𒑁𒑂𒑃𒑄𒑅𒑊𒑋𒑌𒑍𒑎𒑏𒑐𒑑𒑒𒑓𒑔𒑕𒑖𒑗𒑘𒑙𒑩𒑪𒑫𒑬𒑭𖩠𖩡𖩢𖩣𖩤𖩥𖩦𖩧𖩨𖭐𖭑𖭒𖭓𖭔𖭕𖭖𖭗𖭘𝍠𝍡𝍢𝍣𝍤𝍥𝍦𝍧𝟎𝟏𝟐𝟑𝟒𝟓𝟔𝟕𝟖𝟘𝟙𝟚𝟛𝟜𝟝𝟞𝟟𝟠𝟢𝟣𝟤𝟥𝟦𝟧𝟨𝟩𝟪𝟬𝟭𝟮𝟯𝟰𝟱𝟲𝟳𝟴𝟶𝟷𝟸𝟹𝟺𝟻𝟼𝟽𝟾𞣇𞣈𞣉𞣊𞣋𞣌𞣍𞣎𞥐𞥑𞥒𞥓𞥔𞥕𞥖𞥗𞥘🄀🄁🄂🄃🄄🄅🄆🄇🄈🄉🄋🄌🆛🆜🆝🆞🆟🆠🆡🆢🆣🆤 like in bash in most modern locales) and a-z only abcdefghijklmnopqrstuvwxyz (not the thousands of characters other shells would match), but that still done on a per character basis (contrary to what would happened if you switched to the C locale in bash to work around the previous issue where it would work on bytes and break up characters into the bytes constituting there encoding for multibyte ones).

It also doesn't choke on bytes not forming valid characters in the locale (and those bytes would match [^0-9a-zA-Z_.-]), so in the end, the resulting files should only contain characters among 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_.-, and no non-character.

In bash, you could do something approaching with:

for f in *[^$accepted_characters]*.mp3; do
  mv -i -- "$f" "${f//[^$accepted_characters]/_}"

(without the safeguards of zmv).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.