Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

What is a good command to delete spaces, hyphens, and underscores from all files in a directory, or selected files?

I use the following command with Thunar Custom Actions to slugify filenames:

for file in %N; do mv "$file" "$(echo "$file" | tr -s ' ' | tr ' A-Z' '-a-z' | tr -s '-' | tr -c '[:alnum:][:cntrl:].' '-')"; done

But that command only replaces spaces with dashes/hyphens and lowercases capped characters.

I've used the following command in terminal to delete spaces from thousands of filenames in a folder, and it worked pretty fast:

 rename "s/ //g" *

Again, it only deletes spaces, and not hyphens/dashes and underscores as well.

Ideally I don't want any spaces, hyphens/dashes, and underscores in my filenames. And it would be great if the command could be used with Thunar Custom Actions on selected files.

share|improve this question
1  
I note one problem that a lot of the proposed solutions have, is not properly checking the existance of the "new" name before mv'ing the file. Not doing that could be the potential source of a lot of problems. –  mdpc Aug 5 at 2:44
    
Is it possible to modify John1024's command to check for that? –  user8547 Aug 5 at 2:55
    
@user8547 rename -i "s/[-_ ]//g" * –  Sparhawk Aug 5 at 5:30
    
Thank you Sparhawk. Incidentally, for those interested in using this as a Thunar Custom Action, the command for Thunar is: for file in %N; do mv "$file" echo $file | sed -e 's/[ _-]//g'; done –  user8547 Aug 6 at 1:52

4 Answers 4

up vote 7 down vote accepted

The version of rename that comes with the perl package supports regular expressions:

rename "s/[-_ ]//g" *

Alternatively,

rename -i "s/[-_ ]//g" *

The -i flag will make rename use interactive mode, prompting if the target already exists, instead of silently overwriting.

share|improve this answer
    
I think that depends on what rename are you talking about. The one from util-linux-2.24.2-1.fc20.x86_64 does not support regular expressions. –  Cristian Ciupitu Aug 4 at 20:07
1  
@CristianCiupitu I just checked the man page for the version of rename that you found. Based on the arguments, the version of rename that the OP was using looks like the perl version and not the util-linux version. –  John1024 Aug 4 at 20:24
    
For the record, this is the rename man page for the util-linux version. Anyway, besides that note, the important thing is that the OP got his answer (and you an upvote from me :-D). –  Cristian Ciupitu Aug 5 at 0:54
    
@CristianCiupitu Thanks for finding that. Back at you with a +1. –  John1024 Aug 5 at 6:38

I would replace all those tr commands, with a sed substitution command, e.g.:

for file in %N; do 
    mv "$file" "$(echo "$file" | sed 's/[ _-]//g')"
done
share|improve this answer

Not counting mv, you don't really need an outside process for this at all - you can kind of just poof them.

ifsqz() ( LC_ALL=C sqz=$1
    isf() { [ -e "$1" ] || [ -L "$1" ] ; }  
    set -- * ; set -f
    for f do isf "$f" || break
    IFS=$sqz; set -- $f; IFS=
    isf "$*" || mv -- "$f" "$*"
    done
)

Still though, that means a mv invocation per file, and so probably rename is better. Though this should work given only a POSIX mv in $PATH and a POSIX shell.

So, I came up with a kind of crazy demo for this. The test set is generated like:

tee - - - - <<CGEN |\
dd cbs=90 conv=unblock |\
sed 'G;$!N'";s/^/touch -- '/;s/$/'/" |sh
$( #BEGIN CGEN
   LC_ALL=C
   i= n='"$((i=((i=i+1)==10||i==39||i==47)>0?(i+1):i))"'
   printf '%b -_   ---___'  $(
   IFS=0; eval \
       printf '"\\\\%04o\\\\%04o "' "$(
       printf "$n"' "$i" '%s $(
       printf %.252d
#END
))"))
CGEN

In the first place I will be the first to acknowledge that the above command produces results that can be more easily obtained by other means. But other means would likely not demonstrate as well what might be done with $IFS and a little (sick?) imagination.

So the first bit is fairly straight-forward:

  • tee pipes out 5 copies of its input - the heredocument called CGEN

  • dd blocks its input by newlines at 90 bytes per block and pipes that to ...

  • sed joins 2 of those blocks on two \newline characters, 'single-quotes the results, and prepends the string touch -- for every line cycle before piping out to ...

  • sh which then executes all input as shell commands

The #CGEN bit though... Well, briefly...

  • the bottom printf prints 252 0s

  • the next from last receives 252 '' null-string arguments and for each prints the contents of $n followed by the string " $i "

  • eval interprets the arguments of the next up printf before it prints the results of that interpretation as octal nums prepended by 2 backslashes a piece

  • the last printf prints the byte values for those octals 2 at a time followed by the string -_ ---___ for each pair

  • $n is initialized to an equation that will increment $i by one for every evaluation excepting that it skips the values 10, 39, or 47 - (which are \newline, 'single-quote, and /slash in ASCII decimal respectively)

The end result is a directory containing a lot of really ugly filenames containing every byte in my charset from 1 through 255 excepting the single-quote (only skipped to avoid one more sed s/// statement) and the /slash. Those filenames look like this:

(set -- *; printf '%s\n\n##############\n\n%s\n' "${9}" "${34}")  | cat -A

   ---___ww -_   ---___xx -_   ---___yy -_   ---___zz -_   ---___{{ -_   ---___|| -_   ---$
$
___}} -_   ---___~~ -_   ---___^?^? -_   ---___M-^@M-^@ -_   ---___M-^AM-^A -_   ---___M-^BM-^B -_   ---___M-^CM-^C$
$
##############$
$
 -_   ---___M-ZM-Z -_   ---___M-[M-[ -_   ---___M-\M-\ -_   ---___M-]M-] -_   ---___M-^M-^ -_   ---___M-_M-_ -_$
$
---___M-`M-` -_   ---___M-aM-a -_   ---___M-bM-b -_   ---___M-cM-c -_   ---___M-dM-d -_   ---___M-eM-e -_   ---___$

Now I'll get some data on these files:

chksqz() ( LC_ALL=C sqz=$1
    set -- * ; set -f ; IFS= ; tc="$*"
    printf '#%s\n' \
        "There are $# files in this test directory." \
        "All filenames combined contain a total of ${#tc} bytes."
    IFS=$sqz ; set -- $* ; IFS= ; sc="$*"  
    printf "%s '$sqz'" \
        "#Of which ${#sc} bytes are not"\
        " and $((${#tc}-${#sc})) bytes are"
    set +f ; unset IFS
    printf ".\n#%s\n#Total:\t%d\n#Other:\t%d\n#'$sqz':\t%d\n" \
        "And to confirm these figures:" \
        $(  printf %s * | wc -c 
            printf %s * | tr -d "$sqz" | wc -c
            printf %s * | tr -dc "$sqz" | wc -c
))
chksqz '_ -'

OUTPUT

#There are 101 files in this test directory.
#All filenames combined contain a total of 17744 bytes.
#Of which 2692 bytes are not '_ -' and 15052 bytes are '_ -'.
#And to confirm these figures:
#Total: 17744
#Other: 2692
#'_ -': 15052

Ok. Now finally, to action:

ifsqz '_ -'
chksqz '_ -'

OUTPUT

#There are 101 files in this test directory.
#All filenames combined contain a total of 2692 bytes.
#Of which 2692 bytes are not '_ -' and 0 bytes are '_ -'.
#And to confirm these figures:
#Total: 2692
#Other: 2692
#'_ -': 0

Success! You can see for yourself:

ls

????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
???????????????????????????
???????????????????????????
???????????????????????????
????????????????????????????
????????????????????????????
????????????????
??????????????????????
????????????????????????
??????????????????????????
??????????????????????????
??????????????????????????
??????????????????????????
???????????????????????????
???????????????????????????
???????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
????????????????????????????
??????????????????????????
????????????????????????
????????????????????
??????????????????
????????????????????????????
??
????????????????????????????
??????????????????????????
????????????????????????????
????????????????????????????
????????????????????!!""##
??????????????????!!""##$$
????????????????!!""##$$%%
????????????!!""##$$%%&&((
????????!!""??##$$%%&&(())
$$%%&&(())**??++,,..0011
%%&&(())**++??,,..00112233
&&(())**++,,??..0011223344
))**++,,..??0011223344556
**++,,..00??11223344556677
22334455667788??99::;;<<==>>
445566778899??::;;<<==>>??@@
5566778899::;;??<<==>>??@@AA
6778899::;;<<??==>>??@@AABB
8899::;;<<==??>>??@@AABBCCDD
\\]]^^``aa??bbccddeeffgghh
]]^^``aabbc??cddeeffgghhii
^^``aabbccdd??eeffgghhiijj
??@@AABBCCDDEE??FFGGHHIIJJKK
AABBCCDDEEFF??GGHHIIJJKKLLM
BBCCDDEEFFGG??HHIIJJKKLLMMNN
CCDDEEFFGGHHII??JJKKLLMMNNOO
EEFFGGHHIIJJ??KKLLMMNNOOPPQQ
ffgghhiijjkk??llmmnnooppqqrr
gghhiijjkkllmm??nnooppqqrrss
iijjkkllmmnn??ooppqqrrsstt
jjkkllmmnnoo??ppqqrrssttuuvv
kkllmmnnooppqq??rrssttuuvvww
LLMMNNOOPPQQRR??SSTTUUVVWWXX
MNNOOPPQQRRSS??TTUUVVWWXXYY
OOPPQQRRSSTT??UUVVWWXXYYZZ[[
PPQQRRSSTTUUVV??WWXXYYZZ[[\\
RRSSTTUUVVWW??XXYYZZ[[\\]]
ssttuuvvwwxx??yyzz{{||}}~~??
ttuuvvwwxxyyz??z{{||}}~~????
uuvvwwxxyyzz{{??||}}~~??????
wwxxyyzz{{||??}}~~??????????
xxyyzz{{||}}~~??????????????
YYZZ[[\\]]^^??``aabbccddee
ZZ[[\\]]^^``??aabbccddeeff
share|improve this answer
2  
+1 for a creative use of IFS + printf –  John1024 Aug 5 at 6:51
    
@John1024 - what's really fun: set -- 'some arbitrary' args; eval printf '"%s\n"' "$(IFS=0; printf ' "$@" %s' $(printf %025d))" –  mikeserv Aug 5 at 7:10
1  
new="$(IFS=" -_"; printf %s $1)" forks a subshell (except in ksh93) and has issues with tailing newlines. Another option is to use IFS=' -_'; set -- $1; IFS=; new="$*" (and change your while loop to a for loop) –  Stéphane Chazelas Aug 5 at 14:23
1  
[ -e x ] will return false if x is a symlink to an non-existing or non-accessible file. –  Stéphane Chazelas Aug 5 at 15:23
1  
Nice shell Kung-Fu! –  countermode Aug 19 at 12:47

if you have perl, you usually have rename. you can do:

> type rename
rename is /usr/bin/rename

and show how this script is written:

> cat /usr/bin/rename | head -n 5 #firt 5 lines for example
#!/usr/bin/perl -w
#
#  This script was developed by Robin Barker (Robin.Barker@npl.co.uk),
#  from Larry Wall's original script eg/rename from the perl source.
#

This script doesn't support -i flag (this is version in my system), but maybe yours supports. What about arguments. First is regular expressions with PCRE format, it works like filter, modify input name to output name. List of input names you give by asterisk '*'. for example, you do:

> cd /tmp
> rename 's/ //g' *

in real '*' can be expanded to:

> rename 's/ //g' file1 file2 file3 othe files found in current directory

When you have really big count files, you are in trap. shell will expand your line longer than system accepts. then you can do workaround using find or xargs. using 'find' is problem, because rename will be called many times equal to files count in directory. better use xargs with -r option. one rename call modify many files. for example:

> ls | xargs -r rename 's/ //g'   #thats all, names will be appended at the end of this command.

last problem, what does it mean:

's/ //g'

this is regular expression for modify names. after first '/' is space. this is detected, and replaced by string after second '/'. But there is empty string ended with third '/', then space is replaced by nothing. Option 'g' makes this expression repetative. expression will walk for all name from begin to the end, and detects all spaces.

But what if you have tab character or other 'white' character? there is replacement for this '\s'. what other unneeded characters? simply add it to expression. All close with brackets, for example:

's/[\s_-]//g'

this is all. do you see similarity? I think you should read man perlrequick and man perlretut , this explain you (I hope) how regular expression works. you can use rename command in your own script if you need it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.