Using wildcards to elegantly convert thousands of epub files to mobi

Question

I am executing the below command for 1000 files:

ebook-convert <name-of-first-file>.epub <name-of-first-file>.mobi
ebook-convert <name-of-second-file>.epub <name-of-second-file>.mobi

Apparently, instead of manually doing this for 1000 files, one could write a bash script for the job.

I was wondering if there is an easier way to do something like this in Linux though, a small command that would look something like

ebook-convert *.epub *.mobi

Can you use wildcards in a similar way, that works for a scenario like the above?

Answer 1

You can’t do it directly with wildcards, but a for loop can get you there:

for epub in ./*.epub; do ebook-convert "${epub}" "${epub%.epub}.mobi"; done

Zsh supports a more elegant form of this loop.

Instead of a shell script, if your file names don’t contain whitespace characters, and more generally can be safely handled by Make and the shell, you can use GNU Make; put this in a Makefile:

all: $(patsubst %.epub,%.mobi,$(wildcard *.epub))

%.mobi : %.epub
        ebook-convert ./$< ./$@

and then run make, which will ensure that all .epub files are converted to a .mobi file. You can run this repeatedly to update files as necessary — it will only build files which are missing or older than their source file. (Make sure that the ebook-convert line starts with a tab, not spaces.)

Answer 2

With the zsh shell:

for f (./*.epub) ebook-convert $f $f:r.mobi

With $f:r expanding to the rootname (the part without the extension) of $f like in csh/vim...

Or:

autoload -Uz zmv # best in ~/.zshrc
zmv -P ebook-convert './(*).epub' './$1.mobi`

(as ebook-convert doesn't seem to recognise the -- option delimiter, we have to use a ./ prefix and use -P instead of -p to be able to cope with file names that start with -)

zmv is primarily intended for batch renaming, but also for batch copying/linking with -C/-L or can be extended to any form of change/conversion... by specifying the program to do it with -p/-P.

Or:

autoload -Uz zmv
alias ebc='noglob zmv -WP ebook-convert'
ebc ./*.epub ./*.mobi

With -W, zmv captures all wildcards on the source pattern, and converts all wildcard with incrementing ${1}, ${2}, etc in the replacement. So zmv -W './*.epub' './*.mobi' is the same as zmv -W './(*).epub' './${1}.mobi'. noglob disables globbing in the arguments to the command which avoids having to quote.

Or:

autoload -Uz zargs # best in ~/.zshrc
zargs -I@ ./*.epub(:r) -- ebook-convert @.epub @.mobi

zargs being zsh's xargs-like command to batch-process lists of arguments. ./*.epub(:r) gets you the rootnames of the .epub files, and with -I@, zargs runs the ebook-convert command line with each @ replaced with each of those rootnames in turn.

If your ebook-convert command accepted being called as ebook-convert file1.epub file1.mobi file2.epub file2.mobi ..., you could also do:

ebook-convert ./*.epub(e['reply=($REPLY $REPLY:r.mobi)'])

With that glob expanding to ./file1.epub ./file1.mobi ./file2.epub ./file2.mobi... thanks to the eval glob qualifier that runs the provided code for each matching file, where setting $reply defines the list of arguments the glob should expand to.

Or:

(){epub-convert $^@.{epub,mobi};} ./*.epub(:r)

Where we pass the list of rootnames of .epub files to an anonymous functions that uses brace expansion to pass arguments with both .epub and .mobi appended.

In all of those, to restrict the *.epub glob expansion to those which do not already have a newer corresponding .mobi file, you can add that e['[[ ! $REPLY:r.mobi -nt $REPLY ]]'] glob qualifier, or add the check as part of the loop:

for epub (./*.epub) {
  mobi=$epub:r.mobi
  [[ $mobi -nt $epub ]] || ebook-convert $epub $mobi
}

Answer 3

Can you use wildcards in a similar way, that works for a scenario like the above?

Not like ebook-convert *epub *mobi, because of how wildcards - really "shell globbing" - works. But, you can get started with a wildcard.

Shell globbing is, conceptually, very straightforward: find all of the files that match the glob and replace the glob with that file list, handling whitespace and other "special" characters so that the invoked action (often a program; here, ebook-convert) gets each file as a single, separate parameter.

So, given a folder with a.epub, b.epub, and file with spaces.epub, the shell will expand *.epub to a.epub, b.epub, and file with spaces.epub as 3 separate arguments to whatever is being invoked (here, ebook-convert).

Given that same folder, *.mobi won't match anything, so ebook-convert will receive an argument that is literally *.mobi. From ebook-convert's perspective, it's being given a list of three epub files and one mobi file that doesn't exist; how it handles that list of parameters is up to it (at a guess, it'll either complain about too many parameters or will sequentially try to convert each of the epubs into a mobi file literally named "*.mobi").

Note that there's no global guarantee about how a program (or shell built-in or function or script, etc.) will handle a parameter that it expects to be a file name but that contains a glob. Typically, that parameter will be treated as a string literal, and will react to *.mobi not existing just like it would to anything_else.mobi not existing, but there's no law that states that that has to happen.

The same thing happens for other globs; eg., ?.epub would include a.epub and b.epub, but not file with spaces.epub.

As others have noted, you can use globs to power loops - for file in *.epub ; do .... Note that references to "file" need to be quoted to handle spaces: the globbing in for file in *.epub only ensures that file with spaces.epub is a single parameter to the for loop itself, but doesn't extend into the body of the loop (that is, for file in *.epub ; do ebook-convert $file will send three separate parameters for file with spaces.epub: file, with, and spaces.epub).

All of this is also why you often need to quote arguments to programs that expect glob characters: doing so prevents the shell from expanding the glob and actually lets the invoked program see the argument as you wrote it. For example, find . -type f -name '*.epub' will find (and print the names of) all epub files in this directory and any child directories; find . -type f -name *.epub will error out since it'll see b.epub and file with spaces.epub as arguments with which it doesn't know what to do.

Answer 4

using bash...

find . -name '*.epub' | xargs  basename -s .epub | xargs -I {} ebook-convert {}.epub {}.mobi ;

or, for filenames with spaces...

find . -name '*.epub' -print0 | xargs -0 basename -s .epub | xargs -I {} ebook-convert "{}.epub" "{}.mobi" ;

breaking it down...

find . -name '*.epub'

find . -name '*.epub' | xargs  basename -s .epub 

CMD='echo ebook-convert' && find . -name '*.epub' | xargs basename -s .epub | xargs -I {} $CMD "{}.epub" "{}.mobi" ;

the -print0 and -0 options cause those commands to use nulls instead of spaces to separate filenames.

-I creates a placeholder {} that works by substitution.

Ps: although this is a useful technique, I also think a Makefile (see above) is a great solution for this kind of task.

Answer 5

I think that you can do as cute as the %.mobi: %.epub from the GNU make part of the accepted answer --without GNU make and its filenames limitations-- with a little wrapper:

from_to(){
    sp=${1%%%*}; ss=${1#*%}; shift
    dp=${1%%%*}; ds=${1#*%}; shift
    for s in "$sp"*"$ss"; do
        d=${s#"$sp"}; d=$dp${d%"$ss"}$ds
        "$@" "$s" "$d" || exit 1
    done
}

Which you could use as

from_to %.epub %.mobi ebook-convert
from_to dir1/book_%.epub dir2/%.mobi ebook-convert

Mock test example:

% touch {1,2,3}.foo
% from_to ./%.foo bar/%.baz echo translate --
translate -- ./1.foo bar/1.baz
translate -- ./2.foo bar/2.baz
translate -- ./3.foo bar/3.baz

Anyways, you have to use something else that *, as that has a very established meaning in the Unix shell.