Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I've got something fairly simple I want to do. I want to use montage on a directory that contains thousands of images, with very few options, namely:

me@home$ montage -size 256x256 DSC01*.JPG.svg output.png

...but that's just not good enough, as it only grabs about 100 images at a time; neither is

me@home$ montage -size 256x256 *.svg output.png

...which grabs all of the images at the same time, as the resulting file is too big to parse.

What I want to do is to iterate over something like 100-200 files at a time. I guess this could be implemented using a for loop (?), but I'm just a bit confused about how to do that. I guess there's probably a clever way to use find -exec or xargs that I'm not thinking of. I'm using bash, but I use zsh occasionally.

So, in conclusion, I'm looking for a one liner that, given 2600 image files, calls montage about 13 or 26 times (once for each 100-200 files), and given n files, can be called a multiple of n times.

share|improve this question
1  
Are your files all named DSC0100.JPG.svg ... DSC2600.JPG.svg? –  jw013 Jul 12 '12 at 15:46
add comment

4 Answers

up vote 4 down vote accepted

A bash method, using special array features; probably translatable to zsh with some modification:

image_files=(*.svg) # use your own glob expression
n=200               # number of files per command line; adjust to taste
for ((i=0; i < ${#image_files[@]}; i+=n)); do
        montage -size 256x256 "${image_files[@]:i:n}" output-"$i".png
done
share|improve this answer
1  
i found this bit of bash scripting is also very extensible. i just used it to move some files around (16 files per directory) and it worked on the first try, which was a bit of a surprise. thank you. –  ixtmixilix Aug 13 '12 at 0:28
add comment

You can use xargs for that; unfortunately, it is not possibly to combine -I (for instering into the middle of a command line) and -L (for limiting the number of files for a single call to the executable). Therefore, I created this command line as an example (but beware of special characters in file names, they are not supported):

 ls . | \
   xargs -n 100 echo | \
   (a=1; 
    while read args; do 
     echo montage -size 256x256 $args output-$a.png;
     a=$((a+1)); 
    done
   )

Remove the echo if you want to really execute the command.

Caveats:

  • filenames may not contain spaces or other special characters
  • the last montage line might have less than 100 files in it

Update:

This is the corresponding for loop, which (I hope) solves the problem with spaces in file names:

a=0
b=0
lst=
for f in *; do 
  a=$((a+1))
  lst="$lst '$f'"
  if test $a -ge 100; then 
    eval echo montage --args $lst target-$b.png
    b=$((b+1))
    a=0
    lst=
  fi 
done

Update 2: A python solution, which should be immune to special characters in file names

#!/usr/bin/env python
# iterate.py

"""Usage: 
%prog <number per call> <file pattern> <command prefix> -- <command postfix>
e.g.  %prog 100 "DSC01*.jpg.svg" montage -size 256x256 -- output-%i.png """

import sys,subprocess,glob,os

if len(sys.argv) < 5: 
  print __doc__.replace("%prog", os.path.basename(sys.argv[0]))
  sys.exit(1)

def chunks(l, n): 
  for i in xrange(0, len(l), n): yield l[i:i+n]

num, pattern, args = int(sys.argv[1]), sys.argv[2], sys.argv[3:]
files, idx = glob.glob(pattern), args.index("--")
before, after = args[0:idx], args[idx+1:]

for idx,chunk in enumerate(chunks(files,num)):
  subprocess.call( before + chunk + [s.replace("%i",str(idx)) for s in after] )
share|improve this answer
2  
If you are going to recommend using ls in a pipe to parse its output, you should also warn about the many dangers of doing so prominently, and at the beginning to make sure people see it. –  jw013 Jul 12 '12 at 15:49
    
@jw013 +1 Yes, that's definitely a concern. However, his posting let me assume that he was using photos directly imported from a digital camera, which do not contain any special characters. How would you suggest to tackle that problem? –  daniel kullmann Jul 12 '12 at 18:00
    
Yes, it looks like the file names are relatively benign (hence no downvote). However the OP hasn't really specified what they look like beyond *.svg (which is why I posted a comment on the question asking). In the most general case where you need to handle all file names, you'd have to resort to shell globbing and arrays or find -print0 | xargs -0 constructs. See my answer for an example of the former. –  jw013 Jul 12 '12 at 19:35
    
@jw013 Your answer is really nice! I never took the effort to learn how arrays work in bash. Maybe I should. –  daniel kullmann Jul 13 '12 at 7:58
add comment

Here's a version using xargs that's safe for any filename, but requires a temporary file to store the count. Adjust the '-n 100' to adjust how many files per montage. You can also swap the "printf" with a "find -print0", but make sure it doesn't find "count.temp".

echo 1 >count.temp
printf "%s\0" *.svg | xargs -0 -n 100 sh -c '
    a=`cat count.temp`
    montage --blah "$@" output-"$a".png
    let a=a+1
    echo "$a" >count.temp
    '
rm count.temp
share|improve this answer
add comment

With GNU Parallel you can do:

parallel -N200 montage -size 256x256 {} output{#}.png ::: *.svg

It is of course safe for files with special characters (as you can normally expect of GNU Parallel).

Minimal installation

If you just need parallel and do not have 'make' installed (maybe the system is old or Microsoft Windows):

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem
mv parallel sem dir-in-your-$PATH/bin/

Watch the intro video for a quick introduction: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1 or at http://tinyogg.com/watch/TORaR/ and http://tinyogg.com/watch/hfxKj/

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.