Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

How do I pipe the results of a find through a sed to xform the stream, and then use that transformed stream as one of two arguments to a script? IE:

find turns up file1.tiff (among others)
sed transforms file1.tiff --> file1.jpg
a command is executed using both of these arguments: convert file1.tiff file1.jpg

I've tried this a number of ways and can't get it. Here's one attempt that gives the flavor:

find ./out -regex ".*_p[bg]_.*tiff" -exec echo "Processing {}" \; -exec convert {} `echo {} | sed s/tiff/jpg/` \;

I should just give up and write a Python script to do each of these pieces separately, but I'm finally trying to learn UNIX tools and now am obsessed. Help!

share|improve this question
add comment

2 Answers

up vote 8 down vote accepted

I'm not sure you can have a pipe inside -exec. find -exec echo {} \| sed s/tiff/jpg \; doesn't seem to work.

If think your best option is to make a script to do the conversion, e.g.

convert_tiff_to_jpg:

#!/bin/bash
echo "Processing $1"
convert "$1" "${1/%tiff/jpg}"

and call it using find -exec like you had intended:

find ./out -regex ".*_p[bg]_.*tiff" -exec convert_tiff_to_jpg {} \;

There are a few ways to do it using find -print0 | xargs -0, but they are all quite ugly:

find ./out -regex ".*_p[bg]_.*tiff" -print0 |
while IFS= read -d $'\0' -r filename; do
    echo "Processing $filename"
    convert "$filename" "$(echo "$filename" | sed 's/tiff$/jpg/')"
done

or

find ./out -regex ".*_p[bg]_.*tiff" -print0 |
while IFS= read -d $'\0' -r filename; do
    echo "Processing $filename"
    convert "$filename" "${filename/%tiff/jpg}"
done

or

find ./out -regex ".*_p[bg]_.*tiff" -print0 |
    xargs -0 -I FILE bash -c 'F="FILE"; echo "$F" "${F/%tiff/jpg}"'

Note that I changed s/tiff/jpg to s/tiff$/jpg so that if tiff appears anywhere other than the end of the file name, it is not changed.

share|improve this answer
    
Wow, thanks for the detailed answer. Your tips will be a good jumping off point to learning more about BASH, too. Much appreciated. –  shanusmagnus Apr 1 '11 at 3:26
    
You're welcome. On a technical note, find -exec can only run simple commands, not pipelines. This is because find calls exec, rather than system, so no shell syntax is directly supported. –  Mikel Apr 1 '11 at 4:03
    
Just by the way, I'm pretty sure that using \0 as input to a read loop is redundant, and that its purpose is to allow a long commandline string (containing many \0 delimited args) to be passed to a program which is \0 delimiter aware and is only called once, but yet processes the many args internally; each in the same way... This saves a lot of time by avoiding perhaps thousands of individual calls.... A normal newline is fine for the read loop unless any of the filenames have a newline char in them ...Mhhh!?.. maybe I've just convinced myself that it is a good idea after all :) –  Peter.O Apr 1 '11 at 14:55
    
The last one liner isn't ugly, it's pure art :-) –  Phil Jan 23 '13 at 9:37
add comment

You're trying to pass a shell command to -exec, but what it takes is the name of a program and a list of arguments. So you need to tell it explicitly to launch a shell.

find ./out -regex ".*_p[bg]_.*tiff" \
     -exec sh -c 'echo "Processing $0"; convert "$0" "${0%.tiff}.jpg"' {} \;
  • sh -c 'shell code, not using single quotes' is a relatively common idiom when you need shell features in a context that expects a command with arguments.
  • -exec sh -c 'do stuff with "$0"' {} \; is a find and shell idiom. Some versions of find allow writing the {} directly inside the shell code, but this is a bad idea, not only because some find versions don't, but also because this can break horribly if the file name contains special characters (e.g. quotes). So instead the file name is passed to the shell as its 0th argument, and the script uses the shell variable $0 to refer to the file.
  • echo tiff_filename | sed s/tiff/jpg/ is a complex, inefficient, incorrect and fragile way to alter the file name. It's complex and inefficient because you're calling sed. It's wrong because you're replacing tiff anywhere in the file: e.g. tiffany.tiff would become jpgany.tiff. It's fragile because it wouldn't work with some file names (some implementations of echo mangle backslashes). Instead, use ${0%.tiff} to remove the .tiff suffix from the file name in $0.
share|improve this answer
    
Much cleaner. I wonder why it's in $0 rather than $1. –  Mikel Apr 1 '11 at 23:17
1  
@Mikel: Process arguments start at 0, but the first argument (i.e. number 0) is conventionally the executable name, and shells don't let you choose it when executing a command. But sh -c lets you choose $0 for the script, it's the first non-option argument. It might be cleaner to write sh -c 'process "$1"' sh {}. I like $0 in situations like find -exec sh -c 'cp "$0" "$@"' "$destination" {} +, where you need one word plus one list. –  Gilles Apr 1 '11 at 23:35
    
Yes, I would have assumed it would start with $1, but you're right, it's right there in the man page. "If there are arguments after the string, they are assigned to the positional parameters, starting with $0." –  Mikel Apr 2 '11 at 0:46
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.