0

I have the following command:

find stdlib/main -type f -exec sh -c "echo {} | sed -e 's/stdlib\/main\///g' -e 's/\.q//g' -e 's/\//\./g' -e 's~^~/resource:\"{},~g' -e 's/$/\"/g'" \;

The goal is to find all files in stdlib/main (and subdirectories) and format them as such: {filename},{filename-with-stdlibmain-removed-and-extension-removed-and-slashes-changed-to-dots}

The command works perfectly when I run it myself. But I am trying to use it in a makefile as such:

STDLIB_RESOURCES=$(shell find stdlib/main -type f -exec sh -c "echo {} | sed -e 's/stdlib\/main\///g' -e 's/\.neo//g' -e 's/\//\./g' -e 's~^~/resource:\"{},~g' -e 's/$/\"/g'" \;)

When I run the makefile, I get one of these errors per file found:

sed: -e expression #5, char 5: unterminated `s' command

What am I missing here?

  • 2
    What are you missing? Probably the fact that you have quotes within quotes within quotes, and that $(…) is adding one more level of containment. A workaround would be to put the sed commands into a file and invoke it with sed -f. I don’t immediately know how to do it in a self-contained way. – G-Man Apr 12 at 0:15
  • There's gotta be some way to do it in a self-contained way... – Matthew Peterson Apr 12 at 0:16
  • Try removing just the last sed expression (-e 's/$/\"/g'). This is the most likely to be tripping you up as it contains a " character. If that works, then you probably need to escape it better. – Crypteya Apr 12 at 0:58
  • I get this when I do that: /bin/sh: -c: line 0: unexpected EOF while looking for matching /bin/sh: -c: line 0: unexpected EOF while looking for matching `"'' /bin/sh: -c: line 1: syntax error: unexpected end of file – Matthew Peterson Apr 12 at 2:31
  • Checked again and both of the last two sed expressions have " characters. Remove both of them. If that returns successfully, look at how to escape better. Your new error message definitely makes it look like the " that you removed was being interpreted before it could count in the sed command – Crypteya Apr 12 at 5:06
0

The main thing you are missing is that $ is a special character to make, and quoting is different in Make and the shell.

So for example

's/$/\"/g'

the `` pair protect everything inside them to the shell (and incidentally makes the \ unnecessary), but doesn't to make, so to make it looks like

 's/\"/g' 

assuming you don't have a variable called / (which you could in make, but typically not in the shell).

The first thing to do is to replace $ with $$.

0

Using {} in an in-line script that you execute through find is a code injection vulnerability. Don't do that. The first argument to sh -c (the script) should be single quoted and the arguments to that script should be passed on its command line.

Instead, write the find command as something like the following (uses bash instead of sh to be able to use ${parameter//pattern/word} in one place):

find stdlib/main -type f -exec bash -c '
    for pathname do
        string=${pathname#stdlib/main/} # delete initial path
        string=${string%.*}             # delete suffix after last dot
        string=${string////.}           # change slashes to dots

        # output:
        printf "resource:\"%s,%s\"\n" "$pathname" "$string"
    done' bash {} +

Instead of using sed, this uses parameter substitutions to modify the pathnames found by find. The in-line bash script will be executed for batches of found files and will iterate over the pathnames in each batch. The printf would output the data transformed in the same way as what your sed command is doing (if I managed to decipher it correctly, it wasn't just what you described).

How you later cope with filenames containing double quotes and commas is another issue (the output string after resource: would potentially be difficult to parse).

It would be easiest to put the find command into a separate script and call that from GNU make in $(shell ...), or you'll end up with something like

STDLIB_RESOURCES := $(shell     \
find stdlib/main -type f -exec bash -c '    \
    for p do                                \
        s=$${p\#stdlib/main/};              \
        s=$${s%.*};                         \
        s=$${s////.};                       \
        printf "resource:\"%s,%s\"\n" "$$p" "$$s"; \
    done' bash {} + )

in your Makefile (due to the way GNU make handles variables etc.) Also note the :=. You need this for the command to be executed immediately upon assigning to the variable, not every time the variable is accessed.

Related:

  • Congrats on becoming a moderator :) I put you as #1 when I voted. – Crypteya Apr 12 at 5:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.