Move arbitrary string to end of command

Question

I often find that I've concocted some incantation which I'd like to reuse, but it's far too susceptible to ad-hoc tweaking to store as a function. These commands often have some critical variable component, such as a filename, buried deep within. I'd like to move that component to the end of the command.

For example, my latest witchcraft prints the table definitions of all new INSERT statements in a diff file:

for T in $(grep -oP "(?<=^\+INSERT INTO \`)[\w-]+(?=\`)" foo.diff | sort -u) ; do docker-compose exec mysql mysqldump --no-data some_database $T | grep -P "^\s*\`|^CREATE|^\) |^\s*CONSTRAINT" | sed "s/ COMMENT .*,$/,/" | sed "s/CONSTRAINT \`\w*\` //" | sed "s/^) ENGINE.*;/);/" ; done

Formatted for legibility:

for T in $(grep -oP "(?<=^\+INSERT INTO \`)[\w-]+(?=\`)" foo.diff | sort -u)
  do
  docker-compose exec mysql mysqldump --no-data some_database $T
    | grep -P "^\s*\`|^CREATE|^\) |^\s*CONSTRAINT"
    | sed "s/ COMMENT .*,$/,/"
    | sed "s/CONSTRAINT \`\w*\` //"
    | sed "s/^) ENGINE.*;/);/"
done

I'm very likely going to want to use this again in the future on a file that is not called foo.diff, so I'd like the command to end with the file name. My first inclination would be to just wrap the whole thing in a function, and have the filename be the $1 parameter.

However, it is very likely that I'm going to tweak it further as I run it. Add a bit of sed here, maybe a touch of awk there. So I'd still like the whole command to be on the CLI when I Ctrl-P to run it again. Therefore I decided to use a variable:

$ FILENAME=diff-03-04.sql

That's appropriate for this specific example as I'll be using the same file for each incantation today, but there exist other examples where I'd really like to be able to just tweak the end of the command. So this:

$ for Q in $(foo Torvalds bar) ; do baz $Q ; done

Would become something akin to (but not necessarily exactly):

$ for Q in $(foo SOME_MAGIC_HERE bar) ; do baz $Q ; done < Torvalds

Answer 1

It seems to me that you have misunderstood the power of a script or function. Personally I prefer scripts because they are self contained and available regardless of the interactive shell I'm using, but a function can do exactly the same thing. For me, systemwide scripts go into /usr/local/bin and personal (junk) scripts go into ~/bin. Both are in my $PATH.

Create the directory ~/bin. Add it to your $PATH (export PATH="$PATH:$HOME/bin"). Now any executable script placed in that directory will be available to you as a new command.

Let's take your example as a case in point. You want the file name to be a use-time selection, so let's make it the first argument of a script.

Create the file ~/bin/tables:

#!/bin/bash
file=$1
for T in $(grep -oP "(?<=^\+INSERT INTO \`)[\w-]+(?=\`)" "$file" | sort -u)
  do
  docker-compose exec mysql mysqldump --no-data some_database "$T"
    | grep -P "^\s*\`|^CREATE|^\) |^\s*CONSTRAINT"
    | sed "s/ COMMENT .*,$/,/"
    | sed "s/CONSTRAINT \`\w*\` //"
    | sed "s/^) ENGINE.*;/);/"
done

Make the script executable with chmod a+x ~/bin/tables and now you have a new command.

tables foo.diff
tables another.diff

Answer 2

Everybody here has given excellent advice, in the other answer and in the comments. The best method that I've found is to wrap the command in a function and run that function in a single command:

$ _() { echo $1; } && _ beer
beer
$

Additionally, I've replaced the for T in $() syntax with cmd | while read as suggested by Kusalananda. Thus the final syntax is as follows:

_() { grep -oP "(?<=^\+INSERT INTO \`)[\w-]+(?=\`)" $1 | sort -u | while read -r T ; do docker-compose exec mysql mysqldump --no-data some_database $T | grep -P "^\s*\`|^CREATE|^\) |^\s*CONSTRAINT" | sed "s/ COMMENT .*,$/,/" | sed "s/CONSTRAINT \`\w*\` //" | sed "s/^) ENGINE.*;/);/" ; done } && _ foo.diff

Formatted for readability:

_() {
  grep -oP "(?<=^\+INSERT INTO \`)[\w-]+(?=\`)" $1
    | sort -u
    | while read -r T ;
      do
      docker-compose exec mysql mysqldump --no-data some_database $T
        | grep -P "^\s*\`|^CREATE|^\) |^\s*CONSTRAINT"
        | sed "s/ COMMENT .*,$/,/"
        | sed "s/CONSTRAINT \`\w*\` //"
        | sed "s/^) ENGINE.*;/);/"
    done
}
&& _ foo.diff

This method preserves the ability to tweak anywhere in the command that I need, yet easily change the argument (in this case, the filename), without needing to escape quotes in the commands themselves.