33

Most Linux guides consist of pages like "you need to run command_1, then command_2, then command_3" etc. Since I don't want to waste my time running all of them manually, I'd rather create a script

command_1
command_2
command_3

and run it once. But, more often than not, some commands will fail, and I will have no idea, which commands have failed. Also, usually all the rest commands make no sense if something failed earlier. So a better script would be something like

   (command_1 && echo OK command_1 || (echo FAILED command_1; false) )
&& (command_2 && echo OK command_2 || (echo FAILED command_2; false) )
&& (command_3 && echo OK command_3 || (echo FAILED command_3; false) )
&& echo DONE 
|| echo FAILED

But it requires to write too much boilerplate code, repeat each command 3 times, and there is too high chance, that I mistype some of the braces. Is there a more convenient way of doing what the last script does? In particular:

  • run commands sequentially
  • break if any command fails
  • write, what command has failed, if any
  • Allows normal interactions with commands: prints all output, and allows input from keyboard, if command asks anything.

Answers summary (2 January 2020)

There are 2 types of solutions:

  • Those, that allow to copy-paste commands from guide without modifications, but they don't print the failed command in the end. So, if failed command produced a very long output, you will have to scroll a lot of lines up, to see, what command has failed. (All top answers)
  • Those, that print the failed command in the last line, but require you to modify commands after copy-pasting them, either by adding quotations (answer by John), or by adding try statements and splitting chained commands into separate ones (answer by Jasen).

You rock folks, but I'll leave this question opened for a while. Maybe someone knows a solution that satisfies both needs (print failed command on the last line & allow copy-pasting of commands without their modifications).

62

One option would be to put the commands in a bash script, and start it with set -e.

This will cause the script to terminate early if any command exits with non-zero exit status.

See also this question on stack overflow: https://stackoverflow.com/q/19622198/828193

To print the error, you could use

trap 'do_something' ERR

Where do_something is a command you would create to show the error.

Here is an example of a script to see how it works:

#!/bin/bash

set -e
trap 'echo "******* FAILED *******" 1>&2' ERR

echo 'Command that succeeds'   # this command works
ls non_existent_file           # this should fail
echo 'Unreachable command'     # and this is never called
                               # due to set -e

And this is the output:

$ ./test.sh 
Command that succeeds
ls: cannot access 'non_existent_file': No such file or directory
******* FAILED *******

Also, as mentioned by @jick, keep in mind that the exit status of a pipeline is by default the exit status of the final command in it. This means that if a non-final command in the pipeline fails, that won't be caught by set -e. To fix this problem if you are concerned with it, you can use set -o pipefail


As suggested my @glenn jackman and @Monty Harder, using a function as the handler can make the script more readable, since it avoids nested quoting. Since we are using a function anyway now, I removed set -e entirely, and used exit 1 in the handler, which could also make it more readable for some:

#!/bin/bash

error_handler() {
  echo "******* FAILED *******" 1>&2
  exit 1
}

trap error_handler ERR

echo 'Command that succeeds'   # this command works
ls non_existent_file           # this should fail
echo 'Unreachable command'     # and this is never called
                               # due to the exit in the handler

The output is identical as above, though the exit status of the script is different.

|improve this answer|||||
  • 3
    It can be clearer to make do_something a function: trap do_something ERR; do_something() { echo "FAILED" >&2; } -- that also makes it easier to do multiple things in the cleanup code. – glenn jackman Dec 30 '19 at 16:42
  • 1
    Also set -u for variables – D. Ben Knoble Dec 30 '19 at 16:50
  • 1
    Adding set -x to print each command before it is executed gives much better visibility into what step fatally failed, and what went on beforehand. – mtraceur Dec 30 '19 at 19:42
  • 1
    @user000001 You're absolutely right - now the important question is: is that really a problem? Sometimes it is, but a lot of the times, it actually isn't. We need to be discerning with our intuiton that it is bad. For example, at work we have a lot of deployment, build, and automation scripts which spew out thousands of lines due to set -x, always. But it's not actually a problem because we ignore them until something goes wrong, and when it does we love the visibility. On the other hand, for a polished program for which human parsing of output is the normal usecase, set -x is bad. – mtraceur Dec 30 '19 at 22:14
  • 1
    set -e is magic. – hongsy Jan 16 at 14:37
20

Want an unusual solution? If you have make installed then you could put the list of commands in a Makefile for make to execute. Added advantage: you don't have to check if any errors occured. make will automatically check the return value of each command. If it is non-zero, the recipe is terminated with an error. If you want to ignore errors with certain commands, chain them with || true.

Sample Makefile:

.PHONY: all
.SILENT:

all:
    echo "Started list of commands."
    true
    echo "Executing a command which will fail, but I want to ignore failure."
    false || true
    echo "Executing a command which will definitely fail."
    false
    echo "This code will not be reached."

Note: Make (heh) sure to indent your commands like above, but with a tab.

|improve this answer|||||
  • 3
    +1 great out-of-the-box thinking/hack, repurposing make for another task. Portability is pretty good too (though an sh is almost always more readily available than a make). – mtraceur Dec 30 '19 at 19:45
  • 1
    what is the .PHONY: all .SILENT: part for? – törzsmókus Dec 31 '19 at 14:15
  • 1
    @törzsmókus .PHONY: tells make that all of the following targets (in this case, the target all) are not real files. Otherwise, make will normally determine whether to "build" a target based partially on the modification timestamp or existence of the file with the matching name. .SILENT: just tells make to shut up and not print all the commands it's about to run and instead just run them. – 8bittree Dec 31 '19 at 19:28
  • @8bittree thx for the clarification and reference! – törzsmókus Jan 2 at 7:18
16

I believe this only works for bash, but you can try:

set -o xtrace
set -o errexit

Or, if you want to be concise,

set -ex

This will do two things: errexit (-e) will abort the script when there is an error, and xtrace (-x) will print each command just before bash executes it, so in case of failure you know exactly what it was executing.

One downside is that it will clutter the output, but as long as you're OK with that this is a pretty good solution with minimal work.

  • While we're at it, it's usually a good idea to also include set -o pipefail: otherwise, if you run foo | bar, failure of foo will be silently ignored. (Warning: it subtly changes the "exit code" of a pipe statement, so use with caution if you're modifying someone else's script. Also, sometimes you actually don't care about foo's failure, so obviously you can't use pipefail in such a a case.)
|improve this answer|||||
  • 3
    xtrace and errexit are POSIX. -e and -x are even Bourne (where they come from). pipefail comes from ksh93 and is also supported by bash, mksh, zsh, busybox and yash at least. – Stéphane Chazelas Dec 30 '19 at 7:45
  • @StéphaneChazelas It's worth noting that set -o pipefail won't work for dash, which is a fairly common /bin/sh nowadays. -e and -x do work in dash of course. – marcelm Dec 30 '19 at 19:07
  • 1
    @marcelm, true, though it will likely be added there sooner or later since pipefail will make it to the next version of the POSIX specification – Stéphane Chazelas Dec 30 '19 at 21:32
  • @StéphaneChazelas Oh cool, that's good to know! – marcelm Dec 30 '19 at 22:50
3

You could do something like this:

$ for f in command_1 command_2 command_3 command_4
do
  $f
  rc=$?
  if [[ 0 != $rc ]]; then echo Failed command: ${f}; break; fi
done

This assumes the commands have no options, if they do you'll need to enclose each command/option set within quotes.

|improve this answer|||||
  • 3
    could get tricky if any commands set and use variables - probable shell injection. – Jasen Dec 30 '19 at 0:44
  • 1
    This could get messy if any of the commands has multiple words (which is highly likely), especially if any of them is a compound / complex command, or contains quoted/escaped special characters.  @Jasen: Can you give an example of how this approach could be significantly worse than the strawman approach given in the question — simply listing the commands in a script file?  Are you worried about commands changing f? – G-Man Says 'Reinstate Monica' Dec 30 '19 at 7:22
3
failed=0
try(){
(( failed )) && return
"$@" 
ec=$?
if (( ec ))
then
  failed=$ec
  echo "failed($failed): $*" >&2
fi
}

try something args
try other-thing more args

if (( failed )) 
then
  echo "something went wrong: see above." >&2
  exit 1
fi
|improve this answer|||||
  • 1
    The idea behind this script is sound, but I made some edits to fix some typos that were preventing it from working. – user000001 Dec 30 '19 at 7:59
  • @Kevdog777 I have asked on meta about the edit you rejected, feel free to provide your feedback if you want: unix.meta.stackexchange.com/questions/5476/… – user000001 Dec 30 '19 at 10:09
  • 1
    @Romeo Ninov: ^ same as above. – user000001 Dec 30 '19 at 10:09
  • Look, if the commands are try ls -la ~ && ls nonexistant_path try ls -la ~ try echo unreachable then it doesn't work: first try should stop the execution, because ls nonexistant_path fails. But it doesn't stop, and unreachable code executes. the error message produced by ls nonexistant_path is buried beneath long outputs of other commands. It's sad, because unlike the other answers, your answer prints the failed command, and that's important. If it worked correctly for all kinds of commands, I'd mark it as the best answer, despite the low number of upvotes. Can you fix it? – Arqwer Dec 31 '19 at 12:33
  • @Arqwer: I think it should be try ls -la ~ && try ls nonexistant_path – user000001 Dec 31 '19 at 15:57
3

Use this solution if you want to do it in interactive shell:

set -x; command_1 && command_2 && command_3 && echo "Everything OK" || echo "Error while executing last command"; set +x

Output when all commands succeed:

+ command_1
output of command 1
+ command_2
output of command 2
+ command_3
output of command 3
+ echo "Everything OK"
Everything OK
+ set +x

Output when the command_2 fails:

+ command_1
output of command 1
+ command_2
output of command 2
+ echo "Error while executing last command"
+ set +x

The set -x command enables logging of all executed commands. Logged commands are prefixed with one or more pluses (style can be changed using variable $PS4). set +x disables this mode.

You should put the set -x and set +x to the same line because set -x enables logging of all executed commands — including $COMMAND_PROMPT and commands executed by it.


You can also use it for shell scripts:

#!/bin/bash -ex

command_1
command_2
command_3

Shell options can be embedded in the shebang line. -e makes shell exit after first failed command (like when all lines end with &&). -x enables logging of commands. Note that you can pass only one argument to the interpreter. Use /usr/bin/env to pass multiple arguments (#!/usr/bin/env -S bash -ex arg1 arg2 argN).

|improve this answer|||||
1

Perhaps just a function for what you are already doing:

function run_cmd
    ( $* && echo OK $* || ( echo FAILED $*; false ) )

run_cmd echo 1 &&
run_cmd false 2 &&
run_cmd echo 3 &&
echo DONE || echo FAILED

Output of this is:

1
OK echo 1
FAILED false 2
FAILED
|improve this answer|||||
-1

I disagree with almost everything here. Starting with "convenient" and "long list of commands". This is like asking for a "good way to program".

OP seems to explain that he wants control flow. He does not give an idea what kind of (special? compound?) commands it could be.

This:

command_1;
command_2;
command_3;

is either

command_1; command_2; command_3;

or

 command_1
 command_2
 command_3

Here you see "convenience I": a easiest-to-read (long) list of commands.


I dont't understand:

...some commands will fail, and I will have no idea, which commands have failed.

This script:

ls
ls NOSUCH >/dev/null
ls -l >/dev/null && echo "long ls OK"

gives these lines:

cl.sh
ls: cannot access 'NOSUCH': No such file or directory
long ls OK

Even though the second command is "silent", on error, I see it, and the reason why. No stderr redirection required, it is already well directed.

The third line shows that the command and the message is not the same.


set -e is a good answer, but that is a rigid special solution.


My point is: if somebody wants to reinvent the wheel (or invent instant water): go ahead. But say so, and don't talk about some mysterious "long list of commands"


I'd rather create a script...and run it once.

...to save time! I don't think a linux guide's commands can be called a "long list of commands", which need automatisation.

|improve this answer|||||
  • Could you clarify what you mean by "[set -e] is a rigid special solution"? Is this about (lack of) portability to other shells? – user000001 Dec 30 '19 at 20:03
  • 1
    "usually all the rest commands make no sense", but not always. With "exit always" you exit on non-zero, but e.g. grep can give 1 (no match) or 2 (no file), and both can mean information and not error. I just don't buy that long list of commands, and I am for "convenience II" i.e. flexibility with the usual means (if, &&, ). – rastafile Dec 30 '19 at 21:02
  • 1
    This isn't an answer, it's a critique of the OP's question and the OP themself! Please don't post this sort of thing. If you don't find the question interesting, then downvote and move on. There's no call for this sort of snide answer. Especially since, as should be obvious, 3 commands was just an example and there could be several dozen. And it's entirely possible for a command to fail with no error output whatsoever. – terdon Dec 30 '19 at 21:04
  • @rastafile: Thanks, that makes sense. – user000001 Dec 30 '19 at 21:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.