After some googling, I found a way to compile BASH scripts to binary executables (using shc).
I know that shell is an interpreted language, but what does this compiler do? Will it improve the performance of my script in any way?
3 Answers
To answer the question in your title, compiled shell scripts could be better for performance — if the result of the compilation represented the result of the interpretation, without having to re-interpret the commands in the script over and over. See for instance ksh93's shcomp or zsh's zcompile.
However, shc doesn’t compile scripts in this way. It’s not really a compiler, it’s a script “encryption” tool with various protection techniques of dubious effectiveness. When you compile a script with shc, the result is a binary whose contents aren’t immediately readable; when it runs, it decrypts its contents, and runs the tool the script was intended for with the decrypted script, making the original script easy to retrieve (it’s passed in its entirety on the interpreter’s command line, with extra spacing in an attempt to make it harder to find). So the overall performance will always be worse: on top of the time taken to run the original script, there’s the time taken to set the environment up and decrypt the script.
answered Oct 20, 2020 at 8:21
-
3Comeau Computing used to market CCsh, but they’re out of business, and I’m not aware of any other shell compiler. Oct 20, 2020 at 8:45
-
27To be honest, if performance is that big an issue you probably shouldn't be looking at shell scripts in the first place.– ShadurOct 20, 2020 at 10:18
-
12@Shadur In addition, many times shell scripts are calling compiled programs that do the heavy lifting, so it's not too much of a boost anyways. E.g., if you have a script that calls awk, sed, and grep, those are all compiled. Oct 20, 2020 at 16:50
-
5@CaptainMan: If you were programming in a compiled language, you'd read the data yourself and use a regex library instead of fork+exec of multiple separate processes, each of which have to pay the startup overhead for a new process, and for dynamic linking (which is a significant part of the total cost for running
grepon a short file). Also, for large amounts of data, avoiding piping the data between processes, costing some overall memory bandwidth, and synchronization between cores in the kernel. You wouldn't expect a shell-script compiler to do that, hence Shadur's point: avoid entirely Oct 21, 2020 at 4:12 -
4So essentially the only valid purpose of
shcwas for people who write insecure code and want to hide their hard coded passwords and vulnerabilities, or script kiddies who are trying to hide obvious malware with the lowest amount of effort possible?– john doeOct 21, 2020 at 21:24
After some googling, I found a way to compile BASH scripts to binary executables (using
shc).
It's quite unfortunate that that shc contraption is still featured in google search results, even after it has been utterly debunked all these years: shc is not a compiler, and it does not prevent the source code of the script from being looked at and "stolen".
If anything, shc is even stupider than it has to be, because, after unmangling the script source, it's just passing it as an argument to bash -c, which means that it's visible in /proc/<pid>/cmdline to any user, not just the one running the script. That also runs into the Linux's length limit for a single command line argument (128k bytes). But to make things even more ridiculous, the first part of that argument is filled up with white spaces, so it doesn't appear in ps ;-)
Will it improve the performance of my script in any way?
Yes, your script may not work at all, which means that it will terminate sooner.
-
4Its current maintainer has been engaging in a lot of “marketing”, including creating a Wikipedia page for it which has somehow survived... Oct 21, 2020 at 14:53
-
1this is the best answer. Very helpful info to know to steer clear of shc. Oct 21, 2020 at 20:33
-
2The padding with spaces is useless since
pscan easily be told to show the full command-line length, and will do that automatically when piped e.g. togrep. Oct 21, 2020 at 20:48 -
@StephenKitt, to be fair, at the time
shcwas written, that was not usually the case. On many systems, argument list of processes can only read by root (and you need a setuid rootpsthat only displays a few dozen bytes of the arg list), and until relatively recently you couldn't get more than 4KiB from the command line on Linux. Oct 22, 2020 at 16:51 -
1@StéphaneChazelas fair point, since 4.2 five years ago which is quite recent (and somewhat ironically, in a similar timeframe to
shc’s revival and all the fuss around it). Oct 22, 2020 at 17:42
In general, there is no way to compile a shell script, because new source text can be introduced by several method at run time, which has therefore bypassed the compiler phase. That new source would be unable to interact with the compiled-in functions or variables.
Two methods of creating runtime source would be:
Source a side file that may have been created or modified since the original script was compiled.
Construct at runtime an arbitrary command in a string, and exec it.
answered Oct 20, 2020 at 11:53
awk,grepand friends, walking directories) a compiled version isn't going to improve it significantly.