9

I wrote a bash script, and I executed it without compiling it first. It worked perfectly. It can work with or without permissions, but when it comes to C programs, we need to compile the source code. Why?

14
  • 9
    It means C is a compiled language and Bash is an interpreted language. Nothing more, nothing less. – Radon Rosborough Dec 15 '16 at 5:47
  • 10
    I'm voting to close this question as off-topic because it is not related to U&L, and even if it were, it is too broad as one can discuss it endlessly without providing a satisfying answer, not speaking of an answer that fits the general idea of U&L being a knowledge base for solving problems. – countermode Dec 15 '16 at 10:01
  • 4
    @countermode You stack(exchange|overflow) folks and and your trigger-happy close-votes. The question is not broad: it reveals a very specific missing piece of understanding: the difference between compiled and interpreted languages. This takes a couple of paragraphs to explain, and another couple of paragraphs to point out the (dis)advantages of each, plus a footnote to summarize that C and bash had different goals, thus they picked the different approaches. – mtraceur Dec 15 '16 at 10:17
  • 2
    @mtraceur Sorry, I don't mean to offend anyone. As of trigger-happy close-votes this is a bit unfair. Five votes are required to close a question, and if anyone else votes to not close it I'll be fine with it. You're absolutely right about the question, yet I am not sure whether such questions belong to U&L according to unix.stackexchange.com/help/on-topic. – countermode Dec 15 '16 at 10:24
  • 2
    @mtraceur if you think it's not broad, I don't agree with you. There's much to tell about the difference between compiler/interpreter/etc. and interpreted/compiled languages. There are caevats that are easy to hit. Moreover, the question fits "Computer Science" or "Programmers" SE better. – MatthewRock Dec 15 '16 at 10:44
24

It means that shell scripts aren't compiled, they're interpreted: the shell interprets scripts one command at a time, and figures out every time how to execute each command. That makes sense for shell scripts since they spend most of their time running other programs anyway.

C programs on the other hand are usually compiled: before they can be run, a compiler converts them to machine code in their entirety, once and for all. There have been C interpreters in the past (such as HiSoft's C interpreter on the Atari ST) but they were very unusual. Nowadays C compilers are very fast; TCC is so fast you can use it to create "C scripts", with a #!/usr/bin/tcc -run shebang, so you can create C programs which run in the same way as shell scripts (from the users' perspective).

Some languages commonly have both an interpreter and a compiler: BASIC is one example that springs to mind.

You can also find so-called shell script compilers but the ones I've seen are just obfuscating wrappers: they still use a shell to actually interpret the script. As mtraceur points out though a proper shell script compiler would certainly be possible, just not very interesting.

Another way of thinking about this is to consider that a shell's script interpreting capability is an extension of its command-line handling capability, which naturally leads to an interpreted approach. C on the other hand was designed to produce stand-alone binaries; this leads to a compiled approach. Languages which are usually compiled do tend to sprout interpreters too, or at least command-line-parsers (known as REPLs, read-eval-print loops; a shell is itself a REPL).

5
  • 1
    Many "compilers" for scripting languages are just wrappers. I remember a BASIC compiler which would just concatenate the interpreter executable with a zipped source code. – Dmitry Grigoryev Dec 15 '16 at 9:25
  • @DmitryGrigoryev I can easily believe that! For BASIC I was thinking of compilers like Turbo Basic. I think there were some real compilers for DOS BAT files, but I might be mistaken! There's also the common p-code approach... – Stephen Kitt Dec 15 '16 at 9:32
  • 2
    Note that a "true" compiler for the shell is entirely possible. It's just not generally worth the effort/complexity, since a naive variant would just produce a program that generally calls a bunch of execve, open, close, read, write, and pipe syscalls, interspersed with some getenv, setenv, and internal hashmap/array operations (for non-exported variables), etc. Bourne shell and derivatives are also not programming languages that benefit as much from low-level compiler tweaks like code re-ordering, etc. – mtraceur Dec 15 '16 at 10:11
  • @StephenKitt would you mind if I raise a new question relating to a comment of your answer. Although you answer explains a lot. but also raises a new question for me. – AReddy Dec 15 '16 at 11:34
  • Ya sure obviously, I would open a new question. I thought you might want me to ask it on chat. because it relate to your answer. – AReddy Dec 15 '16 at 11:38
7

It all comes down to the technical difference between how the program you can read/write as a human gets converted into the machine instructions your computer understands - and the different advantages and disadvantages of each method is the reason why some languages are written to need compilers, and some are written to to be interpreted.

First, the technical difference

(Note: I'm simplifying a lot here for the sake of addressing the question. For a more in-depth understanding, technical notes at the bottom of my answer elaborate/refine some of the simplifications here, and the comments on this answer have some useful clarifications and discussion as well..)

There are basically two general categories of programming languages:

  1. Another program (the "compiler") reads your program, determines what steps your code says to do, and then writes a new program in machine code (the "language" your computer itself understands) that does those steps.
  2. Another program (the "interpreter") reads your program, determines what steps your code says to do, and then does those steps itself. No new program is created.

C is in the first category (the C compiler translates the C language into your computer's machine code: the machine code is saved into a file, and then when you run that machine code, it does what you want).

bash is in the second category (the bash interpreter reads the bash language and the bash interpreter does what you want: so there's no "compiler module" per se, the interpreter does the interpreting and the executing, whereas a compiler does reading and translating).

You might have already noticed what this means:

With C, you do the "interpret" step once, then whenever you need to run the program, you just tell your computer to execute the machine code - your computer can just run it directly without having to do any extra "thinking".

With bash, you have to do the "interpret" step every time you run the program - your computer is running the bash interpreter, and the bash interpreter does extra "thinking" to figure out what it needs to do for each command, every time.

So C programs take more CPU, memory, and time to prepare (the compiling step) but less time and work to run. bash programs take less CPU, memory, and time to prepare, but more time and work to run. You probably don't notice these differences most of the time because computers are very fast nowadays, but it does make a difference, and that difference adds up when you need to run big or complicated programs, or a lot of little programs.

Also, because C programs are converted into machine code (the "native language") of the computer, you can't take a program and copy it onto another computer with a different machine code (for example, Intel 64-bit onto Intel 32-bit, or from Intel to ARM or MIPS or whatever). You have to spend the time to compile it for that other machine language again. But a bash program can just be moved over to another computer that has the bash interpreter installed, and it'll run just fine.

Now the why part of your question

The makers of C were writing an operating system and other programs on hardware from several decades ago, that was rather limited by modern standards. For various reasons, converting the programs into the computer's machine code was the best way towards that goal for them at the time. Plus, they were doing the kind of work where it was important that the code they wrote ran efficiently.

And the makers of the Bourne shell and bash wanted the opposite: They wanted to write programs/commands that could be executed immediately - on the command-line, in a terminal, you want to just write one line, one command, and have it execute. And they wanted scripts that you wrote to work anywhere where you had the shell interpreter/program installed.

Conclusion

So in short, you don't need a compiler for bash but you need one for C because those languages are converted into actual computer actions differently, and those different way of doing that were chosen because the languages had different goals.

Other technical/advanced details/notes

  1. You actually could create a C interpreter, or a bash compiler. There's nothing stopping that from being possible: it's just those languages were made for different purposes. It's often easier to just rewrite the program in another language, than to write a good interpreter or compiler for a complex programming language. Especially when those languages have a specific thing they were good at, and were designed with a certain way of working in the first place. C was designed to be compiled, so it's missing a lot of convenient shorthand that you'd want in an interactive shell, but it's very is very good for expressing very specific, low-level manipulation of data/memory and interacting with the operating system, which are tasks you often find yourself doing when you want to write efficiently compiled code. Meanwhile, bash is very good at executing other programs, redirecting files/file-descriptors, and working with strings of text, and it has convenient shorthand for those because those are tasks you often want to do in an interactive shell.

  2. More advanced detail: There are actually programming languages which are a mix of both types (they translate the source code "most of the way", so that they can do most of the interpretation/"thinking" once, and do only a little bit of the interpretation/"thinking" later on). Java, Python, and many other modern languages are actually such blends: they try to get you some of the portability and/or quick-development benefits of the interpreted languages, and some of the speed of compiled languages. There's a lot of possible ways to combine such approaches, and different languages do it differently. If you want to delve into this topic, you can read up on programming languages compiling into "bytecode" (which is kinda like compiling into your own made-up "machine language" that you can then interpret quickly and efficiently), and "JIT" (Just-in-Time compilation, where you compile and maybe even recompile the program as you interpret or run it).

  3. You asked about the execute bit: actually, the executable bit is only there to tell the operating system that that file is allowed to be executed. I suspect that the only reason bash scripts work for you without the execute permission is actually because you are running them from inside a bash shell. Normally, the operating system, when asked to execute a file without the execute bit set, will just return an error. But some shells like bash will see that error, and take it upon themselves to run the file anyway, by basically emulating the steps the operating system would normally take (look up the "#!" line at the start of the file, and try to execute that program to interpret the file, with a default of either itself or /bin/sh if there is no "#!" line).

  4. Sometimes a compiler is already installed on your system, and sometimes IDEs come with their own compiler and/or run the compilation for you. This might make a compiled language "feel" like a non-compiled language to use, but the technical difference is still there.

  5. A "compiled" language doesn't necessarily get compiled into machine code, and the whole compiling this is a topic in itself. Basically, the term is used broadly: it can actually refer to a few things. In one specific sense, a "compiler" is just a translator from one language (typically a "higher-level" language easier for humans to use) into another language (typically a "lower-level" language that's easier for computers to use - sometimes, but actually not very often, this is machine code). Also, sometimes when people say "compiler" they're really talking about multiple programs working together (for a typical C compiler, it's actually four programs: the "pre-processor", the compiler itself, the "assembler", and the "linker").

10
  • I don't understand why this question was downvoted. It's an impressively comprehensive answer to a question that is broad and not very clear. – Anthony Geoghegan Dec 15 '16 at 10:52
  • Compiler translates one language to another. It doesn't have to compile to machine language. You can have bytecode compiler. Java to ASM compiler, etc. The makers of C didn't want most power, they wanted the language that would fit their needs. C could be interpreted to some extent. Bash could be compiled - there's shc. Whether or not language is compiled/interpreted - most of the time - depends on tools you use, not the language itself, although some conventions are followed. – MatthewRock Dec 15 '16 at 11:39
  • @MatthewRock I've added a technical note to address the compiling into not-necessarily-machine-language thing. I feel my 1st technical note already covers the "C could be interpreted... Bash could be compiled" thing. I have an idea for how to address the "makers of C didn't want most power" issue, although I think it's pretty clear they designed the language to be efficient on the hardware they were using at the time (the null-byte-as-string-terminator thing was in part due to the fact that doing that let you use one less register when iterating on a string on those, after all). – mtraceur Dec 15 '16 at 12:13
  • @MatthewRock (cont) I think it's fair to say that the makers of C didn't exactly want power, but they did want to abstract OS-writing work, which especially in those days was something where code efficiency was valuable. And it was work that they would've otherwise been done in assembler. So they made a language that correlated closely to the machine code used by the PDP machines for which they were writing their code, which at least as a side-effect, if not as a notable design goal, lent itself to efficiency on that platform, even with naive non-optimizing compilers. – mtraceur Dec 15 '16 at 12:23
  • They wouldn't do it in assembler. They had B. Pitfalls everywhere, and question isn't on topic in here. Overally the answer would probably answer the question, but there are still details that aren't accurate. – MatthewRock Dec 15 '16 at 12:35
6

Consider the following program:

2 Mars Bars
2 Milks
1 Bread
1 Corn Flakes

In the bash way, you wander around the store looking for mars bars, finally locate them, then wander around looking for milk etc. This works because you are a running a complex program called "Experienced shopper" that can recognize a bread when you see one and all the other complexities of shopping. bash is a fairly complex program.

Alternatively, you can hand your shopping list to a shopping compiler. The compiler thinks for a while and gives you a new list. This list is LONG, but consists of much simpler instructions:

... lots of instructions on how to get to the store, get a shopping cart etc.
move west one aisle.
move north two sections.
move hand to shelf three.
grab object.
move hand to shopping cart.
release object.
... and so on and so forth.

As you can see, the compiler knows exactly where everything is in the shop so the whole "looking for things" phase is not needed.

This is a program in its own right and does not need "Experienced shopper" to execute. All it needs is a human with "Basic human operating system."

Returning to computer programs: bash is "Experienced shopper" and can take a script and just do it without compiling anything. A C compiler produces a stand-alone program that no longer needs any help to run.

Both interpreters and compilers have their advantages and disadvantages.

1
  • 3
    Nice analogy... also explains why you can use the same shopping list but not the same machine code on a different architecture (sic!) -- all the stuff is in different locations etc. You may need a different "experienced shopper" though who knows the different supermarket. – Peter - Reinstate Monica Dec 15 '16 at 11:04
5

Programming/scripting languages can be compiled or interpreted.

Compiled executables are always faster and many errors can be detected before execution.

Interpreted languages are typically simpler to write and to adapt being less strict than compiled languages, and don't require compiling which makes them easier to distribute.

9
  • 1
    In bash script also it gives an error when executed. but we don't compile it. – AReddy Dec 15 '16 at 5:45
  • 30
    always is a dangerous word… – Radon Rosborough Dec 15 '16 at 5:46
  • 3
    Optimised Compiled CPython is often slower than optimised asm.js (a subset of JavaScript). Therefore there is an example of it not being faster, and therefore it is not "always" faster. However, it is usually much, much faster. – wizzwizz4 Dec 15 '16 at 7:38
  • 2
    This does not answer the question. – mathreadler Dec 15 '16 at 8:39
  • 3
    always faster is a bold claim. But this go too much in deep on compiler and interpreter theory (and definition). – Giacomo Catenazzi Dec 15 '16 at 11:22
3

Imagine that English is not your native language (that might be quite easy for you if English is not your native language).

There are 3 ways you might read this:

  1. (Interpreted) As you read, translate each word every time you see it
  2. (Optimised-Interpreted) Find common phrases (such as "your native language"), translate them and write them down. Then, translate each word - except the phrases that you've already translated
  3. (Compiled) Ask someone else to translate the whole answer

Computers have a "native language" of sorts - a combination of instructions that the processor understands, and instructions that the operating system (e.g. Windows, Linux, OSX etc) understand. This language is not readable by humans.

Scripting languages, such as Bash, usually fall into categories 1 and 2. They take a line at a time, translate that line and run it, then move on to the next line. On Mac and Linux, quite a few different interpreters are installed by default for different languages, such as Bash, Python and Perl. On Windows, you have to install these yourself.

Many scripting languages do a little pre-processing - try to speed up the execution by compiling chunks of code that will be run often or that would otherwise slow down the application. Some terms you might hear about include Ahead-of-time (AOT) or Just-in-time (JIT) compilation.

Lastly, compiled languages - like C - translate the whole program before you can run them. This has the advantage that the translation can be done on a different machine to the execution, so when you give the program to the user, while there may still be bugs, several types of errors can already be cleaned up. Just like if you gave this to your translator, and I mention how the garboola mizene resplunks, that might look like valid English to you but the translator can tell you that I'm talking nonsense. When you run a compiled program, it does not need an interpreter - it is already in the computer's native language

There is one downside to compiled languages however: I mentioned that computers have a native language, composed of features from the hardware and the operating system - well, if you compile your program on Windows, you won't expect the compiled program to run on a Mac. Some languages get around this by compiling to a kind of half-way language - a bit like Pidgin English - that way, you get the benefits of a compiled language, as well as a small speed increase, but it does mean you need to bundle an interpreter with your code (or use one that is already installed).

Lastly, your IDE was probably compiling your files for you, and could tell you about errors before you ran the code. Sometimes, this error checking can be more in-depth than the compiler will do. A compiler will often only check as much as it needs to so that it can produce sensible native code. An IDE will often run a few extra checks and can tell you, for example, if you've defined a variable twice, or if you've imported something that you haven't used.

5
  • This answer is great, but I think the dynamic compilation used by the Perl "interpreter" is distinct from what is normally called "JIT", so it's probably best to avoid that term. JIT is commonly used to refer to just-in-time compilation from an already compiled byte code to the target machine code, e.g. JVM, .Net CLR. – IMSoP Dec 15 '16 at 10:28
  • @IMSoP Yeah, languages that byte-compile, and languages that JIT compile from byte-code, are indeed different things. I think it might be worth mentioning (I stuck a brief reference to it in the footnotes of my answer) but the general idea that JIT falls into that I think is worth having is that it's possible to be somewhere in the middle between "compiled" and "interpreted", i.e. partially compiled, and partially interpreted. That said, I'm unsure if that's more valuable or confusing/distracting for someone who doesn't yet understand the distinction between compiled/interpreted, like the OP. – mtraceur Dec 15 '16 at 10:45
  • @mtraceur To be honest, even I get a bit lost in the distinction between the models of, say, PHP 7, Perl 5, .Net, and Java. Perhaps the best summary for a beginner is "there are various ways of mixing compilation and interpretation, including using an intermediate representation, and compiling or re-compiling chunks as the program is run". – IMSoP Dec 15 '16 at 10:58
  • @IMSoP I agree. That's sorta the approach I took in my answer, and this comment of yours gave me an idea for how to refine that further. So thank you. – mtraceur Dec 15 '16 at 11:13
  • @IMSoP JIT doesn't only happen on bytecode - for example, node.js has some JIT features. But I agree - I lumped them all under the banner "JIT" for simplicity, as this felt like a beginner question - I'm editing it to use a simpler term. – user208769 Dec 15 '16 at 13:38
1

Lots of people are talking about interpretation vs compilation but I think this is can be a bit misleading if you look closely at it since some interpreted languages are actually compiled into an intermediate bytecode before execution.

In the end, the real reason why C programs need to be compiled to executable format is that the computer needs to do a lot of work to transform the code in a C source file into something it can run so it makes sense to save the product of all that work into an executable file so you don't need to do it again every time you want to run your program.

On the other hand, the Shell interpreter needs to do very little work to convert a shell script into "machine operations". Basically only needs to read the script line by line, split it on whitespace, set up some file redirections and pipelines and then do a fork+exec. Since the overhead of parsing and processing the textual input of a shell script is very small compared with the time it takes to launch the processes in the shell script, it would be overkill to compile the shell scripts to an intermediate machine format instead of just interpreting the source code directly.

2
  • +1, though I do find myself wondering if this isn't "putting the cart before the horse": perhaps the original shells were initially intended to be interactively usable and thus low-overhead enough to not bother compiling, and the relative simplicity was just a design decision based on that? – mtraceur Dec 17 '16 at 5:52
  • Yes, that is another way to look at this question :) – hugomg Dec 17 '16 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.