I wrote a bash script, and I executed it without compiling it first. It worked perfectly. It can work with or without permissions, but when it comes to C programs, we need to compile the source code. Why?
It means that shell scripts aren't compiled, they're interpreted: the shell interprets scripts one command at a time, and figures out every time how to execute each command. That makes sense for shell scripts since they spend most of their time running other programs anyway.
C programs on the other hand are usually compiled: before they can be run, a compiler converts them to machine code in their entirety, once and for all. There have been C interpreters in the past (such as HiSoft's C interpreter on the Atari ST) but they were very unusual. Nowadays C compilers are very fast; TCC is so fast you can use it to create "C scripts", with a
#!/usr/bin/tcc -run shebang, so you can create C programs which run in the same way as shell scripts (from the users' perspective).
Some languages commonly have both an interpreter and a compiler: BASIC is one example that springs to mind.
You can also find so-called shell script compilers but the ones I've seen are just obfuscating wrappers: they still use a shell to actually interpret the script. As mtraceur points out though a proper shell script compiler would certainly be possible, just not very interesting.
Another way of thinking about this is to consider that a shell's script interpreting capability is an extension of its command-line handling capability, which naturally leads to an interpreted approach. C on the other hand was designed to produce stand-alone binaries; this leads to a compiled approach. Languages which are usually compiled do tend to sprout interpreters too, or at least command-line-parsers (known as REPLs, read-eval-print loops; a shell is itself a REPL).
It all comes down to the technical difference between how the program you can read/write as a human gets converted into the machine instructions your computer understands - and the different advantages and disadvantages of each method is the reason why some languages are written to need compilers, and some are written to to be interpreted.
First, the technical difference
(Note: I'm simplifying a lot here for the sake of addressing the question. For a more in-depth understanding, technical notes at the bottom of my answer elaborate/refine some of the simplifications here, and the comments on this answer have some useful clarifications and discussion as well..)
There are basically two general categories of programming languages:
- Another program (the "compiler") reads your program, determines what steps your code says to do, and then writes a new program in machine code (the "language" your computer itself understands) that does those steps.
- Another program (the "interpreter") reads your program, determines what steps your code says to do, and then does those steps itself. No new program is created.
C is in the first category (the C compiler translates the C language into your computer's machine code: the machine code is saved into a file, and then when you run that machine code, it does what you want).
bash is in the second category (the bash interpreter reads the bash language and the bash interpreter does what you want: so there's no "compiler module" per se, the interpreter does the interpreting and the executing, whereas a compiler does reading and translating).
You might have already noticed what this means:
With C, you do the "interpret" step once, then whenever you need to run the program, you just tell your computer to execute the machine code - your computer can just run it directly without having to do any extra "thinking".
With bash, you have to do the "interpret" step every time you run the program - your computer is running the bash interpreter, and the bash interpreter does extra "thinking" to figure out what it needs to do for each command, every time.
So C programs take more CPU, memory, and time to prepare (the compiling step) but less time and work to run. bash programs take less CPU, memory, and time to prepare, but more time and work to run. You probably don't notice these differences most of the time because computers are very fast nowadays, but it does make a difference, and that difference adds up when you need to run big or complicated programs, or a lot of little programs.
Also, because C programs are converted into machine code (the "native language") of the computer, you can't take a program and copy it onto another computer with a different machine code (for example, Intel 64-bit onto Intel 32-bit, or from Intel to ARM or MIPS or whatever). You have to spend the time to compile it for that other machine language again. But a bash program can just be moved over to another computer that has the bash interpreter installed, and it'll run just fine.
Now the why part of your question
The makers of C were writing an operating system and other programs on hardware from several decades ago, that was rather limited by modern standards. For various reasons, converting the programs into the computer's machine code was the best way towards that goal for them at the time. Plus, they were doing the kind of work where it was important that the code they wrote ran efficiently.
And the makers of the Bourne shell and bash wanted the opposite: They wanted to write programs/commands that could be executed immediately - on the command-line, in a terminal, you want to just write one line, one command, and have it execute. And they wanted scripts that you wrote to work anywhere where you had the shell interpreter/program installed.
So in short, you don't need a compiler for bash but you need one for C because those languages are converted into actual computer actions differently, and those different way of doing that were chosen because the languages had different goals.
Other technical/advanced details/notes
You actually could create a C interpreter, or a bash compiler. There's nothing stopping that from being possible: it's just those languages were made for different purposes. It's often easier to just rewrite the program in another language, than to write a good interpreter or compiler for a complex programming language. Especially when those languages have a specific thing they were good at, and were designed with a certain way of working in the first place. C was designed to be compiled, so it's missing a lot of convenient shorthand that you'd want in an interactive shell, but it's very is very good for expressing very specific, low-level manipulation of data/memory and interacting with the operating system, which are tasks you often find yourself doing when you want to write efficiently compiled code. Meanwhile, bash is very good at executing other programs, redirecting files/file-descriptors, and working with strings of text, and it has convenient shorthand for those because those are tasks you often want to do in an interactive shell.
More advanced detail: There are actually programming languages which are a mix of both types (they translate the source code "most of the way", so that they can do most of the interpretation/"thinking" once, and do only a little bit of the interpretation/"thinking" later on). Java, Python, and many other modern languages are actually such blends: they try to get you some of the portability and/or quick-development benefits of the interpreted languages, and some of the speed of compiled languages. There's a lot of possible ways to combine such approaches, and different languages do it differently. If you want to delve into this topic, you can read up on programming languages compiling into "bytecode" (which is kinda like compiling into your own made-up "machine language" that you can then interpret quickly and efficiently), and "JIT" (Just-in-Time compilation, where you compile and maybe even recompile the program as you interpret or run it).
You asked about the execute bit: actually, the executable bit is only there to tell the operating system that that file is allowed to be executed. I suspect that the only reason bash scripts work for you without the execute permission is actually because you are running them from inside a bash shell. Normally, the operating system, when asked to execute a file without the execute bit set, will just return an error. But some shells like bash will see that error, and take it upon themselves to run the file anyway, by basically emulating the steps the operating system would normally take (look up the "#!" line at the start of the file, and try to execute that program to interpret the file, with a default of either itself or
/bin/shif there is no "#!" line).
Sometimes a compiler is already installed on your system, and sometimes IDEs come with their own compiler and/or run the compilation for you. This might make a compiled language "feel" like a non-compiled language to use, but the technical difference is still there.
A "compiled" language doesn't necessarily get compiled into machine code, and the whole compiling this is a topic in itself. Basically, the term is used broadly: it can actually refer to a few things. In one specific sense, a "compiler" is just a translator from one language (typically a "higher-level" language easier for humans to use) into another language (typically a "lower-level" language that's easier for computers to use - sometimes, but actually not very often, this is machine code). Also, sometimes when people say "compiler" they're really talking about multiple programs working together (for a typical C compiler, it's actually four programs: the "pre-processor", the compiler itself, the "assembler", and the "linker").
Consider the following program:
2 Mars Bars 2 Milks 1 Bread 1 Corn Flakes
bash way, you wander around the store looking for mars bars, finally locate them, then wander around looking for milk etc. This works because you are a running a complex program called "Experienced shopper" that can recognize a bread when you see one and all the other complexities of shopping.
bash is a fairly complex program.
Alternatively, you can hand your shopping list to a shopping compiler. The compiler thinks for a while and gives you a new list. This list is LONG, but consists of much simpler instructions:
... lots of instructions on how to get to the store, get a shopping cart etc. move west one aisle. move north two sections. move hand to shelf three. grab object. move hand to shopping cart. release object. ... and so on and so forth.
As you can see, the compiler knows exactly where everything is in the shop so the whole "looking for things" phase is not needed.
This is a program in its own right and does not need "Experienced shopper" to execute. All it needs is a human with "Basic human operating system."
Returning to computer programs:
bash is "Experienced shopper" and can take a script and just do it without compiling anything. A C compiler produces a stand-alone program that no longer needs any help to run.
Both interpreters and compilers have their advantages and disadvantages.
Programming/scripting languages can be compiled or interpreted.
Compiled executables are always faster and many errors can be detected before execution.
Interpreted languages are typically simpler to write and to adapt being less strict than compiled languages, and don't require compiling which makes them easier to distribute.
Imagine that English is not your native language (that might be quite easy for you if English is not your native language).
There are 3 ways you might read this:
- (Interpreted) As you read, translate each word every time you see it
- (Optimised-Interpreted) Find common phrases (such as "your native language"), translate them and write them down. Then, translate each word - except the phrases that you've already translated
- (Compiled) Ask someone else to translate the whole answer
Computers have a "native language" of sorts - a combination of instructions that the processor understands, and instructions that the operating system (e.g. Windows, Linux, OSX etc) understand. This language is not readable by humans.
Scripting languages, such as Bash, usually fall into categories 1 and 2. They take a line at a time, translate that line and run it, then move on to the next line. On Mac and Linux, quite a few different interpreters are installed by default for different languages, such as Bash, Python and Perl. On Windows, you have to install these yourself.
Many scripting languages do a little pre-processing - try to speed up the execution by compiling chunks of code that will be run often or that would otherwise slow down the application. Some terms you might hear about include Ahead-of-time (AOT) or Just-in-time (JIT) compilation.
Lastly, compiled languages - like C - translate the whole program before you can run them. This has the advantage that the translation can be done on a different machine to the execution, so when you give the program to the user, while there may still be bugs, several types of errors can already be cleaned up. Just like if you gave this to your translator, and I mention how the
garboola mizene resplunks, that might look like valid English to you but the translator can tell you that I'm talking nonsense. When you run a compiled program, it does not need an interpreter - it is already in the computer's native language
There is one downside to compiled languages however: I mentioned that computers have a native language, composed of features from the hardware and the operating system - well, if you compile your program on Windows, you won't expect the compiled program to run on a Mac. Some languages get around this by compiling to a kind of half-way language - a bit like Pidgin English - that way, you get the benefits of a compiled language, as well as a small speed increase, but it does mean you need to bundle an interpreter with your code (or use one that is already installed).
Lastly, your IDE was probably compiling your files for you, and could tell you about errors before you ran the code. Sometimes, this error checking can be more in-depth than the compiler will do. A compiler will often only check as much as it needs to so that it can produce sensible native code. An IDE will often run a few extra checks and can tell you, for example, if you've defined a variable twice, or if you've imported something that you haven't used.
Lots of people are talking about interpretation vs compilation but I think this is can be a bit misleading if you look closely at it since some interpreted languages are actually compiled into an intermediate bytecode before execution.
In the end, the real reason why C programs need to be compiled to executable format is that the computer needs to do a lot of work to transform the code in a C source file into something it can run so it makes sense to save the product of all that work into an executable file so you don't need to do it again every time you want to run your program.
On the other hand, the Shell interpreter needs to do very little work to convert a shell script into "machine operations". Basically only needs to read the script line by line, split it on whitespace, set up some file redirections and pipelines and then do a fork+exec. Since the overhead of parsing and processing the textual input of a shell script is very small compared with the time it takes to launch the processes in the shell script, it would be overkill to compile the shell scripts to an intermediate machine format instead of just interpreting the source code directly.