Why is bash history substitution still enabled by default? [closed]

Question

Does anybody know why bash still has history substitution enabled by default? My .bashrc has included set +H for many many years but some other people are still getting bitten by this feature.

Given that pretty much everybody are using terminals with copy-paste features and bash compiled with readline library and history substitution is enabled by default only in interactive shells, is there really any reason to have the feature at all? None of the existing scripts would be broken even if this was disabled by default for all shells.

Try this if you do not know why history substitution is broken:

$ set +H # disable feature history substitution
$ echo "WTF???!?!!?"
WTF???!?!!?
$ set -H # enable feature history substitution
$ echo "WTF???!?!!?"
echo WTF???echo WTF???!?!!?
WTF???echo WTF???!?!!?

(Clearly the feature has major issues if it's disabled by default for all scripting and a feature exists to verify the results before executing: shopt -s histverify.)

See also:

• Why does the exclamation mark `!` sometimes upset bash?
• Why does Bash history not record this command?
• How to echo a bang!
• Bash: History expansion inside single quotes after a double quote inside the same line

Answer 1

If you are already familiar with bash, then dealing with history substitution patterns is not much more likely to bite you than handling any other characters that are special to this shell. However, if one is unfamiliar with the shell or just never used its history substitution features, it will obviously be a surprise when seemingly innocuous unquoted or double-quoted strings triggers it.

In an interactive shell with history substitutions enabled, the ! character is special in pretty much the same way as the $ character is special, i.e. everywhere unless escaped with \ or in single-quoted strings.

As opposed to $ through, history substitutions do not expand in here-documents, and since they are line-oriented, they additionally will happen on lines where the substitution falls within an unquoted context or a double quoted context (in that line when scanned separately). See this bug report for more info.

History substitution is disabled in non-interactive shells (scripts) because the shell's command history capability is not needed there, not because the feature has "major issues". In a script, saving every command to $HISTFILE makes no sense, and history substitution likewise is not something you'd want to rely on in a script.

Whether or not it should be enabled by default or not in interactive shells can be debated (though I'm not entirely convinced that a debate here would matter much to the bash developers). You seem to think that most bash users are having problems with history expansions, but neither one of you and me know how common it is to use them.

Unix shells allow one to modify the shell's behaviour to fit one's personal needs and taste. If you want to turn off history substitutions for all your interactive shells, continue doing what you are doing with using set +H in your ~/.bashrc file, or lobby the bash developers to change the default (which, I believe, would upset and confuse more people than it would help).

Answer 2

History substitution is useful. Take for example

% make-me-a-sandwich
make-me-a-sandwich: Permission denied
% sudo !!
Ok.

Answer 3

Social/cultural inertia.

This question is in the how-humans-work problem space, so I'm going to answer from that angle, without putting forth any opinion about whether or not the feature ought to be on by default.

To start, to make sure you understand the other side, consider that the annoyance you feel about having to go out of your way to turn off the feature, is the annoyance they would feel if they had to go out of their way to turn on the feature.

Combine the above with the fact that enough bash users do use the feature, suggestions of removing it or turning it off by default are met with resistance from the people who are already comfortable with it being there by default.

Also, bash is the default shell for many people (not just in the default login or system shell sense, but in a psychological sense). If your reference frame for shell quoting is bash, if that's the shell you learned first, the fact that ! is a special shell character will feel natural and automatic to you (or at least, when you first learn it, it'll be just part of the way the shell is, just one quirk to accept among many).

And if you think about it, a lot of bash users probably encounter the history substitution syntax in a positive context: they read about it or someone shows it to them and they see the possible usefulness of it, when they're first learning bash.

It's only coming from the peripheral world of other Bourne-like shells, that you'd be bitten by ! being special and thus be inclined to view it negatively: Because if you're used to shells where you never had the feature, then your first exposure to it will be when it screws you when you're trying to get something done in a hurry.

TL;DR: Most users probably don't strongly care what the default is either way, some users like the feature and have the strong advantage of it already being that way, and there hasn't been enough people actively advocating against the feature to overcome that.

Answer 4

Why is bash history substitution still enabled by default?

Because many people use it, and people using an interactive bash shell probably should know the rules to avoid problems and will generally find it helps more than it hurts.

My .bashrc has included set +H for many many years but some other people are still getting bitten by this feature.

So it's a feature you don't use, that doesn't mean a majority of users don't use it. You can petition for the default to be changed, but you have to look at A) the portion of people that care, and B) the ratio of people that prefer it your way. The people that care and don't like it probably already have it disabled. Changing it would help when they get an account on a new computer. People that care and do like it would have to update their settings on every computer they use and every account they get in the future.

Given that pretty much everybody are using terminals with copy-paste features

A clunkier option in my opinion...

is there really any reason to have the feature at all? None of the existing scripts would be broken even if this was disabled by default for all shells.

Yes, people find it useful. What's the use in having a remote control given that you used to be able to just get up and change the channel?

(Clearly the feature has major issues if it's disabled by default for all scripting and a feature exists to verify the results before executing: shopt -s histverify.)

History doesn't really make sense in scripts, but more importantly it could cause security issues. In your case you could avoid the problem by using single quotes. I don't remember this ever causing an issue for me, so I don't know how you can say it has 'major issues'. Has this caused an actual problem for you, or were you annoyed at having to set your defaults on a new computer?

I don't see how it is any different than having to escape or use single quotes in this if you actually want to get some money:

$ echo "Give me $50 or the cat gets it"
Give me $0 or the cat gets it