I do agree, it's an annoying default behaviour (and it seems to be unique to
readline). In my own experience at least, if I move away from a line I've modified, it's generally because I have given up on those modifications and would rather have them discarded than lose the original line!
Leaving those modifications there can be useful if the reason you've given up on modifying that line is because you had to do something else first, and that may be the original justification for doing it that way. Still, overriding the original history entry is bad IMO as you're losing evidences of what you've done before.
The most annoying cases are where you end up clearing the whole line to start a new command fresh, then go to a different line which you accept and realise later on that one history entry seems to have gone.
Also, even if you intended to keep those modifications, if you want to go back to that modified line, you need to search for it again.
The behaviour of
zsh is more useful IMO. You can modify the whole history as long as you don't accept a line and got back and forth between your modified lines, but once you accept a new line, it is added to the history but the old history lines are preserved (while pressing Ctrl+C cancels all modifications without adding a new line to the history).
And if you wanted to save the modifications you had done to different lines, you'd store the line on a queue with Alt+Q (the top of which is automatically pulled at the next prompt), or store it in the killring with Ctrl+U.
readline, you can change the behaviour with
bind 'set revert-all-at-newline on'
set revert-all-at-newline on in
~/.inputrc so it applies to all applications using readline).
Then, you'll get a similar behaviour as in
yash, that is same as
zsh but without the hold queue (but you can still use the killring (pulled with Ctrl+Y, Alt+Y)).