8

I mistakenly have source .bash_profile in the bashrc file and vice versa. Now when I tried to ssh into the machine (ec2), it will stuck at loading bash and get connection closed in a second. Is there a way I could fix it? Could I mount the disk to another ec2 instance to fix the bash files?


Update 1: I tried the following:

%  ssh -i "my-pem.pem" -t ubuntu@<server_address>.amazonaws.com "/bin/bash --noprofile --norc"
Connection to <server_address>.amazonaws.com closed.

Nothing else showed up. Do you have any idea on what was going wrong?

For sanity check, if I do ssh -i "my-pem.pem"ubuntu@<server_address>.amazonaws.com, the message will be

...

28 packages can be updated.
0 of these updates are security updates.
To see these additional updates run: apt list --upgradable

New release '20.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.


Last login: Mon Feb 22 23:17:41 2021 from ip
Connection to <server_address>.amazonaws.com closed.

Solution

Just ssh onto the machine and immediately do Ctrlc and fix the bash files.

1
  • 11
    You should add your solution as an answer, not as part of the OP. This might help someone out in the future. Feb 23 at 3:08
6

[update]

This interesting case deserves a better answer (which I'm borrowing from this answer on superuser).

My original answer below depends on the fact that ssh user@server command is supposed to execute command in a non-interactive shell, thus keeping the fault .bashrc from being executed.

The problem lies with the remote shell. If it is Bash, then there is a special features that makes it execute those rc files anyway:

(man bash) Bash attempts to determine when it is being run with its standard input connected to a network connection, as when executed by the remote shell daemon, usually rshd, or the secure shell daemon sshd. If bash determines it is being run in this fashion, it reads and executes commands from ~/.bashrc, if that file exists and is readable. It will not do this if invoked as sh. The --norc option may be used to inhibit this behavior, and the --rcfile option may be used to force another file to be read, but neither rshd nor sshd generally invoke the shell with those options or allow them to be specified.

That means that if your remote shell is Bash, then even if you run ssh user@server 'rm .bashrc', your .bashrc will be still executed first and your never-ending loop will prevent the rm command from being executed.

Note that scp won't save you since it is also a command executed internally by sshd by calling sh -c scp ....

The only exception is SFTP. sshd has an SFTP server built-in. If sshd is configured to use this internal SFTP server instead of an external one, then the server won't be started with bash -c ....

So, if your remote default shell is Bash, your only option is to use the "Download-edit-upload" method below, not with scp but with sftp.


[original answer]

Depending on your installation, here are several options:

Edit your .bashrc file:

ssh -t user@server 'vi ~/.bashrc'

Run bash without your .bashrc file:

ssh -t user@server '/bin/bash --noprofile --norc'

Run another shell:

ssh -t user@server /bin/sh

(sh is an example; you may try any other shell that is available on your server, like csh or fish)

Download-edit-upload you faulty .bashrc:

$ scp user@server:~/.bashrc /tmp/bashrc
$ vi /tmp/bashrc
$ scp /tmp/bashrc user@server:~/.bashrc
14
  • All of these returned a connection closed message like the one in the updated question. Any thoughts on what was going on are much appreciated.
    – John M.
    Feb 22 at 23:21
  • @JohnM. I have added some ideas but your feedback makes me wonder if there is another problem than the one described in your question (typically you might be denied access because of two many ssh connections in a short period of time). I'm also wondering if your ssh connection allows you to execute anything else than the shell that was configured there (in the past, did you use ssh u@server command or scp?)
    – xhienne
    Feb 22 at 23:32
  • 1
    I'm not sure about the updated answer. I can't reproduce it on an OpenSSH server I have at hand: adding echo X > test to the server's .bashrc and then trying ssh user@remote 'mv .bashrc .bashrc.bak' works alright, no test file pops up. I think there might be some misreading of that part of the manual.
    – Quasímodo
    Feb 23 at 12:47
  • 1
    Dropping by to say that I think I've found the missing link in case you want to update your answer: Most .bashrcs switches off if the shell is non-interactive. Do you also verify this in your server?
    – Quasímodo
    Feb 25 at 21:43
  • 1
    @Quasímodo Well spotted! On my Debian server, I have [ -z "$PS1" ] && return in my /etc/bash.bashrc and something similar in /etc/profile. That indeed explains why I'm not affected by bash's behavior. Whereas I should modify my answer, I lack the time right now, but I'm not sure there is anything wrong in the updated part: this explains the OP's issue and provides an acceptable solution. If you see anything to change, don't hesitate to edit it. I'll come back tomorrow and review it more thoroughly anyway.
    – xhienne
    Feb 25 at 21:59
6

When you run ssh, by default it runs your shell in interactive mode, and in interactive mode it processes your .bashrc. However, if you specify a command to run, it will invoke the shell in non-interactive mode (so it won't process your .bashrc).

You can use that to launch an interactive shell, but specifying command-line arguments to that shell to instruct it not to process your .bashrc:

$ ssh -t user@host "/bin/bash --noprofile --norc"
bash-5.0$

If the problem is in your .bashrc file, this may help you to log in. However, if the problem is in a bash-related profile file (e.g., .bash_profile), then you may still get hung up in the first non-interactive shell.

4
  • 1
    Trouble is, it passes your command to a shell on the far end, rather than directly invoking it. I don't think there's a way to avoid that. Feb 23 at 17:35
  • 1
    @TobySpeight However, it seems that when called like this, the bash called implicitly (the bash that's calling bash here) is neither a login shell nor interactive, so neither the profile nor rc files are sourced. --noprofile is redundant for the child bash.
    – JoL
    Feb 23 at 20:35
  • 1
    I may be nitpicking word-choice but, "When you run ssh, by default it runs your shell." -- That's always the case, I believe. It's not a matter of defaults. "You can, however, specify a different command for it to run." -- You can specify a command for your shell to run, not in-place of the shell.
    – JoL
    Feb 23 at 20:55
  • @JoL yup, you're right. I mistakenly thought that it exec-ed the given command, but as you point out, it execs a shell that runs the given command. Feb 25 at 1:26
6

In general, ~/.bashrc is only sourced by interactive shells, and ~/.bash_profile by login shells (Difference between login shell and non-login shell?). If you only send commands via Ssh, you get neither an interactive nor a login shell, so just renaming one of the files—say, ssh user@host 'mv .bashrc .bashrc.bak'—would work.

But there is an exception, as Xhienne points out: If invoked by the SSH daemon, Bash does source Bashrc.

However, Xhienne and I failed to reproduce that. My test was to append touch testfile to the server's Bashrc and then try a dummy command to see what happened: ssh Quasímodo@server 'echo x'. Result: No testfile was created. Only later did I find out that my Bashrc contains as its first lines

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

I.e., if the shell is not interactive, stop reading Bashrc. This nifty trick is actually present in most distributions as thoroughly discussed in Why does bashrc check whether the current shell is interactive?

Conclusion

Andy's, Xhienne's and my original answers, that amounted to running a command through SSH, will

  • Succeed if that fragment is present in the user's remote Bashrc.
  • Fail otherwise, in which case you can find some options at Server Fault, among which Ctrl-C.
4
  • That won't work, because the ssh daemon actually uses your shell (in interactive mode) to run the passed command. Feb 23 at 17:36
  • One would also need to create an account with Bash as login shell! The question says bashrc though - did that really mean ~/.bashrc rather than /etc/bashrc? It's absolutely unclear what should be tested. My comment was based on the documentation quoted in xhienne's answer. Feb 24 at 12:07
  • 1
    @TobySpeight Updated answer: I have tracked down the behavior to some lines in Bashrc itself.
    – Quasímodo
    Feb 25 at 21:14
  • 1
    That's great, when all the different approaches come together. And I was wrong to say "(in interactive mode)" because Bash doesn't set -i in that case, even though it sources the rc file. Feb 26 at 7:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.