2

so I have this problem, I can't login to my server with SSH. See:

ssh -v myuser@myserver
debug1: Authentication succeeded (password).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Last login: Sun Jul  5 20:10:54 2015 from x.x.x.x
-bash: KSH_VERSION: unbound variable
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
Connection to x.x.x.x closed.
Transferred: sent 2264, received 2800 bytes, in 0.2 seconds
Bytes per second: sent 13241.8, received 16376.8
debug1: Exit status 1

The problem is the "-bash: KSH_VERSION: unbound variable", this kills the connection I think. I know why is it happening…I was working on a script placed in /etc/profile.d, and this script has "set -euo pipefail" (ouch) in it... The "o" option is "nounset" and that exaplains the "unbound variable"... When the script sets the options, it make the vim.sh there failing on the unbound variable.

Now, how get around this? I can't send just a command there over ssh, to remove the file for example, I can't login as another user, I can't use another shell, the "--noprofile" and "--norc" options for bash doesn't work…no trick I tried worked…

Any ideas, pretty please?

closed as off-topic by slm Jan 9 at 4:53

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions describing a problem that can't be reproduced and seemingly went away on its own (or went away when a typo was fixed) are off-topic as they are unlikely to help future readers." – slm
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Does this work? ssh myuser@myserver hostname – Cyrus Jul 5 '15 at 20:29
  • nope, it doesn't :( same problem... – stibi Jul 5 '15 at 20:30
  • The man page for ssh (linux.die.net/man/1/ssh) documents a -N option to not execute a remote command. I don't know if there's a way to pass that to scp so you can update the file, but you might be able to forward ports to the remote machine and get access through them if for example, normal FTP is allowed from localhost. – Andrew Henle Jul 5 '15 at 22:46
  • If this is running Debian (or a derivative), you can try using /bin/dash as the shell instead. Not sure if it will work... Edit: Never mind, it reads /etc/profile. – saiarcot895 Jul 6 '15 at 23:09
2

You have about 1/3 chance (at least I get 1/3) to interrupt (hit Ctrl+C) the login process before /etc/profile is sourced:

$ ssh -v localhost
OpenSSH_6.6.1, OpenSSL 1.0.1e-fips 11 Feb 2013
........
Authenticated to localhost ([::1]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Last login: Tue Jul  7 19:46:42 2015 from localhost
^C-bash-4.2$ 

Edit to clarify a few things:

When sshd opens a session on the server, it does so by reading the user shell (that is /bin/bash in this case).

It does not matter whatever option is given on bash or ssh command to run (including sftp). That ssh command (including sftp) or bash option is passed to a bash instance after a session is opened. You have no control on what that instance does. For example:

client$ ssh localhost bash --noprofile --norc

strace output on the server:

[pid 60047] execve("/bin/bash", ["bash", "-c", "bash --noprofile --norc"], [/* 16 vars */]) = 0
[pid 60047] execve("/usr/bin/bash", ["bash", "--noprofile", "--norc"], [/* 28 vars */]) = 0

As you can see the ssh command ("bash --noprofile --norc") is passed on the first execve as an argument to "bash" with the "-c" option.

That first bash instance that you have no control on with read ~/.bashrc which in turn will read /etc/bashrc which will load the scripts under /etc/profile.d/ (at least on a Red Hat system - I don't know about others).

[pid 60047] open("/home/user/.bashrc", O_RDONLY) = 3
[pid 60047] open("/etc/bashrc", O_RDONLY) = 3

The only way I can think of is the above, been quick enough to abort the login process before the scripts are loaded (but not to quick as to abort the session). - You might want to increase the load on the system (if it's possible) to give you more time during the login process.

ps. if you find the answer useful please mark it

  • nice, this is cool – stibi Jul 7 '15 at 16:52
  • Tip: See @st0ne's supplemental contribution to this answer for a great way to increase your chances of Ctrl+C working. – Mark Thomson Dec 30 '18 at 23:12
1

I had the same problem and finally found a solution.

As @nkms mentioned:

You have about 1/3 chance (at least I get 1/3) to interrupt (hit Ctrl+C) the login process before /etc/profile is sourced

To raise you chances i've used the following loop:

$ (trap '' INT; while true; do ssh myuser@myserver; done)

The script starts subshell '(' and deactivates the ctr-c command for that subshell (with the trap command). Then it loops over the ssh command. If you continuously press ctrl-c or hold down ctrl-c, you raise you chances to hit the right moment.

... works like a charm!

  • 1
    It worked beautifully for me. Of all the things I've ever learned from this site, this one right here had the most "wow, my butt just got saved!" factor of any of them. Thank you for your clever solution. – Mark Thomson Dec 30 '18 at 23:06
  • @MarkThomson: THX! – st0ne Jan 4 at 9:38
0

Does sftp also fail ? If doesn't, you could make use of that to place a good /etc/profile.d file there.

  • good idea, but doesn't work either, it fail with "Connection closed" – stibi Jul 5 '15 at 20:37
  • Shucks. Is logging on via the console an option? Or use something other than ssh to logon (telnet, rsh, ftp by any chance?) Or if you've a backup solution like NetBackup, use that to restore a "good" version of the file? – steve Jul 5 '15 at 20:44
  • For a console, I'll have to ask the provider of the virtual machine. I'll try the other options, but doubt it... – stibi Jul 5 '15 at 20:46
  • 3
    What about scp? It works at a lower level than sftp. Something like scp file root@machine:/etc/profile.d. – Satō Katsura Jul 5 '15 at 20:47
  • scp seems to be also invoking bash process, because it fail in the same manner as all the other attempts… – stibi Jul 5 '15 at 22:15
0

I have two problems with your question.

ps -A|grep bash|wc -l

and the same command in

ssh localhost 'csh -l'

give the same number.

So it does not seem that a bash is run before 'csh -l'.

On the other hand csh does not use /etc/profile.

strace -eopen csh -l

does not show any entry of /etc/profile*.

So I am not sure the problem you have is in /etc/profile*.

Am I missing something?

  • 1
    the problem is definitelly there, I'm not 100% sure about the root cause, but I'll dig it out. Now I have the server ready and fixed again, provider's support helped me. Once the bloody script was gone, SSH worked again. – stibi Jul 7 '15 at 16:45
  • So just deleting a script in /etc/profile.d/ solved the problem? – Jodka Lemon Jul 7 '15 at 16:47
  • Yes. Without the script and its "set -euo", the other scripts in profile.d were invoked without problems, even these with unbounded variables…nothing what would cause exiting these "wrong" scripts with non zero status code and thus taking the ssh connection down with them. That's my current exaplanation :) – stibi Jul 7 '15 at 16:51
  • Jodka: sshd runs your configured shell with -c 'args from client' which in your case is bash -c 'csh -l'. When bash is given a single command in -c, it doesn't run it as a forked child process like usual, instead bash exec's it, so the process that sshd created now is csh. Try ssh localhost 'csh -l;echo done' to see the bash. – dave_thompson_085 Jan 9 at 5:05
0

This is unlikely to work, but you could try sending a value for KSH_VERSION when you make the ssh connection:

KSH_VERSION=x ssh -v -o 'SendEnv=KSH_VERSION' myuser@myserver

The idea is to set the environment variable to something, so that bash won't complain that it's unset. I say it's unlikely to work because the default SSH server configuration wouldn't permit you to set this particular variable. This would only work if someone had reason to make that setting more lenient on this particular server.

  • you are correct in all points :) doesn't work, because of the reasons you exaplained – stibi Jul 7 '15 at 16:44
0

I ended up with the same problem, but was installing something else and fixed it once I removed the below:

/etc/sysctl.conf
fs.file-max=100000

Then

/etc/security/limits.conf:
*               soft    nofile          1000000
*               hard    nofile          1000000

Lastly

/etc/pam.d/common-session:
session required pam_limits.so
-1

Try ssh -t username@hostname /bin/sh --noprofile.

  • Tried that, doesn't work, I think it is because there is a main bash process created which is parent to the one started from the ssh command – stibi Jul 5 '15 at 20:52
  • 1
    And a different shell? – Jodka Lemon Jul 6 '15 at 16:03
  • ksh, csh…both invoking /etc/profile, no go – stibi Jul 7 '15 at 6:49
  • What does csh -f? – Jodka Lemon Jul 7 '15 at 9:49

Not the answer you're looking for? Browse other questions tagged or ask your own question.