Modified keyboard layout under VirtualBox

Question

I'm running VirtualBox on Linux. (The virtualbox 4.3.10 package on Ubuntu 14.04 if it matters, but I'd prefer a version-agnostic solution.) I have a tweaked keyboard layout on the host (under X11), and apparently that causes VirtualBox to misinterpret some keys.

Specifically, the left “Windows” key (LWIN for XKB) is configured with xmodmap to send

keycode 133 = Hyper_L

In VirtualBox virtual machines the key is not recognized at all. I ran showkey on a Linux console to check, it doesn't react when I press this key.

How can I make LWIN send the right keycode in virtual machines? The trigger can be this specific physical key or the Hyper_L keysym. Of course other applications must not be affected; that xmodmap line is staying.

I use Sawfish as my window manager. A Sawfish-specific solution is ok, but I'm not sure it can help: it allows me to define per-window keybindings, but not for modifier keys alone. I want to run a Windows VM, so the press and release events for the key alone must be passed through to the VM (press of LWIN → pass a key press event to the VM), not just combinations like LWIN+key.

If the fact that the key is a modifier makes this too hard, I'm willing to fall back to making the CAPS (Caps Lock) key, which is bound to the Mode_switch keysym (AltGr) and no modifier, appear as LWIN to the VM.

Answer 1

You could try writing a bash script that you could use to boot up virtualbox and make it so that when you start the script it configures the LWIN back to default so that there are no problems relaying a keyboard code. Then you could use xmodmap on the virtual machine itself if you need it.

Should my solution not work I would recommend you take a look at the following two threads. The guy there had a similar problem and I think you may be able to use it. I think this may be a better solution compared to mine. Good Luck!

http://ubuntuforums.org/showthread.php?t=1291982

https://forums.virtualbox.org/viewtopic.php?f=7&t=23419

Answer 2

What I've been doing so far is to implement per-window keybindings in Sawfish — more or less. I define hooks that call xmodmap on a focus change. It's prone to race conditions, but in practice that hasn't been a problem. I switch the keysym for keycode 133 on entering and leaving windows of class VirtualBox (this applies to the manager as well as the VMs but it doesn't matter for what I'm doing).

(define (add-window-class-matcher class-regexp . assignments)
  "Declare properties for windows whose class matches CLASS-REGEXP.
The string matched against is the X class followed by a slash
followed by the X instance, e.g., \"XTerm/xterm\"."
  (apply add-window-matcher 'WM_CLASS class-regexp assignments))

(define (per-window-keymap-hook window index)
  (let ((xmodmap-expressions-property (window-get window 'xmodmap-expressions)))
    (if (symbolp xmodmap-expressions-property)
        (setq xmodmap-expressions-property
              (symbol-value xmodmap-expressions-property)))
    (when xmodmap-expressions-property
      (let ((expressions (nth index xmodmap-expressions-property))
            (args ()))
        (when expressions
          (while expressions
            (setq args (cons (car expressions) (cons "-e" args))
                  expressions (cdr expressions)))
          (apply call-process nil nil "xmodmap" (nreverse args)))))))
(define (per-window-keymap-in-hook window focus-mode)
  (per-window-keymap-hook window 0))
(define (per-window-keymap-out-hook window focus-mode)
  (per-window-keymap-hook window 1))
(add-hook 'focus-in-hook per-window-keymap-in-hook)
(add-hook 'focus-out-hook per-window-keymap-out-hook)

(define virtualbox-xmodmap-expressions
  '(("keycode 133 = Super_L")
    ("keycode 133 = Hyper_L")))
(add-window-class-matcher "^VirtualBox/"
  `(xmodmap-expressions . virtualbox-xmodmap-expressions))

---

One day I might switch my keyboard configuration to XKB and run kbdd.

Answer 3

VirtualBox passes keycodes to the VM, not the interpreted meaning of those keys. Your keycode mapping comes in at a higher level, telling your host X server what to interpret each keycode as meaning.

That said, Virtualbox does (or is that did?) use various heuristics to try to identify the keyboard you are using, and does some sort of mapping. Based on the tickets filed, I'd say it's a messy process, and generally involves the virtualbox people needing to know exactly what the keyboard involved is like.

This might set you in the right direction: https://forums.virtualbox.org/viewtopic.php?f=7&t=57120 . It looks like someone had a similar issue and was able to sort it out. It looks like they were doing xmodmap mapping within the guest, which would be different with windows, but it does look like they were able to get the previously inactive keys to work, so it suggests what you want is possible, and says something about your method of testing in a linux vm.