CapsLock as modifier key for application keymaps

Question

I am developer and currently I am using Ubuntu Linux with PyCharm IDE for development of website. In order to improve typing performance I consider change CapsLock to Hyper_L and assign it to mod3 using xmodmap. After this I want to map mod3 for 'i' key to 'Up'. So far what I did is:

xmodmap -e "keysym Caps_Lock = Hyper_L"

output is like this:

shift       Shift_L (0x32),  Shift_R (0x3e)
lock        Hyper_L (0x42)
control     Control_L (0x25),  Control_R (0x69)
mod1        Alt_L (0x40),  Alt_R (0x6c),  Meta_L (0xcd)
mod2        Num_Lock (0x4d)
mod3      
mod4        Super_L (0x85),  Super_R (0x86),  Super_L (0xce),  Hyper_L (0xcf)
mod5        ISO_Level3_Shift (0x5c),  Mode_switch (0xcb)

then I remove Hyper_L from mod4

xmodmap -e "remove Mod4 = Hyper_L"

Afterwards I tried assign Hyper_L to mod3

xmodmap -e "add Mod3 = Hyper_L"

However I got error as following:

X Error of failed request:  BadValue (integer parameter out of range for operation)
  Major opcode of failed request:  118 (X_SetModifierMapping)
  Value in failed request:  0x17
  Serial number of failed request:  11
  Current serial number in output stream:  11

So, what was the problem here? How I can make it work? Does my approach is right? Or I should use different approach in order to use mod3 for custom keys?

P.S. I am using Microsoft Natural Ergonomic 7000 keyboard

Answer 1

There are a bajillion answers on the internet, most of them confusing.

The key is you need to map 'keycode 66' to your hyper key, remove the mapping of that key from other mod{1,2,4,5}'s (only if present), and then set mod3.

Open ~/.Xmodmap and put the following:

! Unmap capslock
clear Lock
keycode 66 = Hyper_L

! Leave mod4 as windows key _only_
remove mod4 = Hyper_L

! Set mod3 to capslock
add mod3 = Hyper_L

Run this

xmodmap ~/.Xmodmap

If something goes wrong, you can reset your keyboard with:

setxkbmap -layout us

Likely causes of something exploding are that Hyper_L is mapped someplace else. Fix this by running xmodmap without args to figure out what else is bound to Hyper_L. When you know what you need to remove, chuck a 'remove BINDING = Hyper_L' in your .Xmodmap file.

The .Xmodmap file will be loaded in when you open X11 sessions, you should have this mapping in place next time you login or reboot.

This configuration works for my ubuntu 14.04 setup. If you have a different *nix and this doesn't work, please post what you did to get around it here and I'll incorporate it into the answer.

Answer 2

I Know this question is quite old, but I need to share my solution, since (frustratingly) heymantthew's answer didn't work for me. What ended up working for me:

setxkbmap -layout us -option lv3:caps_switch -variant intl

This will enable capslock as the mod3 key, set the layout to "us" and set the variant to "intl". If you use an interactive shell like fish, then you can press tab after "-layout ", "-option" or "-variant" to get a complete list of possible parameters.

Further background: I tried the xmodmap approach a year ago on a Ubuntu machine running i3 window manager, switching from the standard gnome environment. I had modified the "us" xkb keyboard layout, specifically the international one with dead keys– it worked fine when using gnome-desktop-manager (there is an option to set caps-lock as the mod3 key), but I couldn't get it to work with xmodmap. This was my mistake: I thought the gdm GUI was using xmodmap to map caps lock to mod3, but in reality it was setting this X11 option. Now that I switched to Arch I was forced to figure it out.

I recommend to anybody planning on messing with this to read the man pages on localectl, xmodmap, setxkbmap and testing layouts by adding a sleep and revert part:

setxkbmap ... ; sleep 10; setxkbmap <INSERT DEFAULT SETTINGS>

this saved me a good amount of time.

I still don't understand howcome the xmodmap approach does not work, however. Hope this helps someone.