You have got two problems here, apparently.
The REISUB sequence not working could be caused by magic SysRq not being activated - check out
# List of possible values in /proc/sys/kernel/sysrq:
# 0 - disable sysrq completely
# 1 - enable all functions of sysrq
# >1 - bitmask of allowed sysrq functions (see below for detailed function description):
# 2 - enable control of console logging level
# 4 - enable control of keyboard (SAK, unraw)
# 8 - enable debugging dumps of processes etc.
# 16 - enable sync command
# 32 - enable remount read-only
# 64 - enable signalling of processes (term, kill, oom-kill)
# 128 - allow reboot/poweroff
# 256 - allow nicing of all RT tasks
You want 4 (R) + 64 (E,I) + 16 (S) + 32 (U) + 128 (B) = 244 for classic REISUB (the effect is the same as the skinny elephant version; as a mnemonic, it's BUSIER spelled backwards).
For the swappiness of the system: it's working as expected. The cause of the behaviour is that you have lots of memory used - that could be the 95% you're getting - and then something else is requesting a whole lot of new RAM, forcing most of that 95% onto swap and fighting all the way.
The best thing you could do is install more RAM. Otherwise, see whether you can tame either the small processes building up to that 95%, or the big one eating a further 40-50%. The
top utility might help you there.
As a fancy solution you could increase apparent swappiness and keep the small fry "pared down" with some process that monitored the system and, if the big hog was not running and the memory footprint was too large, started allocating and releasing large swaths of memory - say, all remaining memory plus 64M, to force 64M of small fry onto the cache, then +128M, and so on, until the allocation delay grew beyond a given time threshold or the big hog started.
This answer could give you a good start if you really want to do that.