Why can't I resize or maximize Firefox 17 under Sawfish?

Question

I just upgraded from Firefox 16 to Firefox 17.0.1, and my Firefox windows lost their maximize button. The maximize-window command does nothing either. Switching to full screen mode in Firefox makes its menus and toolbars disappear, but the window decorations remain and the window remains the same size. Also, I can't resize the window to anything other than 18x108. All of this worked with Firefox 16.

I'm running Debian squeeze, with Firefox binaries from http://mozilla.org. My window manager is Sawfish 1.3.5. If I run an X session with Fvwm, I can maximize Firefox, so Firefox 17 is doing something that Sawfish doesn't like, but what?

Answer 1

You seems to hit a bug of sawfish:

https://bugs.launchpad.net/ubuntu/+source/sawfish/+bug/1083260

PS: And it is a bad idea to test sawfish with ubuntu repo, can't even login link.

Answer 2

window-maximizable-p returns false if the window has a maximum size hint that is smaller than the display. The maximum size hint is the max_width and max_height values in the XSizeHints structure returned as a response to a WM_SIZE_HINTS message.

Firefox 17 declares maximum heights and widths, with the value 0x40000000. In Sawfish, the window-size-hints function reports these values as 0. As it happens, Sawfish's Lisp implementation uses 2 tag bits on integer values, which leaves 30 bits on 32-bit machines and 62 bits on 64-bit machine. So that value of 2³⁰ is being silently truncated to 0.

I had a look at the C code, and the way it's written, this truncation happens even on a 64-bit machine (because the 32-bit XSizeHints member is shifted first, then promoted to long).

As John Siu found, this is Ubuntu bug #1083260 which was fixed after being noticed in relation to Firefox 17 with a discussion in the mailing list.

---

Now for a workaround you can put in your .sawfishrc, at least to make Firefox 17 work.

Given that a maximum value of 0 doesn't make sense, it's possible to filter this value in Lisp and remove the hint when it seems to be 0. This only solves the issue for maximum size hints that are multiples of 2³⁰, but that's good enough here.

Warning: the code is pretty horrible — sawfish doesn't like patching subroutine-only modules (you can't do much without rep in the namespace — including (require 'rep)).

(let ((struct (get-structure 'sawfish.wm.windows.subrs)))
  (unless (structure-bound-p struct 'window-size-hints-before-max-truncation-fix)
    (let ((old-window-size-hints (eval 'window-size-hints struct)))
      (structure-define struct 'window-size-hints-before-max-truncation-fix
                        old-window-size-hints)
      (structure-set struct 'window-size-hints
                     (lambda (#!rest args)
                       (let* ((hints (apply old-window-size-hints args))
                              (cell (cons nil hints)))
                         (mapc (lambda (key)
                                 (let ((cell (assq key hints)))
                                   (if (zerop (cdr cell))
                                       (setq hints (delq cell hints)))))
                               '(max-height max-width))
                         hints))))))