I would like to open all chapters of my PDF ebooks in the same window but neither evince nor okular seems to be capable of doing that (at least not out-of-the-box).
Is there a way to work this out?
Tabbed provides a "simple generic tabbed fronted to xembed aware applications" and zathura is a simple PDF viewer that is XEmbed-aware.
A much more heavyweight approach would be letting a browser provide the tabbing while having the chapters displayed using plugins, e.g. using mozplugger (even with evince) or using PDF.js (both working with Firefox).
However, were you to use a window manager that supports tabbed layouts, e.g. i3, notion or XMonad (there might be more!), any lightweight PDF-viewer (xembed'able or not) could work just fine. You'd just open several windows in a tabbed window manager layout (details vary with the actual WM) -- like @Marco suggested in his comment.
qpdfview is a lightweight tabbed document viewer that should suit your needs.
It's in the default Ubuntu and Debian repos. More recent versions can be found in the following PPAs: