Oracle website offers 64-bit RPMs for its 11g versions. Do they offer 32-bit somewhere or does someone else do? Are there Debian binaries available?
It doesn't look like there are any. In my experience XE is only a half hearted attempt from Oracle.
About a year ago I asked an Oracle rep directly when 11g-XE would be available (only 10g was available for XE at the time). His answer was never. According to him the database is exactly the same, and the only thing 11g adds is "enterprise features" that would be disabled in XE anyway. He wouldn't go into specifics on what those exactly were, but mentioned Data Guard and RAC as two of them. I then asked him when they'd have 64-bit 10g then. His response was again never. Since XE is limited to using 1GB of RAM, he said, it wouldn't be able to take advantage of 64-bits. I didn't even know there were 11g-64bit packages out until I saw your question.
So apparently someone at Oracle has realized how stupid all of this is. But it still looks like a half hearted attempt to me. It looks like the best answer is use XE-10g for 32-bit and XE-11g for 64-bit.
As for Debian, Oracle still maintains an apt repository at http://oss.oracle.com/debian/, but it only includes the 32-bit 10g. I will be spending some of today debianizing those the 11g-64bit RPMs.