You cannot chroot into different architecture.
By chrooting, you are executing the binaries (from the chroot) on your architecture. Executing ARM binaries on x86 (and x86_64 in that matter) would lead to "Exec format error".
If you want to run binaries from different architecture you will need an Emulation, Qemu is a good candidate for this, but you will need to learn how to use it. This would involve creating RootFS and compiling a kernel for ARM, you will need a toolchain for compiling ARM binaries (and kernel) perhaps. One thing is for sure: Forget the chroot method, you cannot run binaries compiled for ARM on x86 (x86_64).
Edit:
After the small talk with @UrichDangel, I realized, it should be possible to enter the chroot environment with qemu-user programs (qemu-arm in this case).
Chroot should be executing qemu-arm compiled for your host architecture, then the qemu-arm can execute your /bin/sh (compiled for arm)
binfmtfirst, have a look at wiki.debian.org/QemuUserEmulation for a quiet short introduction. An example for configuring binfmt_misc can be found at svn.kju-app.org/trunk/qemu/qemu/qemu-binfmt-conf.sh – Ulrich Dangel Jun 28 '12 at 23:00-staticto the linker options – Ulrich Dangel Jun 28 '12 at 23:07