You cannot chroot into different architecture.
By chrooting, you are executing the binaries (from the chroot) on your architecture. Executing ARM binaries on x86 (and x86_64 in that matter) would lead to "Exec format error".
If you want to run binaries from different architecture you will need an Emulation, Qemu is a good candidate for this, but you will need to learn how to use it. This would involve creating RootFS and compiling a kernel for ARM, you will need a toolchain for compiling ARM binaries (and kernel) perhaps. One thing is for sure: Forget the chroot method, you cannot run binaries compiled for ARM on x86 (x86_64).
After the small talk with @UrichDangel, I realized, it should be possible to enter the chroot environment with qemu-user programs (qemu-arm in this case).
Chroot should be executing qemu-arm compiled for your host architecture, then the qemu-arm can execute your /bin/sh (compiled for arm)