Unfortunately, it's really hard; rpm isn't designed to go backwards. If you want to try, I think this should do it:
sudo dnf --releasever=23 --allowerasing downgrade fedora-release
sudo dnf distro-sync
Note that this is dangerous — the
--allowerasing option definitely allows you to shoot yourself in the foot, and doing a distro-sync that large just plain might not work.
I might instead consider just staying on Rawhide through the Fedora 23 cycle. We (disclaimer — I work on Fedora!) occasionally cause some problems there, but if you follow blogs like Kevin Fenzi's This Week in Rawhide you can generally be aware when they're likely to happen and be careful with updates.