I experienced the same annoying delay with xfdesktop right click menu after moving from Xfce 4.8 to 4.10 on my Athlon64 desktop running Debian Testing (Wheezy). I noticed that this new problem wasn't present on my Atom powered netbook which also runs Debian Testing and on which I'd also just moved from Xfce 4.8 to 4.10. As far as I remembered both machines are set up almost identically with the exception of different modules being loaded for the different hardware. I also noticed that some heavy apps like Firefox didn't open any quicker on the much more powerful desktop PC, and that starting the Xfce desktop UI was slower on the desktop. All slightly odd. A little digging revealed that my desktop was running preload, an adaptive readahead daemon that is suppose to monitor and analyse app usage and cache stuff that is often loaded, hence speeding it up at the cost of a small amount of RAM which would likely have been used anyway. I installed it maybe 4 years ago when running Debian Etch and recall it had a measurable benefit at the time (or I believed it did). Looking in its cache revealed it was referencing libs and apps that had not been on my system for years. I uninstalled it and on rebooting the latency problem with xfdesktop menu icons is gone, the machine boots maybe 10 seconds faster, the xfce environment starts faster and apps open at least as quickly.
I'm pretty sure I've used distributions which have preload installed by default, and it is definitely something often recommended as a responsiveness/performance tweak, usually accompanied by unsubstantiated and hyberbolic claims of effectiveness (awesome, drastic yada yada yada). Uninstall it.