None whatsoever. I used 6mm long M3 socket head caps for the 2 Y and the X pulleys, as well as the acme rod pulley, and an 8mm long M3 for the Z motor pulley. All of them seated quite well, no issues with bottoming out. I had to use a 6mm for the acme pulley, because an 8mm was too long, it hit the screws that hold the Z motor plate to the Z extrusion.
I no longer feel anxious when cranking down the pulley set screws. Incredible peace of mind for very little $$