When you use a down spiral its cutting action causes the bit to push away from the bottom of the work piece. If you have any flex in your machine or work piece, this could cause the bottom of your rough cut to be not as deep as it nominally would be.
You should perhaps run some experiments with sample sections of your larger work. You could try a straight flute or an up cut and see if that makes a difference.
