Noticing that one side of your cut seems good. Was it in a separate tool path or part of the one which is also producing the errors?
Aluminum milling with the XC is not impossible but it does take great care to get good results and frankly the stock mill does not do a very good job at it. The controller can be very buggy and you will need a chip clearance system (forced air works great). What size mill are you running? Milling aluminum on the 1000 or larger XCs is really a challenge due to how much flex they have.
What sort of end mill are you using? How many flutes and what brand? Are you seeing chatter in the areas where you are having issues or is it cutting fine but just out of place?
You can look here for a bit more info for aluminum milling overview: Production level aluminum milling with .8KW water cooled VFD - #86 by Travelphotog But get back with me and I will see if we can hep find the issue here and help you fix it.