You are exactly correct, the down spiral drives the chips into the work piece. However, in most cases you are machining along a path and the chips can escape into the path the bit has left behind.
If you were to use an up-cut bit there could be a few undesirable consequences. Since these work essentially like a drill bit, the act of milling pulls down on the bit and pulls up on the stock. If your bit is a little loose in the collet (or if you thought that your 1/4" collet would actually hold a 6mm bit; I’m talking about me here) that bit might start slipping downward and cut deeper into your stock as your milling progresses. Another thing that could happen is that thin stock held only around its edges might pull up in the center as you mill and this would again cause your cuts to be too deep in the center. Double sided tape or some pin nails might stop this by holding all of the work piece down.
There are other benefits as noted by the other members above. The down-cut bits will leave a cleaner edge on plywoods. There are also compression bits which are an up-cut at the tip and a down cut near the collet. These are expensive but produce nice results.
So are up-cut bits bad? No. I use the all the time. They are inexpensive and with solid stock they efficiently clear away a lot of material.