If you are going to sand down to the metal, I would use 180. If you use the 180 you will need to reprime with high fill primer and re block everything.
I would DA with 320 and sand all the dirt and contaminates off the body. Then apply a couple or three coats of High fill primer and block sand again with 400 wet. If your top coat is a metallic, sand with 600 wet. If you get down to the metal in a few places, no big deal. After sanding and everything is straight, apply a coat of Epoxy Sealer and you will be ready for your top coat.