If I were you, I'd remove the retainer plate and inspect the other side where the cam nose contacts, and smooth the surface if its galled. It should show similar evidence of contact if the retainer is acting like the meat in a sandwich. If not, it would suggest that the cam was being pulled rearward by distributor, or pushed from front(wouldn't think so if there is enough timing cover clearance)?
You may want to remove the cam, and install the retainer between the gear and the cam for testing to see if there is adequate clearance. It may not be easy to turn the cam in the block and be certain that any resistance is from the retainer/gear/cam interference alone. Just a thought.