I had thought that with your gear bolted on, there was no clearance between the retainer plate and the back (thrust surface) of the cam gear. That would mean there was no "outside force" pushing the cam rearward, instead, there was a problem w/either the depth of the step on the nose of the cam (too short), or the thickness of the gear vis-a-vis the thrust face to the bottom of the bore that the stepped nose of the cam fits (too much), or the thickness of the plate (too thick).
In going over our previous conversation on this, I see you measured the cam retainer plate at 0.100", and the step on the cam nose at 0.260".
That leaves the step of the cam nose 0.160" proud of the retainer.
That means the depth of the cam gear has to be an amount less than 0.160" or there will be no clearance for the thrust surface of the gear to the retainer. I don't recall you giving the depth of the gear when we talked about this earlier.
Or to put it another way, w/the cam out and the retainer on the step and the cam gear bolted to the cam, the plate should be free to turn w/about 0.010"-0.015" clearance.
The face of the block doesn't matter as far as how far forward the face is, the cam will move to compensate because it's free to float fore/aft as needed (within reason, obviously). It should be square, however, and the wear pattern shows yours to be off a slight amount. But as long as there's some clearance between the gear and plate, the squareness being off a couple minutes really doesn't matter on a street build.